bf_imagevignette_bf_imageecrire_FonnaTacha_Flickr_CCbySA.jpg

How to produce a document when you are several hundred persons (Part 1)

Card's author : Jean-Michel Cornu
Card's type of licence : Creative Commons BY-SA
Description :

How to turn armchair philosophy into collective intelligence ?

parable of blind men and an elephant 1

It was six men of Indostan to learning much inclined, who went to see the Elephant (Though all of them were blind), that each by observation might satisfy his mind. The First approached the Elephant, and happening to fall against his broad and sturdy side, at once began to bawl: "God bless me!—but the Elephant Is very like a wall! " The Second, feeling of the tusk, cried: "Ho!—what have we here so very round and smooth and sharp? To me 't is mighty clear this wonder of an Elephant is very like a spear! " The Third approached the animal, and happening to take the squirming trunk within his hands, thus boldly up and spake "I see, " quoth he, "the Elephant is very like a snake! " The Fourth reached out his eager hand, and felt about the knee. "What most this wondrous beast is like Is mighty plain, " quoth he; "'T is clear enough the Elephant is very like a tree! " The Fifth, who chanced to touch the ear, said: "E'en the blindest man can tell what this resembles most; Deny the fact who can, this marvel of an Elephant is very like a fan! " The Sixth no sooner had begun about the beast to grope, than, seizing on the swinging tail that fell within his scope, "I see, " quoth he, "the Elephant is very like a rope! " And so these men of Indostan disputed loud and long, each in his own opinion exceeding stiff and strong, though each was partly in the right, and all were in the wrong!

From armchair philosophy 2 ...

We usually seize that if an idea is true then its contrary is false. It is called the law of non-contradiction upon which our logic, as defined by Aristotle, is based. Therefore Eubulide of Millet, who fighted this law, has demonstrated through to the liar's paradox 3 that it was not necessarily true : "A man was saying he was lying. Was this true or false ? ". This sentence cannot be either true... nor false! As well as in the parable of the elephant, some assertions may sound in contradiction but all of them or true 4 . We then talk of antinomy. It's especially the case when we try to have several different points of view on a subject.
Equipped with the law of non-contradiction, we don't spend time searching what is true and what is false but justifying what we have said formerly... and therefore demonstrating that people with different arguments are wrong. Each member spends most of the discussion's time repeating and justifying his own assertion to be sure it will be taken into account. Very often the background of the debate is not about searching the truth but about avoiding being caught out and possibly gaining recognition from other members for having said something seen as true.

...To collective intelligence

To get out of armchair philosophy, it's compulsory first not to look for what is true on a topic but the different points of view on a it. The more people with a point of view, the more complete the view. At this stage, the debate can cope with approximative, not to say apparently false views. The aim is to gather the biggest number of points of view and to create new ones to complete those already found.
But we also have to compromise with our own cognitive limits. Thus, we can only remember the three last elements of a discussion 5 . When we look at the discussion objectively, we can have an overview of the differents assertions or arguments, but there too we are limited and can only remember from 5 to 9 ideas 6 . In order to deal with a subject using collective intelligence, we will then have to apply a method to work with a great number of persons, to map the whole of the ideas but stopping ourselves, first to select some ideas and eliminate others.

The three principles of ideas-co-building

Managing a lifting of collective discussion's difficulties needs to take into account three principles of collective intelligence which are rather counterintuitive but which will be the bases for the construction of a method allowing the production of ideas and contents from several hundred persons.

Size of the group and parts of members 7

As soon as a group is over a dozen people, each member adopts a proactive, observer or idle position and can move from one to another according to a number of criteria. Rather counterintuitively, it is observed that the percentage of active persons stays remarkably constant (principle of 90-9-1) : proactive people are one and over percent and reactive people are between ten and a few dozens of percent.
Different types of groups differing from the number of members :
Small groups up to twelve people whom can be managed restrictively (expecting the action particular to each different member) ;
Intermediates groups between twelve and a hundred people whom need more efforts from the manager to get reactions ;
Large groups between a hundred and one or two thousand people which allows to produce collaboratively... provided a focus is done on reactive persons ;
Very large intermediate groups of several thousand people where the group composed by proactive people becomes easily illogical ;
Very large groups over several dozens of thousand where proactive people are numerous enough to make management less restrictive ;
Large groups between a hundred and one or two thousand people are particularly interesting : they are a prerequisite for groups doomed to become very large, and moreover they represent a good corresponding size to the number of persons which can be gathered on many rather sharp topics. However reactive people need to be taken into account (they can be reached through online systems by push-tools such as email, Facebook or Twitter rather than pull tools as the web or forums) and not only proactive people whom in this case are not enough.

The post factum choice 8

There are several strategies according to the environment around us :
Planning : in a predictable situation where resources are scarce, prediction is needed to optimize them and avoid their spoiling ;
Negotiation: when ressources are scarce but the situation not predictable, negotiation enables a choice in real time since it could not be done in advance ;
The post factum choice: when ressources are abundant (large group, abundant information) but when the situation is not predictable, it is better then to arouse an abundancy of choices and to choose only post factum, within all possibilities ;
Most of the time we do not choose our strategy but we use the one we know better whatever the context. We need to adapt ourselves to our environment in order to choose the best strategy. Sometimes a situation can be predictable for some things and unpredictable for others, some ressources can be abundant, some rare. In this case, we need to adapt ourselves and even to juggle with the strategies.
For instance in a large group of over a hundred persons, thanks to the number of reactive members acting it is possible to arouse the maximum points of view and to choose only afterward those to keep : "Whith enough observers, all appliable solutions to a problem are blindingly obvious ". But if the group is smaller than one or two thousand people, the number of proactive members and moreover the number of persons who join the coordination is weak. The coordination of groups under several thousands must call on planning and/or negotiation strategies.

Mapping for an overview 9

In a debate with several people, and even more in a confrontation, each one tends to defend his idea, to repeat it constantly so it is taken into account. In practice it's often seen that different points of view don 't rule each other out but on the contrary complement each other to give altogether an overview. To go past the facts, me must take into account the two ways of thinking that are each using a different working memory.

The first, based on speech consist in sayings ideas one after the other, just as we make a step after another to progress from a starting point until an arrival. This way of thinking especially allows a rational approach but it hardly takes into account conflict (a starting point, two directions), collective intelligence (several points of view on the same arrival) or else creativity (finding new ways between several starting points and several arrivals) which are all three using another complementary way.

The second way of thinking is based on mapping. It consists in arranging on the same mind map ideas according to their proximity, without trying to select them offhand, to get the more complete vision on ideas and possible progressions. Mind maps (mind mapping in english) which are co-built and projected to all during sessions are very efficient to give a global vision to the whole group and allow therefore to look for new ideas and new points of view rather than having each member focusing on one or two former ideas.

To go further, two possible approaches :
The Method of Loci : During synchronous meetings (online or face-to-face), a map of idea can be coupled with another map, often of territories that each one can keep in his long term memory. It can be a place known to all (for monks in the Middlle Ages, their cathedral) or if this can't be found, a co-built place (in the long term a place is easier to remember than ideas) ;
Textual maps : in asynchronous online exchanges, people who behave reactively (ten times more numerous than proactive people) and the observers (even more numerous) use tools which cannot stand graphics mode very well (email, Facebook, Twitter). Therefore proposing a drawn map needs to share a link to a web page where the map is. But then only half of participants will see the map. The possible use of text laying-out can then be used to allow the drawing of a textual map which won't need to be read in its whole as a text but can be read as a map : lists of bullet points, formulation of short ideas in one line maximum, bold, underlined and italics to enhance some keywords ;

Applying these principles to produce collective intelligence

From the principles presented in the previous parts, we can start to edit some rules to allow the production of ideas and contents with several hundred people. We will deal here especially with online asynchronous exchanges which can be punctually improved by face-to-face or online synchronous meetings.
1. The group must have at least a hundred members.
These won't contribute systematically as in a small group. As long as the group stays under several dozens of thousand people, it's important to focus on the members with a reactive attitude (it is the most common size of group. Even in very large groups of dozens thousand people, only a sub group will take an interest in a specific content). According to the 90-9-1 rule, reactive people will be at least a dozen which will be enough to start a dynamic and possibly encourage other participation.
2. The critical part of facilitator(s).
By definition facilitators do have to be proactive. But within a group of between a hundred and one or two thousand people, they are few. Mistakes or lack of proactivity from a facilitator can lead the whole group to inaction. In a young group (generally less than two years old), the facilitator or the small group of facilitators has a central part. It is even in the freeware Benevolent Dictator for Life. For maturer group, different people can, according to topics, have a leading part. In this case, eventhough animating the discussion is a restrictive part, it is not as much for the whole of the group which may have discussions leading to an achievement or not.
3. First, let people express ideas without choosing.
On the contrary, it is necessary "to open up the fields of possibilities " to point out all ideas that could be added, rather that suppressing those already spoken. Ideas seeming out of hand may happen to be very rich although "a priori " counterintuitive. Even if an idea turns out to be stupid, it can spark off other interesting ones.
4. A map shaped summary gives an overview of ideas exchanges.
In the case of online asynchronous ideas exchanges, it's better to use mind map which anyone can receive. It does not need to be wholy read as a text would, but can be glanced through like a map (with bullet lists, bold and underline laying out...) That point requires the more work. Tools and methods enable to reduce this time.
5.At least a few items of information need to be "pushed ".
To reach reactive people, some items of information need to pushed (information is sent directly to an account that he persons read regularly : email, Facebook ou Twitter). But according to the number of members, the energy of the discussion and the more or less great agreement from members to receive information directly, there is also a need to give a wide access to the whole of information with pull tools (the person fetches by herself the information by visiting a forum, browsing archives or other webpages). A fair balance is then to be found between what is sent to all and what is not sent but has to be looked for by those who want (from the mailing list where everything is received by everyone to the sending of summaries only, including the extra sending of a selection of some stimulating contributions encouraging readers to react).
6.Iterations of contributions/summaries contribute to collective intelligence 10 .
The mapping of different points of view allows to have an overview (as the parable of blind men and an elephant). But collective intelligence really starts when participants lean on what others have said (or to be sharper on the global map of the discussion) to propose new ideas that they would not have had otherwise. Thus each contribution increases the level of collective intelligence and enables proposals, some particularly innovating and smart.

Method to produce a collective text when you are up to several thousand participants

This method aims to produce content collaboratively, not only by including former contributions but also and most of all contributions resulting from exchanges of ideas. It is based on regular text map summaries (a text that can be read as a map rather than scoured, with bullet lists, bold an underline laying out to enhance some words, etc.).
This methods is concentrated on "large online groups ", large enough to obtain reactions without to much effort (a hundred or more members) but still not reaching the needed size to allow a concentration on proactive people only (over several thousand). It is a major part of online groups wishing to produce contents on a specific topic. In this case the stress is the most reactive persons whom are generally ten times more numerous than those with a proactive attitude.
The two parts below concentrate on tools and how to make the group up for those who are creating groups or those whose groups are still too small. The next part on web watch, common understanding and ideation is the heart of the method to build a structured overview of collective ideas. The two last parts on ideas selection and writing enables to have a text easy-to-read for people whom did not take part in the discussion or did not know the topic very well.

Implementation of tools

Tools for discussion
The first stage is to choose push (information is brought to participant : email, Facebook, Twitter...) and pull (the participant seeks for information : forum, webpages..;) tools. For a rather small group of up to several hundred people whom are all using emails, a simple mailing list is enough. The records of the list enable proactive people to seek for old items of information and eases the facilitator's job in charge of mapping.
More and more often, participants read regularly their messages with different tools : some are on Facebook and scarcely read their emails, some follow Twitter but have deserted Facebook. Some only use one of these three tools, sometimes two but rarely all of them. Other groups use a general social network (Linkedin, Viadeo) or a network particular to their community (based on softwares Elgg, Diaspora, Movim, Daisychain...). There is therefore a necessity for : either keeping up with all the different tools used by the group's members or... a reduction of the group to the members whom only use such or such tool.
More over, when a group gets bigger, the number of contributions grows too and can overtake the bearable level for a participant. In an online world where post people suffer from "infobesity " (too much information), even in a relatively small group, some can be annoyed by emails coming from the discussion. To prevent the cancellation of subscriptions or disaffections (emails automatically filed without reading, not to say tagged as spam...), only the most important information is to be sent to all or to those who want : regular mapping of the discussions, a selection of some contributions gathered in the same message to stimulate partipation, etc. In that case it is even more important that the whole of the contributions should be available (by pull way) to allow those who wish, and of course to facilitators who make maps, to find the detailed contributions. It is thus by allying push and pull tools that discussion will allow the sending of some messages to everyone (to reach the reactive people) but keeping the amount of messages at a reasonable level (to avoid over-information).
Find more about the subject : the Fing, link between email and social network under Elgg 11
Since 2010 and after the testing of quite a few online tools for its collaborative works (mailing lists, blogs, forums), the Fing has progressively implemented its social network, an Elgg platform which enables to standardize its contibutors collaborative network environments : some members being involved in several discussions, managing subscriptions to scattered platforms and mailing lists was indeed a problem.
At first, the choice was made to combine the web platform (to publish) and the email (to exchange). At the launching of Digital questions in mid 2012 , the Fing chose to interface the two modes. Each forum of its network allows web or email interaction : for example, a forum subject is posted on the web and notified by email to the 260 persons of the Digital Questions group, who can react either by return of email or by logging in on the platform. Users seem to choose email for quick answers and web when elaborated answers are needed.
This practical detail also allows to have, like on any forum, several parallel threads of discussion, provided a special care is given to titles. It eases access to new comers and open-cast work and lowers the entrance barriers. Activated on forums, this feature can also be easily activated on comments from other publications : blogs, document-sharing, events...


Get to know more : ADEO group, use of Google groups in push and in pull 12
The ADEO group is a firm of 70000 persons dispatched in 13 countries and 27 Business Units (BU). Very much decentralized, turned towards the sharing of Knowledge and Power, ADEO has launched for nearly 20 years into numerous steps of shared Vision with all the staff of some of its BU.
The Communauté Produit, Achat et Supply-Chain (PAS), grouping together the central buying services, the logistic departments of the BU and the PAS Group's Direction has initiated in mid-2011 a transverse step : VisionPAS 2023, the Vision on cooperation PAS of the ADEO Group by associating thus more than 2000 staff. Different collaborative techniques have been used to extract the true substance : work units, creativity seminar, Design Thinking mode prototypes, … but none of these involving more than 150 persons altogether.
In order to do the writing of our target in 10 years, we have decided to divide in 8 great main lines according to the following structure : Benchmark, TOFW (Threats, Opportunities, Forces, Weaknesses), 10 years vision. Nearly 50 work units of 15 persons which have allowed us to make up this very rich and complementary material (see : parable of blind men and an elephant) . We have then realized a first summary on each topic. The importance was then to find ways to make the whole community react on this VO to make the most of collective intelligence. But very soon in an international group with no reference language, the linguistic issue aroused. We didn't have either CRM tools, enriched repertories nor a firm social network. In consequence we have implemented a 6 weeks Digital Debate targetting 1500 persons helped by Google Groups.
The need :
  • Multilingual Forum to favour individual speech.
  • Possibility to send mass emails from the Forum towards mailboxes with an option to answer directly on the Forum without having to join (this criterion made us exclude the Nabble tool that does not allow mass-emails sending)
The solution :
  • 7 Forums = 7 Google Groups (1 per language: french, english, spanish, italian, polish, portuguese, russian) bringing in a group of translators
  • One week of intensive processing on a topic with different and coherent push tools according to
  • 1. Launching of the debate by the sending of a summary on the topic
  • 2. Sending of inspiration on the same topic : open-mindedness, proposal of external perspectives
  • 3. Publication of the latest contributions : the message we want to send is : "the debate is progressing, your colleagues are taking part in it, new ideas are rising, join in ! "
  • 4. Publication of a new summary enhanced by the debate : contributors recognize their hand in the wording of the final deliverable and notice the enhancement of the final summary thanks to the collective debate.
  • A simple system to contribute : answer by email which feeds instantaneously the thread of the forum OR direct contribution on the forum by posting a comment. On the forum, contributions on a topic can be seen indistinctly.
Strong points :
  • Strategic topics tackled in 7 languages : richness of contributions made easier by individual expression.
  • Volunteers within the company for the translation helped to make translations reactive and flexible, which was vital to stick to our rather short deadlines.
  • No hierarchical diagram : all ideas are kept and exploited similarly in the final wording of the final deliverable. Besides, contributions put forward in Flash emails onlyy quote the contributor's surname, not his/her name.
push by daily email : contributor's sollicitation through the media they use most today. Contributions are for stocked in one and same place : the Goggle Group (1 per language). Each person "must " receive information but she is free afterwards to follow or not the thread on an additional tool, here the Google Group. In order to avoid missing the "best " contributions, we "push " to all a selection of these latest.
Difficulties :
  • A forum per language but no transversality between the 7 forums : what is posted in the polish forum cannot be seen by Spanish. EXCEPT that the dissemination of "best comments " in the flash could come from the 7 forums and summaries were common in all languages.
  • Only the coordinator had subscribed to Google Group so that participants would receive summaries but not all contributions. With no CRM, sendings were done from a Gmail account wit a return adredd that was the Gmail account's o ne.Even the Gmail account was opened so that participants could join in if they wanted.
  • Need to have a Gmail account to have access to Google Groups.
  • Need of fitting tools (enlarge repertories, CRM, …) for that sending volume.
  • The not-always-easy-to-implement need for accomplices enabling the bustling of debates.
Results and figures :
  • A 6 weeks live debate on 8 strategic topics.
  • A participation rate of around 13% with more than 400 multilingual commentaries multi-lingues which have enriched the Vision notebooks.
  • Contributions done in 7 languages : "only " 55% of commentaries are in French.
  • 8 input "Vision V1" notebooks of our International Meeting which gathered in february 2013 for 3 days 700 PAS community managers of the Groupe ADEO to do a collective reading 13 .
To conclude, this first ADEO's large scale Digital Debate was rich in learnings. It allowed us to follow the major stages referred to in the paragraph "Applying these principles to produce collective intelligence ". It allowed us to validate this participative method and will surely call out for more.

Tools for capture and text maps
To create a text-map summary which enables the group to have an overview, interesting contributions need first to be captured within the different messages (there can several in the same message), eventually given a shorter (less than a line) and more explicit name and then organized into a hierarchy. This last action may need to create new entries in the hierarchy to gather several ideas which can be found there.

Find more about it : reorganize levels while discussing
Let's imagine a discussion on the implementation of this method where the current vision is described by the following text-map :
  • Tools of discussion
  • Mail (push tool : information sent directly to participants)
  • Take into account those who like Facebook rather than email
  • Forum (Pull tool : the participant fetches himself the item of information he wants)
Contributors propose to add the idea of also using Twitter as well as other social networks. The map could then be reorganized under the following shape :
Tools of discussion
  • Pushtools (information sent directly to participants)
  • Mail
  • Facebook
  • Twitter
  • Other social networks
  • Allowing several push tools to leave a choice to participants ?
  • Pull tools (the participant fetches himself the item of information he wants)
  • Forum
  • Something else ?
In this case, not only the idea of "email " moves to the level of "push tool " which includes Facebook, Twitter and other social networks, but the person who does the mapping had the idea to add the possibility of mixing tools and also organized identically the pull tools to leave room for other choices. In doing that, we do not have exactly a summary of the discussion but rather a map of the current vision understanding of the problem. Reorganizing a map often gives additional ideas and even the map-maker can add ideas, which can be completed or corrected by participants during the next iteration of contributions.

The mapping of exchanges can be done by hand with eventually post-it notes on a wall in order to reorganize ideas easily. But when the discussion is important, one iteration of the map can take about 5 hours and this happens once or twice a week during the phase of ideation... organizing such a discussion is greedy in time for facilitators and particularly for those who make or complete maps.
Too reduce the mapping time and therefore allow the animation of groups even by persons for whom it's not the "official job 14 ", time must be reduced to one or maximum two hours a week. The aim of the app Assembl developed by Imagination for People in partnership with the Institut du Nouveau Monde in Québec, is to ease the capture of smart contributions, to help renaming them and to reorganize them easily despite the small size of a computer screen.

Find more about it : Assembl a tool to map contributions 15
Assembl is an online discussion system aimed for groups of people which have to produce collectively a deliverable (opinion, consensus, document, patterns, alternatives, etc.) on any subject. Although it is relatively easy nowadays to mobilize people on a stake through social networks, for a multitude of purposes the quality of a deliverable does not increase the number of participants. This is the main problem tackled by Assembl.
First of all by combining a chronological discussion (necessary to ease implication, feeling of membership and group dynamics) and a more structured and synthesized presentation of the discussion (necessary to enable each participant to have an overview on exchanges and proposals whatever the time and the level of attention he can devote to it).
Assembl enables humans to play a facilitator part in a team. With the help of tools making these tasks productive, they point out key-ideas, disseminate them synthetically and guide participants towards constructive discussions.
Assembl tries not to repeat what we see as weaknesses of preceding systems, thus, Assembl:
  • Doesn't force participants to write their contributions in a special format (the structure must help the discussion, not take its place)
  • Acknowledges that some participants will like better a push mode (for example: mailing lists) and some a pull mode (for example : : web forums web, Facebook groups), and enables them to chat together by breaking those "havens " of discusion 16 .
  • Doesn't brake existing communities by forcing migrations. It can be introduced progressively in the current mailing list of an already active community.
  • Doesn't disconnect maps of discussions that gave birth to it. Reactions to discussion are available from the global version and vice-versa.
  • Doesn't make obligatory a structure of discussion (numerous systems are focused on for/against debates) and imposes less constraints to methods of animation.

Continuation of this text available here

  • 1 parable of Jaïnism, made famous by the american poet John Godfrey Saxe in the middle of the XIXth centuray. Source :
Sanskrit Heritage Dictionary. [online]. [Accessed 4 February 2014]. Available from : http://sanskrit.inria.fr/DICO/index.html, Quoted par Wikipédia : The Sanskrit Heritage Dictionary. Wikipédia [online]. [Accessed 4 February 2014]. Available from : http://fr.wikipedia.org/wiki/The_Sanskrit_Heritage_Dictionary
  • 2 The equivalent expression in english could be "bar-room politics " or even better "armchair philosophy " refering to cultured but idle people , whom talk a lot but act little (rather than to people who would have had too much too drink and would talk nonsense) : café du commerce. WordReference? [online]. [Accessed 4 February 2014]. Available from : http://forum.wordreference.com/showthread.php?t=70335&highlight=comptoir
  • 3 A paradox which might have been invented by Eubulide of Millet (IVth century) from the Cretan of Epimenide. Paradoxe du menteur. Wikipédia [online]. [Accessed 4 February 2014]. Available from : http://fr.wikipedia.org/wiki/Paradoxe_du_menteur
  • 4 Alfred Korzybski, author of General Semantics, understood during the first World War that the mechanisms of thinking which caused the war rested on the postulates of Aristotle's logic (principle of identity, of non contradiction and of excluded middle). He expressed then a new non-aristototelitian logic based on new postulates corresponding to scientific advancements in the XXth century : Sémantique générale. Wikipédia [online]. [Accessed 4 February 2014]. Available from : http://fr.wikipedia.org/wiki/S%C3%A9mantique_g%C3%A9n%C3%A9rale
  • 5 It 's about a limitation of one of our working memories called phonologiacl loop, which only allows us to keep in mind three items in a chain of ideas. For the model of the different working memories, see : BADDELEY, Alan D. and HITCH, G. J. Working memory. In : BOWER, G. H. (ed.), The psychology of learning and motivation : Advances in research and theory Volume 8. New York : Academic Press, 1974. p. 47–90. ISBN 9780080863597 0080863590 0125433085 9780125433082.
  • 6 This second working memory concerns the whole of the project or ideas we can remember in our short term memory. It's named after a visuo-spatial notebook. It enables us for example to count post factum the windows in a house when we don't see it anymore... considering their number is limited. It's also this same working memory which allows to create new ideas by linking two former ones. MILLER, George A. The magical number seven, plus or minus two : some limits on our capacity for processing information. Psychological review. 1956. Vol. 63, no. 2, p. 81.
  • 7 To know more about the topic, see the complete text: "Size of groups and roles of members"
  • 8 To know more about the topic, see the complete text: "//Post factum// choice"?
  • 9 To know more about the topic, see the complete text: "Mapping to give an overview". These ideas were presented formerly in : CORNU, Jean-Michel. Modes de pensée et conflit d’intérêt. In : Nouvelles technologies, nouvelles pensées ? [online]. Limoges, France : FYP éditions, 2008. Innovation, ISSN 1961-8328. ISBN 978-2-916571-03-4. Available from : http://www.cornu.eu.org/files//ProspecTIC_pensee2.pdf
  • 10 See also the Delphi method which enables aware people to improve their forecasting on a topic by an iterative approach enhancing the fields of convergence and incertainty : LINSTONE, Harold A. and TUROFF, Murray (eds.). The delphi method [online]. Addison-Wesley Reading, MA, 2002. [Accessed 4 February 2014]. ISBN 0 - 201 - 04294 - 0. Available from : http://is.njit.edu/pubs/delphibook/#toc
  • 11 This part was written by Jacques-François Marchandise de la Fing
  • 12 This part was written by Victoria Masson and Jean Duclos from the ADEO group
  • 13 See the paragraph "Text Wording : collectivre re-reading "
  • 14 The animation of a debate may be done by volunteers or professionals whom will find interesting to be in the heart of the discussion in order to catch ideas and subtelties fully. This job of animation should not officially be part of the working time.
  • 15 This part was written by Benoît Grégoire from Imagination for People. Imagination for People | Repérer et soutenir des projets sociaux créatifs. [online]. [Accessed 4 February 2014]. Available from : http://imaginationforpeople.org/fr/
  • 16 See the paragraph "Size of groups and roles of members" about the difference between proactive and reactive participants
Keywords :