Effectiveness of Brainwriting Techniques: Comparing Nominal Groups to Real Teams 169 4 Metrics For the nominal group data, only quantity and quality were measured since only these two metrics showed few differences between the idea generation methods. The same process as before was used (Linsey, et al., accepted). A new evaluator scored the quantity and quality data for the nominal groups. Prior to scoring the nominal groups, the evaluator was trained on two teams’ results and then two additional teams’ were scored by the evaluator to determine inter-rater agreement. Inter-rater agreement for quantity was 92% with a Pearson’s correlation of 0.91. This indicates there is strong agreement between the two evaluators. 5 Results and Discussion Interacting groups with appropriate idea generation methods can be more effective than nominal groups. The results show that real teams in rotational conditions develop a larger number of ideas than equivalent nominal groups (Figure 4). This result is consistent with the theory that one of the reasons for the observed productivity losses in real interacting groups as compared to nominal groups is due to production blocking (Mullen, et al., 1991; Nijstad and Stroebe, 1999). Production blocking occurs is when one team member is talking (producing ideas) and other teams members are listening. This causes them not to produce ideas. This result is also consistent with other hypothesized reasons for the productivity loss includging performance matching (individuals see how much their teammates are producing and adjust their productivity to match), and evaluation aprehension (Mullen, et al., 1991; Nijstad and Stroebe, 1999). A clear interaction effective is observed in Figure 5 through the non-parallel lines. An ANOVA shows that there is a statistical interaction between the viewing condition and the representation meaning that both are statistically important and the effect of the viewing condition depends on what representation is used [Viewing Condition: F(1,48)=2.2, p=0.15, Representation: F(2,48)=26.3, p<0.001, Interaction: F(2,48)=9.0 p<0.01 and MSerror=30.7]. The representation implemented does not affect the nominal groups (individual idea generation), but has a substantial impact for the real groups. In real groups, the representation effects the communication between group members, whereas with individuals, the representation mainly serves to externalize internal ideas. The statistical analysis in this paper does not include data from any of the sketches only conditions because the prior study (Linsey, et al., accepted) indicates that the results from sketches only conditions are likely significantly affected by the fact that US mechanical engineers are typically not taught to free- hand draw. So only the data from Words Only and Word & Sketches is analyzed and compared. To maximize the number of ideas a team generates, a team should use annotated sketches to communicate their ideas. A hybrid 6-3-5/C-Sketch method that includes rotational viewing should be implemented. For an individual working alone, it does not matter what representations is used. Fig. 5. Interaction effects between words only and words & sketches representations and the viewing condition 5.1 Quality The representation has no effect on quality (Figure 6) or the distribution of quality (Figure 7) for the individual idea generation (nominal groups). This is not particularly surprising since the individuals are not communicating their ideas to anyone else and the quality scale is rather coarse. If the quality scale were finer, it might indicate differences between the representations. Fig. 4. Average number of ideas per team. Error bars are +/- one standard error 170 J. Linsey and B. Becker The various conditions do have some effect on the quality of the ideas generated and the quality distribution (Figure 6 - 8). The prior study (Linsey, et al., accepted) did indicate that sketches only conditions tended to produce both higher quality ideas on average and fewer low quality ideas (Figure 6 and Figure 7), but this was likely due to the fact that many low quality ideas like “chemically removing the peanut shell” or “genetically engineering a peanut without a shell” are difficult to draw and therefore would have not been included by the participants. Fig. 6. Quality results. Each error bar is +/- one standard error 0 5 10 15 20 25 30 35 Number in Category Condition and Team Quality Score Distribution # of 2s # of 1s # of 0s 1a 1b 2a 2b 3a 3b 4a 4b 5a 5b 6a 6b Fig. 7. Distribution of team quality scores (Quality Scores 1=technically feasible, 2=feasible for the context) The quality results indicate that words only should not be used for a large number of quality ideas in a team setting. The viewing condition (gallery verse rotational) had little effect on the average quality or the distribution. These results indicate that when teams implement an effective method for idea generation, they can outperform the combined results of individuals (nominal groups). This result is in contrasted to results from Osborn’s Brainstorming method where nominal groups generally outperform real teams (Mullen, et al., 1991). 6 Conclusions Brainwriting techniques that include a combination of sketches with annotations, such C-Sketch or Gallery, can assist a team in creating more ideas than the combined efforts of the same number of individuals working alone with redundant ideas removed, referred to as “nominal groups”. In contrast to this, prior experimental results from other studies on Osborn’s Brainstorming show that interacting groups are less effective than nominal groups. These results indicate that designers should carefully select their group idea generation approach in order to obtain a successful process. To maximize the impact of a group idea generation, teams should sketch their ideas and add annotations to enhance interpretation. Methods where individuals can all simultaneously work as opposed to methods where one person speaks at a time (e.g. Osborn’s Brainstorming), will produce a greater number of ideas. A hybrid 6-2-5/C-Sketch method, where teams sketch adding annotations and then rotate ideas, is best for group idea generation. This study compared nominal groups to real groups using techniques very similar to 6-3-5, C-Sketch and Gallery. Nominal groups were compared to real groups in a 3X3 factorial experiment. The first factor was how the teams represented their ideas (words only, sketches only or words & sketches) and the second factor controlled how ideas were exchanged (rotational viewing, gallery style, or no exchange-nominal groups). This factorial design leads to teams generating ideas in conditions very similar to 6-3-5, C- Sketch and Gallery. It was found that real teams using rotational viewing (e.g., 6-3-5, C-Sketch) created a greater number of ideas as compared to nominal groups. In contrast to this, real teams using gallery viewing produced significantly fewer ideas than the nominal groups. Fig. 8. Quality score distribution for individual idea generation (combined to form nominal groups) Effectiveness of Brainwriting Techniques: Comparing Nominal Groups to Real Teams 171 References Adams JL, (1986) Conceptual Blockbusting. Cambridge, MA: Perseus Books Aiken M, Vanjani M, Paolillo J, (1996) A comparison of two electronic idea generation techniques. Information & Management 30: 91–99 Baxter M, (1995), Product Design: A practical guide to systematic methods of new product development. London: Chapman & Hall Gryskiewicz SS, (1988) Trial by Fire in an Industrial Setting: A Practical Evaluation of Three Creative Problem- Solving Techniques. In Innovation: A Cross- Disciplinary Perspective, (Gronhaug K, Kaufmann G, Eds.), 205–232. Oslo: Norwegian University Press Higgins JM, (1994) 101 Creative Problem Solving Techniques. Winter Park, FL: The New Management Publishing Company Lewis AC, Sadosky TL, Connolly T, (1975) Effectiveness of Group Brainstorming in Engineering Problem Solving. IEEE Transactions on Engineering Management EM-22:119–124 Linsey JS, Clauss EF, Kurtoglu T, Murphy JT, Wood KL, Markman AB, (accepted) An experimental study of group idea generation rechniques: Understanding the roles of idea representation and viewing methods. ASME Journal of Mechanical Design Mullen B, Johnson C, Salas E, (1991) Productivity Loss in Brainstorming Groups: A Meta-Analytic Integration. Basic and Applied Social Psychology 12:3–23 Nijstad BA, Stroebe W, (1999) Persistence of brainstorming groups: How do people know when to stop? Journal of Experimental Social Psychology 35(2):165–185 Otto K, Wood K, (2001) Product Design: Techniques in Reverse Engineering and New Product Development. Upper Saddle River, NJ: Prentice Hall Pahl G, Beitz W, (1996) Engineering Design – A Systematic Approach. New York: Springer Paulus PB, Yang HC, (2000) Idea Generation in Groups: A Basis for Creativity in Organizations. Organizational Behavior and Human Decision Processes 82:76–87 Shah JJ, (1998) Experimental Investigation of Progressive Idea Generation Techniques in Engineering Design. DETC’98, 1998 ASME Design Engineering Technical Conferences, Atlanta, GA Shah JJ, Kulkarni SV, Vargas-Hernández N, (2000) Evaluation of Idea Generation Methods for Conceptual Design: Effectiveness Metrics and Design of Experiments. Transactions of the ASME Journal of Mechanical Design 122:377–384 Shah JJ, Smith SM, Vargas-Hernandez N, (2003) Metrics for measuring ideation effectiveness. Design Studies 24:111–134 Shah JJ, Vargas-Hernández N, Summers JS, Kulkarni S, (2001) Collaborative Sketching (C-Sketch) – An Idea Generation Technique for Engineering Design. Journal of Creative Behavior 35:168–198 Van der Lugt R., (2002) Brainsketching and how it differs from brainstorming. Creativity and Innovation Management 11:43–54 VanGundy AB, (1988) Techniques of Structured Problem Solving. New York: Van Nostrand Reinhold Company Vidal R, Mulet E, Gómez-Senent E, (2004) Effectiveness of the Means of Expression in Creative Problem-Solving in Design Groups. Journal of Engineering Design 15:285– 298 Methods and Tools for Design Creativity Front End Industrial Design (FE-ID) - Developing New Tools and Models for Industrial Designers to Operate at the Front End of New Product Development Paul W. Wormald Virtuality – Offering Opportunities for Creativity? Anthony Williams, Ning Gu and Hedda Haugen Askland Thinking Inside the Box: Model, Tool, Team and Setting for Building Design Wim Zeiler Signs of Collaborative Ideation and the Hybrid Ideation Space Tomás Dorta, Annemarie Lesage, Edgar Pérez and J.M. Christian Bastien Front End Industrial Design (FE-ID) - Developing New Tools and Models for Industrial Designers to Operate at the Front End of New Product Development Paul W. Wormald National University of Singapore, Singapore Abstract. The front end of new product development is often a focus for discussion about innovation. This paper presents a new model for how the role and place of industrial design can be re-positioned so that it has influence and impact at this front end. The aim of this is to develop the capability for industrial designers to generate creative ideas, or opportunities, that have resonance and relevance within the context of new product development, but that are made explicit before a design brief exists. A front end industrial design process model, developed for undergraduate designers and evaluated by industry, is described in detail. Keywords: industrial design, ideas, new product development, front end innovation, undergraduate design education 1 Introduction Design creativity should be about ideas as well as beautiful artefacts. Ideas, as well as designed outcomes, need to be 'beautiful'. So a debate surrounding design creativity can include some discussion about ideas. Beautiful ideas which are successfully resolved into products are often judged as attractive product innovation. Creative product innovation is a highly prized goal for most commercial enterprises. Von Stamm (2003 p2) states "One of the big concerns for many companies is how to generate more and better ideas - how to become more creative." This paper describes an approach which aims to develop and enhance creative ideas by moving the influence and impact industrial design has on new product development (NPD) to before, as well as after, a design brief is written. This, and the desire to have those ideas rooted in evidence and sound process, requires new knowledge and abilities, and new modes of working, including new tools and designerly outputs. What has emerged, mainly from industrial design education curriculum development, is a overall process model which could be adopted in a re-think of how industrial design can better contribute to successful NPD (in commercial and other enterprise arenas). This process model has been refined over half a decade of university-level design education. It has been reviewed by large international companies, design consultancies and design research companies. The aims of this paper are: To reveal and illustrate a process through which industrial designers can successfully impact on the front end of NPD, partly by generating creative and targetted ideas. To contribute and stimulate some debate and discussion concerning the future role of industrial design in NPD. 2 The Front End of NPD Figure 1 shows a basic diagram of commercial NPD. Fig. 1. NPD diagram The diagram identifies various stages and activities. Obviously, this diagram is not to scale in the sense of time. It shows that design (industrial, engineering etc.) typically starts after a brief is formulated. Its major feature is the (pre-brief) area before a formal design brief emerges. It is well known that companies will attempt to utilise methods and approaches to enable successful innovation in the development of new products. The 176 P.W. Wormald methods and approaches used very early on in the new product development process are often referred to a 'front-end' processes (Koen 2002 and Cagan and Vogel 2001). They can include such activities as user/customer research, brand management, trend surveys, and market analysis. Additionally, these methods and approaches are often seen as being less strict, rigorous or even less well understood than some of the 'downstream' processes such as product design, manufacturing engineering, and product certification. Hence, this front end of NPD is sometimes referred to as the 'fuzzy front end' (FFE). With regard to industrial design activity in the early NPD processes then Veryzer (2005) and Jurotovac (2005) demonstrate how industrial designers make successful contributions to the overall process. There has been limited pedagogic research into the issue of how industrial design education can exploit these new areas of opportunity. Design students pursuing user research activities early in project work is discussed by Siu (2003, 2007), Lopes (2008) and Lofthouse (2008). They all point to the potential benefits of enhanced innovation and designer empathy with users. Some of the background issues concerning FFE thinking are covered by Wormald (2009). He provides a full background to this paper, particularly the drivers for change which provided the impetus for industrial design educational curriculum development. 3 A Process Model for Front End Industrial Design (FE-ID) Figure 2 shows a process model for the front end of NPD activities undertaken by industrial designers, hence the term front end industrial design (FE-ID). It is important to note that this model has been formulated following six annual cycles of pedagogic action research. This action research was instigated to investigate and support changes to the author's curriculum development in the subject of industrial design at a UK university. The diagram is a more detailed view of how the role of industrial design can be modelled during that, notoriously 'fuzzy' period. The diagram has been formulated to clarify and visualise the various stages and outputs of the front end investigation and synthesis processes. The following sections describe each of the major processes, with associated tools and output models. Examples of work completed by industrial design undergraduates are presented. Each stage or step in the process can be broken down into sub-areas for subsequent analysis and possible synthesis. The generated outcomes can lead to further stages and further subsequent analysis. The process begins with a review, or revealing, of various contextual issues. Within the broad, fuzzy, arena of very early new product thinking there will be Fig. 2. FE-ID process diagram Front End Industrial Design (FE-ID) - Developing New Tools and Models for Industrial Designers at the FE of NPD 177 a multitude of influences and pressures. These are indicated in the NPD diagram (fig 1). There is, of course, no sense of what the nature of a new product will be. There is no 'big idea', there is no design brief, there is only a sense, or urge, that a new product is necessary. The contextual issues are stated as clearly as possible, but with enough flexibility to allow for exploration and wide-ranging relevant research. Four context areas are identified: User/consumer Scenario/theme Global/PEEST Company/brand For 'user/consumer' a target user group is outlined. This usually entails identifying simple demographic information such as age range, gender, and occupation. For 'scenario/theme' some broad user activity or behaviour is outlined. This would have relevance to the target user group and the company/brand. Examples could be 'cooking', 'keeping fit', or 'local travel'. PEEST stands for Political, Economic, Environmental, Social, Technology. There will be a 'global/PEEST' context to be investigated, especially relating to the overall theme being explored. Each of the Political, Economic etc. areas can be reviewed for possible insights. Finally, the company and its relevant brand is a necessary component of any sound NPD thinking. Following the above basic clarification, the context areas can be researched. This research can be conducted independently, in parallel, by a team of design researchers. Investigation can be overlapping and related, but it is best to be able to have a view of the separate areas initially. The different contexts will be researched using different methods and strategies. Different types and forms of data will be gathered. Possible research methods used and types of data gathered for each context area are identified below. Context - user/consumer Data gathering activities: - Ethnographic observation - Interviews, questioning - Conversation, chatting - Photography, video Types of data gathered: - User stories - Detailed demographic information - Photos of people and environments - User attitudes and emotions - User quotes - Observational texts Context - scenario/theme Data gathering activities: - Observation - User interviews - Photography, video - Internet searching Types of data collected: - Photo essays of behaviour - User stories of scenario experiences - Observational texts - Comment and opinion on theme Context - global/PEEST Data gathering activities: - Internet searches - Newspaper, trade magazine reading - Media analysis Types of data collected: - News snippets/articles - Technical briefing notes - Cultural references (photos, notes) Context - company/brand Data gathering activities: - Analysis of company publications and websites - Discussion with company management - Identification and analysis of competitor products/brands - Consumer interviewing Types of data collected: - Product (company and competition) imagery - Company financial and market information - Brand values - Forms of brand manifestations As data gathering progresses the process of attempting to make sense of it all begins. As meaning and sense develops for the design researchers, increased understanding of the context areas will drive new avenues of investigation in an iterative way. The design researchers will aim to synthesise specific areas of 'meaning' from the various data sources. Arising from the context areas these areas of meaning include: User goals; User lifestyle; Insights into user behaviour; The experience of the user relating to the scenario/theme; Relevant PEEST insights; Company market position - and subsequent insights; 178 P.W. Wormald Brand territory; Illustration of brand values - characteristics. In this work, an insight can be the answer to the question "What do we now understand from the evidence, that we did not know before?" Insights are highly valued and valuable 'nuggets' of information that relate strongly to the overall contextual issues. For the industrial designer an important part of the process of making sense of the data takes the form of communicating its meaning in particular and specialised forms, namely 2D boards (printed or digital) containing rich visual and textual information. The industrial designer uses standard graphical techniques to present meaning and understanding in easily accessible and engaging forms. In the FE-ID process (fig 2), these are identified as BRAND, PERSONA, and EXPERIENCE. These are separate boards, and a more detailed explanation of each, with examples, follows. BRAND The aim of this board is to 'bring the brand to life'. It should reveal the brand territory (the metaphorical ground the company's brand occupies in the market). It should illuminate the underlying meaning of the brand values. It should successfully explain and illustrate any of the brand messages (such as taglines and jargon used in advertising and marketing). The content of the BRAND board would typically include: Company name; Brand name; Brand visuals (such as logotype, product visual language); Iconic products of the brand Analogous, comparative products / brands / competitors; Explanatory texts. Fig. 3. Example of BRAND board PERSONA The aim of this board is to 'bring the user to life'. A persona is a standard, accepted way of defining and visualising the target user group. It is often used in marketing activities, and software design (Cooper 1999), and product design (Pruitt and Adlin 2006). There may be several personas for each project. The content of each the PERSONA board would typically include: Photograph of a person (with 'character'); Name; Categorisation (a form of 'micro' description); Basic demographic facts (such as age, occupation, location); Lifestyle information (pictures and text), to paint a picture of the character; Goals, needs or expectations which are specific to the scenario/theme; A short narrative, telling a story relevant to the user and theme. It is important to note that the details above are highly credible, but are actually fictional. This is partly for ethical reasons. Fig. 4. Example of PERSONA board EXPERIENCE The aim of this board it to 'bring the user experience (of a specific scenario/theme) to life'. The emphasis on this board is to reveal the actual behaviour of users, not simply to report what the users say that they do. Stappers and Sleeswijk (2007) describe an approach they call "Context Mapping" which aims to reveal similar issues surrounding user experience. The content of the EXPERIENCE board would typically include: Photos of users in relevant environments and situations; Photos of users actually doing tasks, jobs, activities; . in Creative Problem-Solving in Design Groups. Journal of Engineering Design 15:285– 298 Methods and Tools for Design Creativity Front End Industrial Design (FE-ID) - Developing New. Engineering Design. DETC’98, 199 8 ASME Design Engineering Technical Conferences, Atlanta, GA Shah JJ, Kulkarni SV, Vargas-Hernández N, (2000) Evaluation of Idea Generation Methods for Conceptual Design: . before a design brief exists. A front end industrial design process model, developed for undergraduate designers and evaluated by industry, is described in detail. Keywords: industrial design,