Developing a Coding Scheme to Analyse Creativity in Highly-constrained Design Activities 159 and coding could not begin until the design activity had finished. Table 2. Sequence of activities for the experiment A Logitech digital pen and paper note book was used to convert and store a digital copy of the design sketches and written notes that the designer created though the process. The digital notebook also contained a column that was used by the researcher to code the design outputs. Almost all of the design outputs were also modeled using computer-aided design (CAD), and it is these representations, which were then presented in sequence with significant descriptive notes, that were formally coded. The design task itself is not described in this paper, as it is a real task previously performed in industry and the researchers are planning to compare the outputs from the experiment with those from industry Table 2 above describes the sequence of activities that took place in the experiment and highlight the particular roles that the various researchers (referred to as HY, TJH and EAD) played. Figure 1 below shows an example of how the design ideas were presented in sequence and how they were formally coded. The first two columns show the numbering system used for the design ideas. B1 represents an initial, unique design idea. B2, B3 and B4 show the iterations of this design idea. The last five columns show how each of the design ideas were coded using both the 1st and 2nd coding schemes developed in this research. 3 Results Looking at the whole data set in the first round showed that, for 18 out of the 30 design ideas coded the researchers agreed, whilst in 12 cases the coders were not in agreement. It was this disagreement and the following discussion that lead the development of the 2nd coding scheme. Table 3 below shows a sample of the discussion from the 12 cases where the coders were not in agreement. Table 3. Sample of the coding disagreement discussion Some observations and recommendations were drawn from the disagreements above: A reference design that the new design idea is compared to should be specified before coding. This seems obvious in retrospect, as it is impossible to code the initial set of ideas without a reference design: a change needs to be coded relative to something. Once the initial Activity description Resear- cher ID role played (designer- researcher) 1 st round of design and analysis Development of the 1st coding scheme: Creative Modes of Change TJH researcher Briefing on the highly-constrained design task HY designer Development of design ideas HY designer Review and iteration of the design ideas HY TJH designer Coding the design ideas using 1 st coding scheme HY researcher Assessing the quality of design ideas using company’s Criteria Decision (MCDA) table HY TJH designer Inter-observer coding of the design ideas using 1 st coding scheme TJH EAD researcher 2 nd Round of design and analysis Development of the 2 nd coding scheme HY researcher Development of design ideas – With creativity tools HY designer Coding the design ideas using 2 nd coding scheme HY researcher Assessing the Quality of design ideas using the company’s MCDA table HY TJH designers Analysis of results all researcher 160 E.A. Dekoninck, H. Yue, T.J. Howard and C.A. McMahon ideas are coded, subsequent design iterations can be coded relative to each one preceding it. Although all coders were given definitions for each code, coders should be trained in advance using an example to elicit queries and tease out any problems with the coding scheme. One of the requirements of New Auxiliaries (NA) is to bring in new functions that are not listed in the original functional requirement. This caused some disagreements when new elements were added to the system, and highlighted the need for clear definitions of ‘system’, ‘element’ and ‘function’ (see section 3.1.2) The coding scheme could be improved by defining New Auxiliaries as an additional element/module instead of additional function. For differentiating whether a new function is added, the outcome of the modification can also be coded as Additional Function or Reduced Function; Quite often Technology Pull (TP) or Improved Understanding (IU) cannot be indentified without knowing the rationale from the designer who made the modification. For example in design idea E2, researcher HY coded the concept as New Design (ND) since being the designer, he knew that the reason for the modification was the need to integrate a hinge to a flap. However, the other researchers (TJH and EAD) considered the change as the result of new material (thin and flexible plastic) therefore coded that change as Technology Pull (TP). In idea F2 a very similar situation arose, but the other way round. It is therefore clearer to separate Improved Understanding (IU) and Technology Pull (TP) from rest of the modes of change and code them as the factors that drive the design modification (or design rationale). Researcher TJH suggested Modularization as a new mode of change when coding design idea A2, in order to provide a mode of change that is opposite to Functional Integration (FI). A similar approach was applied to New Auxiliaries (NA) where; Trimming could be introduced as a new MOC that describes the modification that discards unnecessary element to improve performance. Fig 1. Sample of design idea formal coding sheet Developing a Coding Scheme to Analyse Creativity in Highly-constrained Design Activities 161 3.1 Introducing the 2nd coding scheme The modified scheme comprises three ‘levels’ of design change to be coded: the factor that drives the modification, the design modification itself, and the final resulting effect on the system from the modification. Figure 2 shows the three levels used in this 2nd coding scheme. Fig. 2. Three levels of the 2nd coding scheme. 3.1.1 Factor that drives the design change The coding in this section describes the various types of rationale which can drive the specific design modifications. Design rationale includes ‘not only the reasons behind a design decision but also the justification for it, the alternatives considered, the trade-offs evaluated and the argumentation that led to the decision’ (Lee, 1997). These are not obvious by simply looking at the design modifications themselves. Even for the same design modification, the underlying rationale may be different and therefore usually best described by the designer who made the modification. New requirement (NR) - One or more new requirements raised by market/organization/designer, or any other party, that requires new design ideas to achieve. Improved understanding of design performance parameter (IU) - Through modelling and empiricism engineers benefit from the discovery - or better understanding - of relationships between the design parameters and the performance. This understanding can then go on to drive various design modifications. Technology Pull (TP) - The adoption of a novel and appropriate technology or material to expand the design space, which can then in turn drive various design modifications. This may simply have a direct relationship to performance, such as changing material to reduce weight. However, it could lead to more complex relationships. One example observed in recent research, was where a new material coating was adopted, which enabled a different spray coating process, and eradicated post process machining, thereby producing substantial benefits. Design Improvement (DI) - Without adding any new requirement, the rationale of the modification is only to further improve the performance of the system. During the iterations of design ideas, the designer sometimes sees opportunities to set higher targets for the system. This raises the standard for the design ideas without adding any new requirements. 3.1.2 Design Modification These define the ways in which each design idea presented differs with respect to the reference design. In this study, the initial unique design idea presented (e.g. B1) was compared to a common solution already on the market. The subsequent design iterations (e.g. B2, B3, etc.) were coded relative to each one preceding it. The codes presented below are based on the assumption that in highly-constrained design tasks, the designer is usually designing ‘elements’ (parts) of a sub-system, which perform particular ‘functions’ for the ‘system’ (or super-system). The different types of changes that are seen as the design ideas evolve are defined as: Parameter Change (PC) - In this change the parameter of an existing design element is modified. However the ‘performance - attribute’ relationships governing the design are not changed as a parameter is adjusted. Thus changing the ‘number of wheels on a car’ is not a parameter change, as new ‘performance – attribute’ relationships are inevitably formed when changing the number of wheels. New Auxiliaries (NA) - In this change a new function which was not a part of the system, 162 E.A. Dekoninck, H. Yue, T.J. Howard and C.A. McMahon and is distinct from any other function within the system, is added into the system. Modularisation (MD) - In this change the functional requirements of a system are fulfilled by an increased number of sub- systems, parts or features. This may for example, be beneficial to the design in terms of: increasing reliability, adaptability, or performance. Suh (1990) for example, advocates decoupling functions such that each function has a single associated part or feature. Functional integration (FI) - In this change any two or more elements within the system are combined into a single element that performs the same function. New Design (ND) - In this change an existing function is performed by a completely new element. Trimming (TM) - This change occurs when any element is discarded. 3.1.3 Modification outcome Codes in this section describe the different types of outcome observed for the overall system. These describe changes in the overall function or performance or the final resultant benefits to the system from the creative design modifications. Better performance (BP) - The existing system performs better. Additional function (AF) - Extra function is added to the system. The function may or may not have been part of the original functional requirements. A creative design modification occurring during the process may add additional beneficial functions to the system. Reduced function (RF) - Function is discarded from current design, a direct opposition to Additional Function, in order to improve the overall performance of the system. 3.2 Reviewing the coded concepts on a timeline Figure 3 on next page presents all the design ideas on the project day-by-day timeline. For example, A3 (PC) means the third iteration of the initial idea A1, where Parameter Change is the Mode of Change evident in the design. Only the agreed coding from round 1 is included in brackets behind the concept numbers. In order to use the 1st and 2nd round of design and analysis as a single data set, only the Design Modification codes of the 2nd Coding scheme are presented in this diagram as they are coded at the same ‘level’. There were 6 days between the two rounds were no new concepts were generated. It is possible to detect some patterns of modes of change that occur throughout a creative design process, these are discussed in section 4.2. Each of the final concepts (e.g. A9, B5, C3, etc) was given a Quality score from the company’s Multi Criteria Decision Analysis (MCDA) table. The company’s MCDA table consists of eight criteria against which each concept is scored, these are added up to generate the Quality score. The MCDA includes functional criteria such as ‘hold low vacuum’ and ‘hygienic’ as well as business criteria such as ‘product cost’ and ‘development time required’. The Quality score is shown below in bold and is out of maximum of 72. Whether particular patterns lead to more successful outcomes in terms of solution quality is discussed in section 4.2. 4 Discussion This section discusses the design modification codes (middle ‘level’) from the 2nd coding scheme as these were analysed in more depth than the results from the other two levels. It also makes general observations about the modes of change observed. 4.1 Discussion of the 2nd coding scheme In practice in the study, the codes were created through the action research cycle, using a type of content analysis, where definitions of codes were adjusted, and new codes were created, in order to be able to code the entire data set. In retrospect, it is possible to view the codes created in this research as describing two fundamental aspects that change: the functions that are performed by the design and the actual designed elements that perform those functions. They change by creating, discarding or integrating. Figure 4 below shows how the definitions of the codes presented in section 3.1.2 can be placed in the matrix. Fig. 4. Matrix of Design modification codes relating to changes in elements and functions. Element existing new integrate discard Function existing PC ND MD FI new NA (b) integrate (a) discard TR Developing a Coding Scheme to Analyse Creativity in Highly-constrained Design Activities 163 This helps to highlight the difference between New Design (existing function is performed by a new element) and New Auxiliaries (new function is added into the system). The matrix also highlights an anomaly in one of the codes. Functional integration (FI) is actually defined as the integration of elements, and should perhaps be relabeled as Element Integration (EI). The table also points towards the opportunity to define other Design Modifications that did not arise in this experiment but could be useful both for coding future experiments, for example: the integration of existing functions through the design of a new element (a); or creation of a new function by integrating existing elements (b). Although the single case presented here does not allow detailed analysis of the three ‘levels’ coded in the 2nd coding scheme they do provide some insights into the nature of creativity in highly-constrained design tasks. At the top level, it may be possible to develop/specify tools that stimulate designers to think of strategies that then drive successful design modifications. These types of tools would have to work through stimulating/guiding design rationale. Looking at the middle level where the design ideas themselves are coded, it may be possible to specify particular creativity tools to stimulate particular design modifications. This experiment was able to initiate this work which is reported in (in preparation for ICED11). It is worth noting that before this can be done, a much larger study is needed to understand the design modifications - or patterns of them - that deliver the most creative results in highly-constrained design tasks. At the third (outcome) level it may be possible to develop/specify tools that stimulate designers to think of strategies for the system that then drive successful design modifications at the sub-systems level. 4.2 Patterns in Modes of Change Studying the data in Figure 3, it is possible to detect some patterns of modes of change that occur throughout a creative design process. These findings are tentative observations due to the limited number of coded instances. In most cases it is clear that the initial idea (e.g. B1, C1, D1, etc) starts with a New Design (ND) (round 1) or a New Auxiliary (NA) (round 2) followed by iterations of the ideas in the form of Parameter Change (PC). In some cases this works the other way round where successive iterations of Parameter Change (PC) lead to New Designs (ND) in the final instance (e.g. A8 and F6). This may happen where the designer feels they have pushed the idea to its limits and thus comes up with a totally new direction to explore. The difference between the number of New Design (ND) and a New Auxiliary (NA) codes between the two rounds is likely to be mainly due to changes in the coding scheme. day 1234567891011 A1, B1(ND) A2,C1 (ND) D1 (ND), E1 (ND) F1 C2 (FI), C3 (PC) 32 , D2 (PC) 38 , E2, E3 34 day 12 13 14 15 16 17 18 19 20 A3 (PC), A4 (PC) B2 (PC), B3 (PC) F2, F3 A5(PC), A6, A7(PC), A8(ND), A9 42 , B4(PC), F4, F5(PC), F6(ND), F7 24 B5 40 day 1234567891011 G1 (ND) 46 , H1 (NA) I1 (NA) 27 , J1(NA) K1(NA), K2 (ND) 38 L1 (NA), M1 (NA) 35 , N1(NA) H2 (PC) 44 , L2 (NA), L3 (PC) 21 , O1 (NA) 31 J2 (PC) 26 , N2 (PC) 35 2nd Round of design 1st round of design 1st round of design Fig. 3. Overview of all design ideas, coded on the project timeline day-by-day. 164 E.A. Dekoninck, H. Yue, T.J. Howard and C.A. McMahon Each of the final concepts (e.g. A9, B5, C3, etc) was given a Quality score from the company’s MCDA table. The score is shown in Figure 3 in bold and is out of maximum of 72. From this data there is no clear quality difference between the design output from round 1 without creativity tools (average score: 35) and round 2 with creativity tools (average score: 34). However the pattern in which solutions were generated was significantly different, where in round 1 most initial ideas (6 in total) were iterated several times (usually through Parameter Change), round 2 yielded many more initial ideas (9 in total). There was however no pattern in the quality scores linked to the time spent (number of days) or any benefit of ‘carrying the ideas’ through (number of iterations). Coding design output in this way may contribute one way of mapping the way designers move around the design space, and particularly the strategies that are used by creative designers to skip from one ‘train of solutions’ to new avenues. 5 Conclusions This paper shows that it is possible to categorise design changes into different creative modes of change using the coding scheme developed. The coding scheme can be made more robust by: ensuring design change is always coded relative to a reference design; tightening up definitions of ‘system’, ‘element’ and ‘function’; and using a matrix, such as the one presented in Figure 4, to develop a more complete set of codes. A much larger study with more designers working on different types of highly-constrained design task is needed, in order to draw conclusions on the modes of change and their relationship to creativity. Design research would benefit even more if such a study was conducted in industry. The single case presented here does show that there can be creative steps in each type of mode of change. One promising area identified for further research is to look at the patterns of modes of change that occur throughout a creative design process. Some common patterns were identified in this paper, but there were no links between patterns and final outcomes in terms of solution quality. The methodology could be made more robust if the designers and researcher coded separately and data was triangulated with direct observations, ‘thinking aloud’ protocol or reflective interviews. Although in this case we did not measure creativity as part of the study, the coding tool developed will help to map the way designers move around the design space, and particularly the strategies that are used by creative designers to skip from one ‘train of solutions’ to new avenues. The coding scheme can ultimately perform two functions for design research: firstly by understanding existing practice in greater detail (e.g. conducting a study of particularly talented/creative designers working on highly-constrained design tasks); or using even early outcomes iteratively to specify/develop tools to stimulate creativity in highly-constrained design tasks (e.g. cycles of action research that develop and test tools stimulating/guiding particularly creative design rationale). References Bjork E, Ottosson S, (2007) Aspects of consideration in product development research. Journal of Engineering Design 18(3):195–207 Brown DC, (2010) The Curse of Creativity. In proceedings of DCC10: The 4th International Conference on Design Computing and Cognition, Stuttgart, Germany, 12–14 July Hales C, (1986) Analysis of the engineering design process in an industrial context. Mechanical Engineering: Cambridge, University of Cambridge Lee J, (1997) Design rationale systems: understanding the issues. IEEE Expert 12(3):78–85 McMahon CA, (1994) Observations on modes of incremental change in design. Journal of Engineering Design 5(3):195–209 Pahl G, Beitz W, (1984) Engineering Design. Design Council/ Springer: London Suh N, (1990) The Principles of Design. Oxford University Press: USA Vincenti W, (1990) What Engineers Know and How they Know It. John Hopkins University Press: Baltimore, MD Effectiveness of Brainwriting Techniques: Comparing Nominal Groups to Real Teams Julie S. Linsey and Blake Becker Texas A&M University, USA Abstract. Engineering designers need effective and efficient methods for idea generation. This study compares the effectiveness of group idea generation techniques to the combined efforts of individuals working alone with redundant ideas removed, so called “nominal groups”. Nominal groups compared to real interacting groups is a standard approach for determine if a group idea generation method can produce better solutions then individuals working alone. This study compares nominal group data to existing data on a series of group idea generation techniques. Results show that groups using rotational viewing and representing their ideas with words & sketches, a hybrid 6-3- 5 method, outperform nominal groups in number ideas and have an equal level of quality. This result is in contrast to comparing Brainstorming groups to nominal groups where nominal groups outperform Brainstorming groups. These results indicate that a team can be more effective than individuals working separately. Keywords: creativity, idea generation, brainwriting 1 Introduction and Background Over one hundred formal idea generation techniques have been developed in areas such as psychology, business, and engineering (Adams, 1986; VanGundy, 1988; Higgins, 1994). Some methods like Osborn’s Brainstorming have received significant evaluation whereas for many graphical methods there is little data available. One of the first studies using Osborn’s Brainstorming method in engineering design included engineering professionals working on a realistic engineering problem and showed that groups using brainstorming produced fewer ideas than the combined efforts of an equivalent number of individuals working alone (Lewis, et al., 1975). This result, called productivity loss, is consistent with the vast majority of studies on variations of Osborn’s Brainstorming (Mullen, et al., 1991). While the data on Brainstorming techniques is extensive, there is far less data available on brainwriting techniques where communication is through written words or sketches. For brainwriting techniques, some data suggests that groups can be more effective than the combined individual efforts (Gryskiewicz, 1988; Paulus and Yang, 2000). Recent studies have focused on the development and evaluation of more effective idea generation methods in engineering and design related fields, including industrial design and architecture (Shah, 1998; Shah, et al., 2000; Van der Lugt, 2002; Shah, et al., 2003; Vidal, et al., 2004). These studies have used a mixture of sketches, verbal descriptions of ideas, and physical models in the idea generation process. Prior work on graphical brainwriting techniques (e.g., Brainsketching, C-Sketch, Gallery), has not compared nominal groups (non-interacting individuals whose non-redundant results are combined) with real interacting groups. Our study compares nominal groups with group ideas generation methods: Brainsketching, C-Sketch, 6-3-5, and the first phase of the Gallery method. These methods are gaining popularity and exposure in the engineering research community, in addition to industrial application. They also form a diverse set of group idea generation techniques that vary in how ideas are exchanged and in the types of representations used (written words, sketches, etc.). To understand the theoretical basis of these method, we dissect them into two key factors (1) how a group’s ideas are displayed to other members (“rotational view” or all are posted in “gallery view”) and (2) the form of communication between group members (no communication, written words only, sketches only or a combination of words and sketches.) All other method parameters are kept constant for all experimental conditions. 1.1 Osborn’s Brainstorming The term “brainstorming” is frequently applied to idea generation techniques in general and not just to the technique developed and named by Osborn. Osborn’s Brainstorming begins with a facilitator explaining the problem. A group then verbally exchanges ideas 166 J. Linsey and B. Becker following four basic rules: (1) criticism is not allowed, (2) “wild ideas” are welcomed, (3) building off each others’ ideas is encouraged, and (4) a large quantity of ideas is sought. Despite the face validity of these rules, much research demonstrates productivity loss in brainstorming compared to an equal number of individuals working alone (nominal groups) (Mullen, et al., 1991). Silent Sketching More Silent Sketching Review and Discussion Fig. 1. Illustration of Gallery method 1.2 Brainsketching In Brainsketching, individuals begin by silently sketching their ideas on large sheets of paper including brief annotations. Group members exchange drawings and silent sketching continues for another period of time (VanGundy, 1988). This technique allows for a visual means of expression, and so it is well suited for product design. Van der Lugt used teams of advanced product design students to compare Brainstorming to a variant of Brainsketching (that included the explanation of ideas between exchanges) (Van der Lugt, 2002). The Brainsketching variant led to more cases in which group members built on previously generated ideas than did Brainstorming. 1.3 Gallery In the Gallery method, individuals begin by sketching their ideas silently on large sheets of paper. After a set amount of time, participants discuss their ideas and move about the room studying others’ ideas. This review phase is followed by a second stage of silent sketching (VanGundy, 1988; Pahl and Beitz, 1996; Shah, et al., 2001). The review phase allows team members to clarify their ideas, and it provides social interaction. Fig. 2. Illustration of 6-3-5 and C-Sketch. Six people silently describe three ideas on a sheet of paper and then exchange papers 1.4 C-Sketch / 6-3-5 For 6-3-5 (Shah, 1998; Otto and Wood, 2001; Shah, et al., 2001) and C-Sketch (Shah, 1998), six (“6”) participants are seated around a table, and each silently describes three (”3”) ideas on a large sheet of paper. The ideas are then passed to another participant. This exchange goes on for five (“5”) rounds. For the original 6-3-5 method, ideas are described using only words. In contrast, the C-Sketch, method permits only sketches. One advantage of C-Sketch over 6-3-5 is that sketches are typically ambiguous, and so one person may misinterpret aspects of someone else’s sketch, which may lead to new ideas (Shah, et al., 2001). Other variations of 6-3-5 have also been proposed (VanGundy, 1988; Otto and Wood, 2001). One variation permits annotated sketches (Otto and Wood, 2001). In experimental comparisons with different conditions than those reported in this paper, C-Sketch and Gallery outperformed 6-3-5 (words only) for variety, quality and novelty of ideas (Shah, et al., 2001). Novelty is how unique a particular idea is and variety is how much of the design space is captured by a set of ideas. This previous study used groups of mechanical engineering undergraduates, mechanical engineering graduate students and professional designers. Each group was evaluated on all three techniques and a different design problem was solved for each of the techniques. This design eliminated individual differences as a noise variable but caused the technique results to be confounded with the design problem. Effectiveness of Brainwriting Techniques: Comparing Nominal Groups to Real Teams 167 2 Experimental Approach and Research Questions Engineers seek a robust idea generation method for predictably producing a large quantity of high quality, novel product solutions. Using a factorial design of experiments, our study explores the influence of the representation used to communicate ideas and how ideas are displayed to individuals. We seek to answer the following research questions: Research Question: How do the nominal groups compare to real groups in terms of quantity and quality of ideas? This research questions is addressed systematically in the following sections. We discuss our experimental method, metrics for evaluation, data analysis approach and the results. 3 Experimental Method We conducted a factorial experiment in order to explore the effects of two key factors on the outcome of group idea generation. The first factor controls how participants view the ideas, either all ideas are posted via gallery (on the wall), sets of ideas are rotated between participants, or they are not exchanged (individual idea generation-nominal groups). The second factor controls how participants represent their ideas. Participants either use written words only, sketches only, or a combination of written words and sketches to communicate ideas to their teammates. A 2 (Display of ideas: “gallery” or “rotational view”) X 3 (Representation: words only, sketches only, or words combined with sketches) factorial experimental design is used (Table 2). No oral discussions are allowed during the session; all communication is written. This approach produces methods similar to 6-3-5 (Pahl and Beitz, 1996), C-Sketch (Shah, 1998), Brainsketching (VanGundy, 1988), or Gallery Method (Pahl and Beitz, 1996), as shown in Table 3. All participants solved the peanut sheller problem (Linsey, et al., accepted). 3.1 Factor 1: Display of Ideas One key factor in this study is whether ideas are displayed all at once or whether participants see only a subset at any given moment. In the “gallery view” condition, all ideas generated by the team are posted on the wall, so all participants can see all of the ideas at the same time. This approach results in a method similar to Gallery Method or Brainsketching (VanGundy, 1988; Pahl and Beitz, 1996). In the “rotational view” condition, ideas are passed around the table, so that each participant sees only a subset of the ideas at any given moment. This condition is similar to 6-3-5 or C-Sketch (Pahl and Beitz, 1996; Shah, 1998; Otto and Wood, 2001). 3.1.1 Gallery View Condition- Similar to Brainsketching or Gallery Method For the first 10 minute period, each student is given a number of paper sheets and told to write down at least two ideas on separate sheets of paper. Sheets are collected as participants finish, but are not displayed until the end of the period. The time period length is based on the available time and recommendations from the literature, which vary from five to 15 minutes (VanGundy, 1988; Baxter, 1995; Shah, et al., 2000). The ideal time period for the methods under evaluation is not explicitly known and is not one of the experimental parameters. At the end of the first period, all sheets are numbered and posted gallery style on the wall. In the four subsequent 7.5 minute periods, ideas are posted as they occur and participants are told to execute one of the following options: 2. Add new ideas to one of the posted drawings. Participants can request a drawing by writing down its number on a small sheet of paper. 7. Make a separate drawing that is related to the Table 1. Experimental conditions Factor 2: Representation Words Only Sketches Only Words and Sketches Factor 1: View Condition 1 3 5 Gallery View Rotational View 2 4 6 Individual 7 8 168 J. Linsey and B. Becker ideas that are already posted, and write the number of the linked idea on the new sheet. 8. Start a completely new sheet after reviewing the posted ideas. For the first 10 minute period, each participant is given a number of paper sheets and told to write down at least two ideas on separate sheets of paper similar to the “gallery view” condition. At the end of the period, the experimenter collects all sheets and systematically redistributes them such that each participant views each set of papers once. Participants cannot identify which one of their teammates had the sheets previously. In the four subsequent periods, lasting 7.5 minutes each, participants have the same options as in the “gallery view” condition: to add ideas to an existing sheet, to create a new product solution linked to another sheet or to start a completely new product solution. The exception here is that participants focus on the specific set of papers given to them at a particular instance in time. Table 2. Experimental conditions and similar formal method Experimental Condition Similar Formal Idea Generation Method 1 Electronic Gallery (Aiken, et al., 1996) 2 6-3-5 3 4 C-Sketch 5 Gallery 6 Brainsketching 3.1.2 Rotational View Condition- Similar to 6-3-5 or C-Sketch For the first 10 minute period, each participant is given a number of paper sheets and told to write down at least two ideas on separate sheets of paper similar to the “gallery view” condition. At the end of the period, the experimenter collects all sheets and systematically redistributes them such that each participant views each set of papers once. Participants cannot identify which one of their teammates had the sheets previously. In the four subsequent periods, lasting 7.5 minutes each, participants have the same options as in the “gallery view” condition: to add ideas to an existing sheet, to create a new product solution linked to another sheet or to start a completely new product solution. The exception here is that participants focus on the specific set of papers given to them at a particular instance in time. 3.1.3 Nominal Groups For the nominal groups, individual were assigned to work alone and were given the same amount of time. 3.2 Factor 2: Representation The second experimental factor prescribes how the participants communicate their ideas to other participants (words only, sketches only with no words, or a combination of words and sketches). At the end of the sessions and after completion of the surveys, participants in either of the group sketches-only conditions labeled their sketches with brief descriptions to facilitate evaluation. American mechanical engineers are typically not taught to draw free-hand and therefore their sketches are usually difficult to interpret without annotations. The prior study (Linsey, et al., accepted) shows that the sketches only data shows a different pattern of results likely due to the poor sketch quality and effort required by teammates to interpret the drawings. For this reason, individual data was not taken and therefore no nominal groups. Boiling Water Water Mill by a Waterfall Cam Ver t ica l Crushing Plate Grate Hopper Graduated Concentric Crushing Surfaces Conveyor Collection Bin Hand Crank Conveyor Drive Grate Fire Water Inlet Hopper Ver t ica l Crushing Plate Hopper Fig. 3. Set of examples which were briefly and accidently shown in class to the nominal group participants The nominal group data was taken two semesters after the group data was collected. The same professor taught the class and the same experimenter collected the data. During the semester the nominal group data was collected and prior to data collection, the participants in the nominal groups were accidently shown example peanut shelling machines (Fig. 3). These ideas were only shown briefly in class and the participants’ data does not appear to be influenced. The nominal groups were formed by randomly assigning the results from five individuals to a group and removing redundant results. Data is from twenty- four individuals whose results were used to create forty nominal groups. . the highly-constrained design task HY designer Development of design ideas HY designer Review and iteration of the design ideas HY TJH designer Coding the design ideas using 1 st . stimulating/guiding design rationale. Looking at the middle level where the design ideas themselves are coded, it may be possible to specify particular creativity tools to stimulate particular design modifications researcher 2 nd Round of design and analysis Development of the 2 nd coding scheme HY researcher Development of design ideas – With creativity tools HY designer Coding the design ideas using