ORGANIZATIONAL LEARNING THROUGH POST-PROJECT REVIEWS IN R&D doc

Organizational learning through post-project reviews in R&D Maximilian von Zedtwitz IMD – International Institute for Management Development, P.O. Box 915, CH-1001 Lausanne, Switzerland, zedtwitz@imd.ch Post-project reviews are one opportunity to systematically improve performance in subsequent projects. However, a survey reveals that only one out of five R&D projects receives a post-project review. Post- project reviews – if they take place – are typically constrained by lack of time and attention as well as lack of personal interest and ability. They focus mostly on technical output and bureaucratic measurements; process-related factors such as project management are rarely discussed. In this paper we review the role of post-project meetings as a tool to improve organizational learning at the group level. Based on 27 in-depth interviews with R&D managers carried out between 1997 and 2001, we categorize four classes of learning impediments. These difficulties are not easily resolved, as is illustrated by examples from Hewlett-Packard, DaimlerChrysler, SAP, Unisys, the US Army, and others. We propose a five-level post-project review capability maturity model, identifying some of the key capabilities that need to be in place in order to advance to the next process maturity level. Most companies reside on the first or second maturity level. Our conclusion is that many companies give away great potential for competence building by neglecting post-project reviews as a tool for systematic inter- project learning. 1. Introduction W hat do you take away from a finished project? A product? A lesson? A bad feeling? The essence is that most companies have not established a structured approach to learning from projects after their comple- tion. Even worse, most projects that have been prematurely terminated never undergo a retrospective analysis on their causes of failure. A recent survey indicated that 80% of all R&D projects are not reviewed at all after completion, and most of the remaining 20% were reviewed without established review guidelines. This paper raises the issue of organizational learning through post-project reviews. A post-project review is here defined as a formal review of the project examining the lessons that may be learned and used to the benefit of future projects. Post-project reviews – or post- reviews – are more prevalent in industries where knowledge is the main output of a project. For instance, consultants regularly perform after-action reviews in order to capture lessons learned from their engagements. Project reviews are commonly used as phase reviews, particularly in projects managed under the stage-gate paradigm. The final review, however, is either focused exclusively on technical issues or dropped altogether due to time and management constraints. Post-project reviews should aim at capturing process knowledge for enhancement of future project work, and hence differ from regular project reviews, project audits (Duffy, 1989; Neale and Homes, 1990), or project evaluations (Saladis, 1993). In this paper we propose a five-level capability maturity model for post-project review processes in R&D. Based on our review of the literature on organizational learning and R&D project manage- ment, we identify eight fundamental impediments to post-project reviews in R&D. We reflect on a few R&D Management 32, 3, 2002. # Blackwell Publishers Ltd, 2002. Published by Blackwell Publishers Ltd, 255 108 Cowley Road, Oxford OX4 1JF, UK and 350 Main Street, Malden, MA 02148, USA. observations on how to cope with these impediments to promote organizational learning based on team performance. 2. Research methodology At the intersection of group learning and R&D project management practice, the study of post-project reviews has not attracted widespread interest from R&D management researchers until recently. Little research has been published in the academic or management literature. In addition, practitioners have repeatedly voiced their concern about the lack of established criteria and guidelines to conduct post-project reviews, but their focus on future project challenges usually distracts them from a retrospective analysis of the past. The research leading to this paper is based on a combinative effort of doctoral research, in-depth interviews with R&D managers, and survey explora- tion. As the goal of the paper is to identify current R&D post-project review practices and propose managerial recommendations on how to improve them, we chose to begin with a broad interviewing sample for variety and reach of insight, preparing for a focused and limited in-depth investigation that will eventually produce a richer and more substantiated output in our ongoing study. As our level of analysis, we chose the boundary between individual and team learning. Between 1997 and 2001, we conducted a total of 27 interviews with R&D managers of leading companies from a wide range of technology-intensive industries and geographical regions. The interviews were con- ducted according to an interview guideline that focused on project management in R&D, specifically learning and information sharing in R&D teams. Interview minutes were returned to the interviewees for feedback, comments and additional interpretation. Where possi- ble, we participated in project meetings and post- project reviews. This personal observation comple- mented the interview data well. Data triangulation, as demanded by Yin (1991), was not possible in every instance as very little has been published by either the interviewed companies or third parties on the investi- gated projects. However, we had access to post-project reports of some of the studied companies. In addition to the interviews, we conducted two surveys of R&D post-project review practice in 2000 and 2001. Sixty-three R&D directors and managers of different organizations participated in our question- naire of the current state of R&D post-project reviews in their companies, including information about frequency and motivation of post-project reviews as well as process-related factors during their execution. As this is ongoing research, we expect our under- standing to be refined with the progress of this work. Our analysis based on the initial survey (see Koners and Zedtwitz, 2001) was confirmed by findings from the second survey conducted a year later. Considering all currently available data, however, we are confident that the present report presents a fairly reliable picture of the fundamental issues of managing post-project R&D reviews. 3. Post-project reviews in organizational learning and R&D management 3.1. Defining post-project reviews According to the great satirist and critic George Bernard Shaw, ‘we learn from experience that men never learn anything from experience’. Luckily, re- search in organizational learning over the past decades shows an improving understanding of how people and organizations learn (see Easterby-Smith et al. (2000) for a concise and recent review). Among a number of important mechanisms, the post-project review is one of the most structured and most widely applicable approaches to passing on experience from one team to the next. We define a post-project review as the final formal review in the course of a project that examines any lessons that may be learned and used to the benefit of future projects (see also the projectnet.com definition and Wideman, 1992). This section, as well as the following section on team learning (including Figure 1 and Table 1), is strongly based on the contribution of Ursula Koners to our joint publication reporting findings from the initial post-project review survey (Koners and Zedtwitz, 2001). The main objective of post-project reviews is to initiate and facilitate continuous learning on all levels within an organiza- tion. Learning effectively protects and enhances an organization’s competitive advantage (e.g. Wheel- wright and Clark, 1992; Johannessen et al., 1997; Gupta, 1998; Drejer and Riis, 1999; Takeuchi and Nonaka, 1986). Learning is critical in R&D organizations since it is the precondition for sustaining significant improve- ments over long periods of time. The body of knowl- edge on learning in R&D and New Product Development has been growing in recent years, but the variety of learning concepts studied is immense and the underlying processes are still scarcely known (e.g. Reger and von Wichert-Nick, 1997). One major drawback is the absence of investigations on learning in R&D after a project is actually finished. Most of the existing literature on post-project reviews in R&D repeats their importance and the fact that few organizations conduct post-project reviews regularly. Generally, the final phase of an R&D project concentrates on issues like the financial closure, the project appraisal and the reallocation of equipment and workforce to future projects (EIRMA, 1998). This Maximilian von Zedtwitz 256 R&D Management 32, 3, 2002 # Blackwell Publishers Ltd 2002 paper wants to advance the discussion further in the direction of inter-project learning capabilities – and barriers – in post-project reviews in R&D. 3.2. From team learning to organizational learning Argyris (1977) defined organizational learning as ‘a process of detecting and correcting error’. The notion of organizational learning has since evolved (see e.g. Crossan and Guatto, 1995; or Garvin’s (1993) defini- tion of ‘an organization skilled at creating, acquiring and transferring knowledge and at modifying its behavior to reflect new knowledge and insights’). We distinguish between three levels of learning: individual learning, team and group learning, and organizational learning (see Figure 1). At the basis of all learning is the individual (e.g. Barker and Neailey, 1999, p. 60). Individual learning is the domain of psychology and cognitive science; organizational learn- ing is studied by social psychologists and organization theorists. In this context, team learning may have a pivotal role in distributing, processing and interpreting individual experience for organizational memory and knowledge. Post-project reviews focus on the link between team and organizational learning, and – as we will discuss below – on the link between individual and organiza- tional learning. The process of organizational learning through team learning is best described in Argyris and Scho ¨ n’s (1978) concept of single-loop and double-loop learning. Single-loop learning pertains to the detection and correction of mismatches between experience and a reference system without questioning or altering the values of the system. Double-loop learning takes place when a mismatch is detected and used to adapt and correct the reference system. Post-project reviews (should) focus on double-loop learning. After the completion of the project – when the project team is ready to step back and reconsider what it has done and what has happened – the team is in the best position to review cause-and-effect relation- ships and should be required to propose improvements for the management and execution of future projects. Unfortunately, for a number of reasons which we outline further below, few teams find the appropriate amount of time for this exercise. Most learning is confined to individual learning, and substantial poten- tial for group learning is missed. Although individual learning is an important ingredient of organizational learning, both the depth and reach of individual experience are on average more limited. 3.3. Learning in the life of an R&D project A review is certainly not the only way to learn in or from a project. Although learning and knowledge accumulation in R&D are at the core of R&D, knowledge dissemination to other scientists and R&D managers is poorly facilitated and supported. Learn- ing, state Ayas and Zeniuk (2001, p. 64), ‘is not a natural outcome of projects knowledge created within a project is not always diffused, and lessons learned may not be shared across projects’. Learning through reviewing is one of the most frequent approaches to capitalizing on previous experience, and many R&D project management concepts (such as stage-gate (O’Connor, 1994; Cooper and Kleinschmidt, 1991) or milestone techniques) are based on formal points of reflection. 1 Figure 2 illustrates a schematized phase model of R&D project reviews (Balanchandra and Brockhoff, 1995, p. 32). It would be dangerous, however, to focus exclusively on formal methods of learning and neglect other informal and tacit means of knowledge and experience building. Some of the greatest advances in R&D management in the 1990s have been soft and informal management techniques, which tend to be extremely effective at the individual learning level. However, they are difficult to systemize and replicate. For example, SAP carries out a number of different reviews over the course of an R&D project. Reviews Individual Learning Team/Group Learning Organizational Learning Levels of LearningHow does learning occur? Knowledge acquisition Information Distribution Information Interpretation Organizational Memory How is knowledge embodied? • Personal memory • Personal experience • Notes • Individual capabilities • Networks • Reports • Products • Team-specific expertise • Technologies • Stories / Anecdotes • Databases • Procedures • Processes • Core competencies Figure 1. From team learning to organizational learning (based on Huber, 1991). Learning through post-project reviews # Blackwell Publishers Ltd 2002 R&D Management 32, 3, 2002 257 can be held as specification reviews, design reviews, coding reviews, and even reviews of online documenta- tion. Reviews are designed to ensure that everyone has access to the same project-critical information; infor- mation that is important for discussing problems and finding solutions, discovering potential risk areas, status tracking, introducing new design proposals, coordination with other groups, and success checks. Reviews are not necessarily formalized and are some- times held in small, informal group meetings. How- ever, the results of reviews should always be recorded in project documentation. Learning from reviews is not limited to the life-time of an R&D project. While a final post-project review closes out the current project, it may also create the grounds for a new project. For instance, after the development of a client=server wholesale banking software product by Unisys, a post-project meeting (post-mortem in their terminology) was called includ- ing the programme manager, the customer project manager, the responsible Unisys International Engi- neering Center project manager, the Unisys Interna- tional Banking Service Center development director, its support manager, and its GUI engineer, the product’s marketing representatives, and the European marketing manager. Based on reviews carried out by the Engineering Center the future benefits of lever- aging the development work, including process improvements for future projects and specific require- ments for product internationalization, the Engineer- ing Center and the Banking Service Center agreed to collaborate on a localization project of the banking software. Here the post-project review was not only a triggering event to identify additional market potential for the just completed development project, but was also a means to help marketing people use engineering information for future project bids. Post-project reviews thus help improve learning from one R&D project to the next. Post-project reviews introduce a systematic way to double-loop learning by making project-specific knowledge and Idea Stage I: Feasibility Stage II: Development Stage III: Test Market Stage IV: Commercialization Review Review Review Review Project Graveyard Figure 2. Stages in the life of an R&D project (source: Balanchan- dra and Brockhoff, 1995, p. 32). Organizational and Technical Knowledge Pool Idea Stage I: Feasibility Stage II: Development Stage III: Test Market Stage IV: Commercialization Review Review Review Review Idea Stage I: Feasibility Stage II: Development Stage III: Test Market Stage IV: Commercialization Review Review Review Review Post-Project Double- Loop Learning Single- Loop Learning Figure 3. Double loop learning through post-project reviews in R&D projects. Table 1. Differences between single- and double-loop learning. ‘Single loop’ ‘Double loop’ Characteristics * Occurs through repetition and routine * Occurs through use of insights and non-routine * Well-understood context * Ambiguous context * Occurs at all levels in organizations * Occurs mostly in upper levels Consequence * Behavioural outcomes * Insights and collective consciousness Examples * Institutionalizes formal rules * New missions and new definitions of direction * Adjustments in management systems * Agenda setting * Problem-solving skills * Problem-defining skills * Development of myths, stories and culture Application in post-project reviews å Discussion of variances in expenditures, missed deadlines etc. å Suggestions for the application of lessons learned to future projects å Retrospective analysis of major obstacles experienced å Deep analysis of cause-effect relations regarding major obstacles experienced Source: adapted from Fiol and Lyles (1985, p. 810). Maximilian von Zedtwitz 258 R&D Management 32, 3, 2002 # Blackwell Publishers Ltd 2002 experience available to a corporate-wide pool of organizational and technical knowledge (see Figure 3). But few companies have achieved double-loop learning with the help of post-project reviews. Post- project reviews can limit double-loop learning if they are too inward-oriented and focused on a reactionary agenda. Table 1 illustrates the differences between what constitutes single-loop and double-loop learning in current PPR practices. Most can be generally described as ‘single-loop’ and are therefore restricting their inherent learning potential. 4. A review of current post-project review practices in R&D 4.1. Results from an exploratory survey on R&D post-project reviews Although most R&D organizations seem to under- stand the potential benefit of post-project reviews, they still do not make full use of this learning opportunity. We conducted two surveys among a group of R&D directors and senior R&D managers on the extent and quality of R&D post-project reviews in which they had participated. The venues were two executive training events on innovation management held at a business school in September 2000 and October 2001. We collected a total of 63 valid responses. This number is too low to warrant in-depth statistical evaluation, but it does allow some interesting observations about the state of R&D post-project reviews. On average, 19.4% of R&D projects in which the respondents had participated had been post-reviewed. Although post-project reviews were conducted in almost every company and within most departments represented, post-project reviews were conducted mostly on an ad hoc basis or after particularly large projects. Only 12 (of 63) respondents indicated that their companies tried to post-review as many projects as possible, and 55.6% of the respondents stated that their companies had not established formal guidelines on how post-project reviews were to be conducted. 17.5% of the respondents indicated that post-project reviews followed similar practices to those employed in earlier projects, and only 11.1% of the respondents’ companies had defined review practices. The most popular means of disseminating know-how between projects was through individuals moving to new projects (52.4%) or through written documentation (39.7%). Just eight of 63 respondents claimed that post- project review results were effectively used to improve project management or were an integral part of inter- project learning. 39.7% stated that their review focused mostly on technical criteria, and just 6.3% found that their companies applied sound and consistent criteria to every post-project review. The complete questionnaire and its results can be found in the Appendix. Our survey confirms a benchmarking study of 79 highly regarded R&D organizations conducted by Menke (1997) who stated that less than a quarter of the 79 organizations made full use of post-project reviews (see also Kumar, 1990). However, we are aware of the limitations of our investigation due to the restricted scope and sample of the survey. But even with these considerations in mind we must conclude from our survey that great learning potential is lost by not taking formalized team learning seriously enough. Among the 63 valid respondents, 59 indicated they would prefer to see more post-project reviews to follow up on completed R&D projects. This is no easy task, as we will see from the following chapter. 4.2. Post-project reviews in industrial R&D What post-project review practices have been estab- lished in R&D-intensive companies, and have any of them best-practice character? Considering the low attention and time that R&D management devotes to post-project reviews, and the multitude of difficulties and obstacles faced in executing these reviews, we were expecting that only a few companies would have developed all-encompassing approaches that could serve as leading examples for their peers. We collected a number of examples that illustrate how some companies have addressed specific problems of post-project reviews. Hewlett-Packard (HP) is one of the companies that has been heralded for its fairly consistent and established implementation of post- project reviews. Some of its business units do not only have official project closure meetings but also ‘retro- spective’ project meetings looking exclusively at the development process and potential improvements. Their main aim is to gather root causes for problems witnessed during the project and then to make recommendations on how such events can be avoided in future projects. This effort is based on an HP-wide project management initiative established in 1989 and later reinforced in a 1993 HP project management council. Since then, HP has been striving to develop project management as a company-wide core compe- tence. This strategic emphasis on project management helped to accommodate post-project reviews despite the additional administrative requirements. Special review facilitators focus on coaching post-project review meet- ings. Still, the frequency of post-project review varies between business units. The need for reviews has not yet trickled down into every R&D department, and the quality and result of each post-review meeting depends on the skills and talents of the review facilitator. Most companies rely heavily on personal networks to disseminate experiences from project management, including all forms of personnel rotation, temporary project assignments and core team hand-overs. Some companies also establish central service centres within their central R&D whose members are in effect human Learning through post-project reviews # Blackwell Publishers Ltd 2002 R&D Management 32, 3, 2002 259 dissemination tools for project experience. Daimler- Chrysler, for example, offers support and coaching staff to its R&D project managers from its FTK=P department. These coaches are not involved in the technical work of the project but support the project team in all questions concerning the project execution and facilitation process. Depending on the size and strategic importance of the projects, these coaches are responsible for up to five projects and are able to capture and re-apply their in-depth knowledge to future assignments and fellow project members. In order to retain and develop a cadre of highly skilled global R&D project managers, Hoffmann-La Roche’s Pharma Division instituted an ‘International Project Management Department’, coordinating a resource pool of about 50 project managers for all R&D projects worldwide (see also Gassmann and von Zedtwitz, 1998). The members of this ‘virtual’, geographically decentralized department are also assigned as managers to projects as part of a global programme to ensure standards in quality and project procedures. Upon completion of these projects, project managers return to the virtual resource pool. Since there are more projects in the pipeline than there are potential project managers available, they are imme- diately reassigned to new projects. Since the director of this department reports directly to the corporate board, the internal political position of R&D project managers has been improved recently. The director of the ‘International Project Management Department’ is also on the International Project Committee, which decides over the approxi- mately 60 global R&D projects at Roche Pharma. The directorassumesaroleasinterpreterorliaison between project managers and top management, thus representing the interests of international project management at Roche. This virtual pool promotes the project management idea. Experienced project managers are dispersed around the world and wander from project to project – no matter where the project will be conducted. Roche thus manages to retain much of the valuable procedural know-how to conduct and lead international projects not only inside the company, but also in a position where it can be reapplied when needed. Although top-management support is important in recognizing the strategic significance of post-project reviews, it is not always beneficial that top manage- ment personally participates in them. Depending on the company culture, the presence of research directors might prevent open and frank discussions of experi- enced problems. DaimlerChrysler, on the other hand, classifies R&D projects into ‘A’, ‘B’ and ‘C’ projects (depending on their budget and strategic importance); ‘A’ projects are reviewed in front of the CTO, ‘B’ projects with the relevant R&D director, etc. However, the higher the hierarchical level of participants, the more likely it is that the post-project review develops into a marketing event rather than an analysis of the finished project. DaimlerChrysler introduced a new review proce- dure in R&D in 1998, giving directions on the format of Powerpoint slides as well as details on mandatory information to be given. Agilent stresses the impor- tance of cross-functional participants to enhance the array of issues raised during the post-review (see also the Unisys example above), underlining that ‘low-key’ events also help to achieve generally more honest results than formal meetings. A similarly informal approach is adopted by many software development companies (e.g., SAP), although some of them regard post-reviews as wild brainstorming sessions without fixed agendas. Professional networks and clubs at Dow Corning and DuPont also provide platforms for informal know-how exchange of project management. Sinofsky and Thomke (1999) have collected a number of hands-on guidelines on how to conduct a post- project review session. Again, the approach chosen depends heavily on the existing company culture and underlying motive for conducting post-project re- views. Some companies insist on maintaining the post- review as a completely internal meeting with no outsiders present. Others, for instance Novartis and Hewlett-Packard, opt for external facilitators in order to profit from the perspectives of an objective outsider to the project. External facilitators may come from an internal coaching or quality management department, like at DaimlerChrysler FTP=K or Agilent, or hired externally to do the job. These outside facilitators are usually consultants specialized in process develop- ment. Obviously, post-project reviews take place after a project is finished. The time span between project closure and the post-review meeting, however, varies considerably. Some companies choose to conduct post- reviews immediately after the delivery of the last project milestone when the memory is still fresh and most project members still present. Other companies prefer to wait until after market introduction, when customer feedback can be taken into account as well. Schindler, for example, waits approximately two years after the actual product launch before conducting the post-project review. While all these examples show that current post- project review practices in R&D are quite hetero- geneous, it does not seem to be appropriate to impose a single best practice of post-project reviewing. Different objectives and needs, different markets and industries, different cultural contexts, and differ- ing degrees of innovation all influence the way post- project reviews need to be conducted. But before we propose a framework to develop better (rather than best) review practices, we must understand some of the underlying resistances to reflective learning for future projects. Maximilian von Zedtwitz 260 R&D Management 32, 3, 2002 # Blackwell Publishers Ltd 2002 5. Resistance to learning from post-project reviews If post-project reviews were simple, few companies would pass up the chance for additional systematic learning. However, a number of factors both at the individual and the group level prevent the free flow of information and the retention of critical project knowledge for future use. Based on research done by Ursula Koners at Cranfield University (see also Koners and Zedtwitz, 2001, p. 128 –131), we identified eight major problem factors, which we grouped in four areas as barriers to learning from post-project reviews (Figure 4): 1. Psychological barriers; 2. Team-based shortcomings; 3. Epistemological constraints; 4. Managerial problems. 5.1. Psychological barriers There is no organizational or team learning without individuals creating and sharing information. All hu- man beings are limited in their capacity to learn based on their experiences. Our minds are programmed through the process of evolution to a certain balance of reflection and action. Given the complexities and challenges of our lives, particularly under high pressures in modern society, it seems understandable that our commitment to reflection on the past – particularly with no immediate benefits to oneself – is rather limited. This mechanism is effective at the conscious as well as the subconscious levels. Here we focus on two impediments pertaining to post-project review management: 1. The disinclination of team members to objectively reflect upon past actions and their consequences, particularly their own actions; 2. The bias to remember easy-to-categorize incidents and the tendency to repress ambivalent experiences. 5.1.1. Inability to reflect. The business life of today is generally not based on reflections of the past. One popular ‘excuse’ is based on the claim that past models and experiences do not apply if circumstances change (Kransdorff, 1996). Most often it is found that managers have little awareness of past actions and rationales. According to Busby (1999) the idea that ‘experience is a teacher in its own right’ is very dominant. In other words, one should not be surprised that people find it difficult to communicate about the past since the past is not the objective of management. Another reason for this inability to reflect is based on the fact that traditional approaches to project management do not treat learning and reflection as central (Ayas, 1997); and consequently these activities are not considered as a vital part of the project management task. In R&D, blue-sky researchers are the ones most likely to engage in learning and reflection for its own sake, but they are in the minority compared to product-oriented development engineers and scientists working on problems in applied research. Once learning and reflection become a means and not an end of a process, they are more easily forgotten. 5.1.2. Memory bias and ambivalent experiences. Be- sides the selectivity of reflection, man is confronted with the selectivity of memory. Psychology and psychoanalysis have made great advances in our understanding of why the human mind remembers certain things and forgets others. Several layers of memory have been identified. Repression is commonly considered to apply to unpleasant experiences, forcing us to forget the actual incident that led to the experience but leaving us with a mental imprint influencing future behaviour. What is the role of repression of group experiences? Unpleasant experiences are not really forgotten and are often very present. However, it is the ambivalent experiences that – both at the individual and the group level – become those memories that pose the greatest Team-based • Reluctance to blame • Poor internal communication Difficult to generalize Tacitness of process knowledge Epistemological • • • Time constraints • Bureaucratic overhead Managerial Inability to reflect Memory bias Psychological Barriers to Learning from Post Project Reviews • • Figure 4. Four major barriers to learning from post-project reviews. Learning through post-project reviews # Blackwell Publishers Ltd 2002 R&D Management 32, 3, 2002 261 threat to the comfort of people, and are hence repressed unless they are (at least at the group level) raised and discussed. Since our world is not black-and- white, most of our experiences are ambivalent. If we were not able to prioritize which experiences to discuss and which to repress, we would spend all day debating and interpreting our experiences and possibly stalling any progress whatsoever. Repression is a natural way of limiting everyday complexity. Repression or selective memory in projects can override potentially important and valuable information for future use. Future project teams, creating a new team spirit and a new moral understanding, may understand this experience in a completely different perspective. It is therefore important to analyse what repressions may have taken place at the group level to carry over any underlying experiences from a previous project. 5.2. Team-based shortcomings While team members are put together to work hand-in- hand towards the same common goal, they are often poorly suited to stand up face-to-face and give each other a piece of their mind. Frank feedback may hurt and potentially affects the relationship between in- dividuals for years: many people choose not to risk their social networks for the sake of project quality. At the same time, much neutral information is not passed on because its importance is undervalued or its perceived validity limited: 1. The reluctance of individual team members to blame other team members or their direct superiors, and to take accountability; 2. Poor team-internal communication and the failure to share different interpretations of common experiences. 5.2.1. Reluctance to blame. One explanation why reviewing past projects does not always lead to a successful learning experience is that it necessarily involves looking back at problems and critical events in the past. Although Gulliver (1987) established in his in-depth case study of the BP organization that people genuinely want to help the company grow more profitable by reviewing past behaviours, reviews often suffer from the reluctance to allocate blame and criticism since they might uncover cynical or embar- rassing events. Unfortunately, this reluctance can be stronger than the realization that the organization has a potential to learn constructively from a project team’s experience (Krans- dorff, 1996). In extreme cases, project members deflect the blame away from themselves, citing unclear goals, insensitive and unfair leaders, and ignorant clients (Barry, 1991). By the same token, individuals are unwilling to take accountability for failure, either because failure may have a negative impact on their career record, or because they are embarrassed to acknowledge failure in front of others. Research in this area is difficult to generalize beyond the particular case, and the degrees of reluctance and accountability probably depend on specific company culture and incentive systems. 5.2.2. Poor team-internal communication. Excellent communication has always been held as critical for effective R&D teams. Tushman (1979), Allen et al. (1980), and Katz and Allen (1982) described how team- internal communication affects the productivity of the R&D function. Imai et al. (1985) or Dimanescu and Dwenger (1996) are mentioned here, in lieu of many others who underlined the importance of cross- functional communication for new product develop- ment teams. While this is a significant step forward, in practice internal communication across functions and groups is hindered by team-internal regrouping: Some people get along better with each other, being able and willing to share information more easily with some than with others. In pathological situations, team- internal factions intentionally hide critical information in order to gain unfair advantage in performance recognition or promotions. Teams separated by physical distances have usually little opportunity to meet in person; they are hence restricted to electronic or written communication. Team members with different technical, functional, or cultural backgrounds do not share the same vocabulary or referential context, which leads to misunderstanding and reduced knowledge exchange. While there are great advantages associated with the inclusion of people from different backgrounds, locations, and cultures, team-internal communication is certainly not enhanced. 5.3. Epistemological barriers Even if human beings had the time and interest to fully devote their attention to the reflection and analysis of what happens around them, they would still find it difficult to grasp the most important issues and draw important conclusions for future behaviour all by themselves. The human intellect is limited. At the same time, certain kinds of experiences are inherently hard to express and hence not easily shared with colleagues. Group-based discussion on commonly shared experi- ence may thus be the only way to bring forth distributed key lessons for future project teams: 1. The difficulty of abstraction from single occur- rences and events to formulate more general recommendations; 2. The inherent difficulty of articulation of tacit knowledge and the precise formulation of ambig- uous assumptions and beliefs. 5.3.1 Difficulties in generalizing from specific projects. Abstract knowledge may be the basis for competitive Maximilian von Zedtwitz 262 R&D Management 32, 3, 2002 # Blackwell Publishers Ltd 2002 advantage if it is well embodied in an organization (Hamel and Prahalad, 1994). Generalizing from context specific project experiences is one of the main hurdles and difficulties for post-project reviews in general. The human mind is not made to abstract experiences to a general level so that they can be applied to a wide range of future projects. Further- more, project results (no matter if they are positive or negative) are often naively extrapolated in a simple linear fashion. The reality, however, is much more complex, so that the outcome of a project depends on a whole variety of interlinked variables which again are very difficult to generalize. 5.3.2. Tacitness of process knowledge. The nature of process-related knowledge, experiences and insights implies that they cannot be shared in the same way as information stored in reports, databases or prototypes. Polanyi (1966) refers to this phenomenon as ‘tacit knowledge’ and describes the underlying problem of organizations: ‘We know more than we can tell’. Durrance (1998) draws from Polanyi, Takeuchi and Senge’s work and claims that especially Western cultures still prefer explicit knowledge, which is quantifiable and definable. Despite increasing aware- ness of the difference between tacit and explicit knowledge, companies are still struggling to convert tacit into explicit knowledge so that it can be shared by the entire organization. Senge (1990) suggests reflec- tion or careful observation – a task mainly found in post-project reviews. 5.4. Managerial constraints R&D is, like any other corporate function, under constant performance pressure. This pressure leaves little time to step back and reassess a project that has occurred in the past. Furthermore, researchers are known to shun guidelines and project management rules imposed by a central department. These require- ments are often interpreted as unnecessary and obstructive to the actual R&D work. Hence we have: 1. Lack of time to deal with the past in a business that typically looks three to five years ahead; 2. Reluctance to comply with bureaucratic and admin- istrative requirements in association with following up on post-project reviews. 5.4.1. Time constraints. The most often stated reason why post-project reviews are not conducted is lack of time. People are unlikely to devote time and effort to yesterday’s problems since natural incentives favour moving ahead to the next problem instead of spending valuable time on reviewing a just completed project. This has been verified by Kotnour (1999) who – based on results from his questionnaire survey of 43 experienced project managers – states that ‘most project managers viewed producing lessons learned as a valuable and important exercise. However, they felt they did not have enough time to complete a formal lessons-learned process’. Particularly in an economy more and more deter- mined by Internet-speed competition, time will be increasingly short. An R&D manager of a large IT company declared that ‘if you have the time to think, you’re doing something wrong’. When time is a critical resource, retrospection and contemplation are left to others. 5.4.2. Bureaucratic overhead. Corporate control sys- tems tend to concentrate on budget, time and output. Not surprisingly, if post-project reviews are conducted at all, a substantial part of each review is devoted to complying with these control demands. Particularly when post-project reviews are considered as additional bureaucratic chores imposed by a central project management department, project teams rarely have the necessary enthusiasm to engage in additional review work. Moreover, the information requested in a post-project review is often incompatible with information solicited during the execution of the project, and hence undermines the confidence of the team in the entire control system. As a consequence, the purpose of a post-project review is often reduced to ensure that the project complied with all bureaucratic procedures, and the important inter-project learning function is neglected. If learning is on the review agenda, it is often for the benefit of the team members only. Kotnour (1999) discovered that the learning goals of project managers focus on costs, scheduling, performance and customer satisfaction – in line with what most corporate incentive systems favour. In combination with time constraints, the disinclination of project teams to comply with bureaucratic procedures tends to keep the effort spent on post-project reviews to a mini- mum. 6. A capability maturity-based model for organizational learning through post-project reviews in R&D There is a growing body of evidence that projects may prove immensely beneficial to the long-term success of companies when these companies systematically in- corporate reflective practices into their project man- agement processes (DeFillippi, 2001, p. 6). In the two previous chapters we have outlined why post-project reviews have been difficult to conduct, and how selected companies have found formalized ways and means to gain insight from R&D projects despite these obstacles. Overall, we have not been able to find a single best practice in post-project reviews in R&D. Individual Learning through post-project reviews # Blackwell Publishers Ltd 2002 R&D Management 32, 3, 2002 263 approaches tend to be company-specific and optimized to problems perceived to be most urgent by each organization. A number of methodologies for structured learning from projects have been formulated to provide R&D managers with practical guidelines (e.g. Collier, De- Marco and Fearey, 1996; Barker and Neailey, 1999; Kearth, 2000). If these methodologies were embraced by R&D organizations, post-project reviews would be conducted with greater consistency and greater applic- ability of their outcomes to other R&D projects. However, the introduction of a new reflective process, which is typically not perceived to add value to current business activities, is resisted unless the organization has reached a level of maturity that is receptive to company-wide knowledge sharing. In the field of software engineering and develop- ment, a capability maturity model (CMM) has been proposed by the Software Engineering Institute to describe the capabilities of software development organizations and to provide guidance on how to establish and improve software development processes (see Carnegie-Mellon University, 1995). Analogous to CMM, we propose a five-level capability maturity model for post-project reviews. Our post-project review maturity model is designed as a framework allowing the progression from an immature unrepea- table review process to a mature, well-managed review process (see Figure 5). The maturity model is organized into five levels: Initial; Repeatable; Defined; Managed; Optimizing. Each level is described in terms of key processes that contribute to the degree of implementation and institutionalization of the review processes in place. 6.1. Level 1: Initial Organizations at this level of process maturity char- acterize post-project reviews as ad hoc or even chaotic. If a review process has been defined, then its success depends heavily on the skills and talent of individuals who conduct it. Reviews are seldom planned but triggered rather in reaction to a major project-related event, such as a complete failure or a particularly successful project. Because the review process is poorly defined and its execution not standardized, its outcome is unpredictable and incomparable to other review results. Most organizations still appear to be at this level. Our survey indicated that one in eight companies never conducts post-project reviews, and that half of the remaining companies use arbitrary or random selection criteria to review projects. More than half of all respondents stated that their companies have no post- project review guidelines in place, and about a quarter responded that the review quality was based mostly on the capabilities of the team members. 6.2. Level 2: Repeatable The organization has established post-project review guidelines and sound review practices: the review process is repeatable and comparable to previous reviews. Managing and planning of new reviews is based on experience with similar reviews, although specific review practices may differ. With principal review policies in place, major disasters in review sessions or in review outcomes may be avoided. A simple example of a repeatable review practice is a post-completion report, which is usually based on a report template to ensure inclusion of relevant project improvement areas. Such written reports, however, focus mostly on quantifiable and explicit data, such as costs, scheduling and technical=design deviations. Nevertheless they provide a good means for transfer- able recommendations for corrective action, if these suggestions are made available at the group or management level. Maturity of Post-Project Review Processes Optimizing Managed Defined Repeatable Initial • Ad hoc PPR • Reaction-driven reviews • Based on capabilities of project individuals 1 2 3 5 • Establishment of PPR policies • Introduction of sound review practices • Based on experience with similar reviews • PPR process standardized • Establishment of sound and consistent review criteria • PPR responsibility assigned to a unit • PPR goals quantified and measurable • Corrective action can be taken • Quality of transferable knowledge predictable Post-Project Review Capabilities 4 • Organization-wide PPR • Consistent inter-project learning • Proactive review of PPR pro- cesses Figure 5. A capability maturity model of post-project review (PPR) processes. Maximilian von Zedtwitz 264 R&D Management 32, 3, 2002 # Blackwell Publishers Ltd 2002 [...]... It can also assist in defining, modelling, and measuring the maturity of review processes used in R&D organizations This model should provide a basis for identifying and communicating good practices and help eliminate bad ones As such, it can be a powerful tool to guide R&Dintensive companies through their development and evolution of inter-project learning capabilities In fact, the post-project review... awards of this kind) Without failure there is little learning: failure must become an accepted part of innovation management 6.5 Level 5: Optimizing In this final level of process maturity, post-project reviews are established organization-wide in order to ensure consistent inter-project learning Lessons learned are disseminated to targeted project teams elsewhere in the organization The post-project. .. obtain the greatest learning for the invested effort and time These organizations ignore that good reviews are those that produce learning for future action, and not for database graveyards 6.3 Level 3: Defined In level 3 the review process is documented, standardized, and integrated in the overall project management process for both management and engineering activities Reviews are conducted according... followed up by a post-project review? Avg.: 19.4% 1c) Would you encourage your organization to conduct * more post-project reviews? 93.7% * fewer post-project reviews? 1.6% * the current amount is acceptable 4.8% 2) In your organization, which one of the following statements applies? * Post-project reviews are conducted only in R&D 16.4% * are conducted mainly in R&D 34.4% * are conducted in R&D as well... Publishers Ltd 2002 Learning through post-project reviews 4) How are post-project reviews managed in your R&D department? (check all that apply) * There are no established post-project review guidelines 55.6% * Post-project reviews are conducted by external facilitators 9.5% * A department or unit has been assigned with post-project review responsibility 4.8% * Post-project review guidelines are ‘available... Transactions on Engineering Management, EM-27, 1, 2 – 12 Argyris, C (1977) Double loop learning in organizations Harvard Business Review, September –October, 115–125 Argyris, C and Schon, D.A (1978) Organizational Learning ¨ London: Addison-Wesley Ayas, K (1997) Design for learning and innovation Long Range Planning, 29, 6, 898– 906 Ayas, K and Zeniuk, N (2001) Project-based learning: building communities... Cooper, R.G and Kleinschmidt, E.J (1991) New product processes at leading industrial firms Industrial Marketing Management, 20, 137– 147 Crossan, M and Guatto, T (1995) The evolution of organizational learning Working paper series no 95 – 07, Western Business School Daudelin, M.W (1996) Learning from experience through reflection Organizational Dynamics, 24, 3, 36 – 49 DeFillippi, R (2001) Introduction:... that constitute each level 7 Conclusions Mistakes are only worthwhile making if you can learn from the experience In this contribution we have underlined the importance of post-project reviews for organizational learning, outlined some impediments to conducting post-project reviews, and proposed a model for assessing and improving post-project review practices We have given only limited practical advice... whole exercise is, in fact, wasted Electronic review documents in central data repositories and reports in company-internal newsletters are outputs of but not inputs to projects Reviews must be conducted with targets in mind For instance, project managers of selected upcoming projects should attend post-project reviews as observers Project management coaches who assist in all process and post-project phases... repeatable Major savings in cost and time, as well as quality improvements are possible In order to achieve a repeatable and defined review practice, team members must be trained in maintaining and executing project activities that improve the postproject review exercise at the end Both the military service and industrial research departments are used to maintaining a logbook, recording important events . managing post-project R&D reviews. 3. Post-project reviews in organizational learning and R&D management 3.1. Defining post-project reviews According. different reviews over the course of an R&D project. Reviews Individual Learning Team/Group Learning Organizational Learning Levels of LearningHow does learning