Exploring Values and Norms of Engineering Through Responsible Innovation and Critiques of Engineering Cultures

Paper ID #34487 Exploring Values and Norms of Engineering Through Responsible Innovation and Critiques of Engineering Cultures Dr Rider W Foley, University of Virginia Dr Rider W Foley is an assistant professor in the science, technology & society program in the Department of Engineering and Society at the University of Virginia He is the principal investigator at University of Virginia on the ’4C Project’ on Cultivating Cultures of Ethical STEM education with colleagues from Notre Dame, Xavier University and St Mary’s College He is also the co-leader of the ’Nano and the City’ thematic research cluster for the Center for Nanotechnology in Society at Arizona State University Rider is a Research Collaborator with the Sustainability Science Education program at the Biodesign Institute His research focuses on wicked problems that arise at the intersection of society and technology Rider holds a Ph.D in Sustainability from Arizona State University, and a Master’s degree in Environmental Management from Harvard University and a Bachelor’s degree in Environmental Science from University of New Hampshire Before earning his doctorate, he has worked for a decade in consulting and emergency response for Triumvirate Environmental Inc Rachel Sinclair, University of Virginia Rachel Sinclair is a graduate with a Master of Public Health and Bachelor of Arts, major in Psychology, from the University of Virginia She is beginning her professional career as an Associate Clinical Research Coordinator at the Mayo Clinic Prior research experience has involved neurodegenerative disorders, pathogens, mental health outcomes and policies, and engineering ethics education Araba Dennis, Purdue University Araba Dennis is a second-year PhD student studying race, culture, and institutional definitions of inclusion c American Society for Engineering Education, 2021 Exploring values and norms of engineering through responsible innovation and critiques of engineering cultures Abstract Engineers are often taught that ethics entail the adherence to a code of conduct, which can guide their professional behavior Alternatively, engineers may learn that research ethics are represented by the principles of Responsible Conduct of Research Both of these approaches ask for engineers to learn, accept and conform to values that are instantiated by external organizations This is intended to support an individual’s decision-making in the face of discrete moral or ethical quandaries Prior scholarship by Donna Riley, Erin Cech and Amy Slaton offer critiques of engineering culture that point to the myth of objectivity, reductionism, and grit, while underscoring an uncritical acceptance of authority Cech’s work on the “culture of disengagement” and the pillars of depoliticization, socio-technical dualism and meritocracy demonstrated shifts in values among students at four very different universities in Massachusetts Out of political science and technology assessment the concept of responsible innovation is gaining traction as a means to bring values and norms into the center of innovation activities, exemplified in works by Rene von Schomberg and David Guston However, the critiques of engineering cultures are often absent from concepts of responsible innovation For if responsible innovation is a normative shift towards a better future, then understanding what is undesirable about the cultures of engineering and innovation must be equally important This research explores how engineering students at a university in Virginia express values and norms when asked the question: “What is engineering?” The research design captured long-form essays prior to and after taking an engineering ethics course Those essays were coded thematically for dimensions of irresponsible innovation, e.g., myth of objectivity, depoliticization and reductionism, as well as for dimensions of responsible innovation including stakeholder engagement, future anticipation of consequences, and adaptiveness Methodologically, the results suggest that long-form essays, in addition to the surveys used by Cech and the case studies offered by Riley, offer an intriguing method to analyze the emergent values and norms among engineering students Secondarily, the empirical results suggest subtle shifts in the discourse about what engineering is and, thus recognition of values that might underpin cultures of responsible innovation Keywords: Content Analysis, Engineering Education, Engineering Ethics Introduction Engineers are often taught that ethics means the adherence to codes of conduct, which offer guidance for handling difficult situations as professionals On the other hand, some engineers learn about the principles of Responsible Conduct of Research and the rules that determine good behavior [1] Both of these approaches ask for engineers to learn, accept and conform to the values instantiated by external organizations Those approaches are intended to support an individual’s decision-making in the face of discrete moral or ethical quandaries Yet, prior scholarship by Joseph Herkert [2] suggests there is a multi-layered set of ethical obligations that range for microethics––or individual decisions––to macroethics, which reflect the professional society’s values and ethical obligations Macroethical dilemmas result in the “problem of many hands”, as described by van der Poel and Royakkers [3] This brings to light the notion that individuals or even large organizations are not solely responsible for engineering processes and uncertain outcomes For it is clear that no individual or discrete organization has complete control and authority for the complex socio-technical innovation process from design to implementation, nor for the maintenance and disposal of engineered systems [4-6] The challenges associated with such a lack of complete control or authority are often handled in one of two ways that align with distinct philosophies of engineering The first of these is the pursuit of greater control and authority as exercised by myriad forms of power, which is much discussed in engineering ethics and manifests in multiple forms Scholars focused on the study of engineering practices have detailed how the elusive sense of control can create even greater vulnerabilities when macroethical challenges arise As Bryan Newberry [7] detailed in the case of New Orleans and Hurricane Katrina the desire for greater human control may very well have contributed to many of the compounding errors that yield catastrophic results More recently, a second approach has emerged in the teaching and scholarship and stands in stark contrast to aspirations for greater control This approach seeks to foster reflexivity and learning about one’s own context and broader societal implications of engineering practice Robbins [8] offers the notion of the “reflexive practitioner” as an emergent theme in engineering ethics However, there are few examples for how such reflexivity can be demonstrated in the education and maturation of engineers This project aims to address that knowledge gap in a small, but important way, by assessing reflective writing by engineers in an undergraduate program This paper offers data from 65 students that wrote 1-page essays in response to the question: “What is engineering?” The essays were analyzed using the mirrored concepts of (ir-)responsible innovation, which are reviewed in the next section The results offer but one example of how reflective writing can offer a means to assess students’ learning outcomes The discussion returns to the novelty of the mirrored concept of (ir-)responsible innovation and offers pathways forward for future research to address the many limitations of the research design for this study, namely the single-university focus and small sample This paper aims to demonstrate an assessment technique that moves from rule-following ethics education to an approach that represents the students’ values and beliefs Conceptual Framing The following review of literature serves as an overview of the conceptual foundations that inform the (ir-)responsible innovation as a framework for analysis Joseph Herkert’s [2] discussion of microethics and macroethics provides a foundational lens for understanding how students might become oriented towards expressions of responsible innovation Herkert explains this distinction as microethics being primarily, “concerned with ethical decision making by individual engineers and the engineering profession’s internal relationships ” [2, p 374] While microethical decisions are those that are generally considered to be “the right thing” with regard to one’s particular duties or responsibilities, they fail to consider the organizational or social conditions that engender the need for such individual decisions Conversely, macroethics revolves around the “collective social responsibility” and “societal decisions about technology” relating to the engineering profession as a whole Stressing a macroethical approach to engineering pedagogies allows engineering students to consider responsible innovation as a conduit for fitting individual subjectivity onto a larger social context While this micro and macroethical framing sets the stage for this research, it is not used directly for analysis Irresponsible innovation: Foundations and facets Donna Riley’s book, Engineering and Social Justice, builds upon Herkert’s proposed framework of macroethics and how it relates to an established engineering culture Riley’s [9] critiques of engineering culture take a normative position on the values and principles that inform engineering practice, which represents an important shift in the field Prior work on engineering cultures, led by Gary Downey and others, offers a detailed and descriptive analysis of engineering cultures and how nationalism and shared identity were important aspects that differentiated German, French and English cultures of engineering [10] Riley [9] shifts away from such a nationalistic perspective and instead focuses on six main characteristics of engineering culture that appear to transcend nationalistic differences The first is a positivist epistemology that is centered upon a myth of objectivity, such that engineering can be performed without any inputs based upon subjective knowledge and the flaws associated with that way of understanding the world This myth builds upon a commitment to problem solving that aims to reduce complex social problems into discrete technical components and then focuses on solving those technical problems Such a reductionist approach is typified in Alvin Weinberg’s essay that was reproduced and contextualized as a core lesson for engineers [11] Riley’s book on social justice and engineering highlights the narrow focus on technical skills and competencies centered around the “hard” sciences and mathematics as a way of reinforcing this principle Regarding the career orientation of aspiring engineers, there is a long tradition of engineering as it relates to the military, with Virginia Military Institute being one of the first secondary schools with a dedicated focus on engineering [10] That tradition is alive and well, yet there is an emergent emphasis on corporate organizations and partnerships that fund engineering research at universities and offer clear career pathways for university graduates What Riley [9, p 39-40] strongly critiques is how this facilitates an uncritical acceptance of authority and foregrounds profit-oriented engineering with economic calculations as the only proxy for social acceptability The “rule-following” ethics education offered in many universities places an emphasis on the individual’s decisions and de-emphasizes the organizational impacts and broader implications of any given design or project As Jessica Smith points out in her research at the Colorado School of Mines, the ethics associated with material provisioning is often used to justify (or set aside) local concerns about the impacts of mining and other processes that extract natural resources [12] Those impacts are often deemed as hyperlocal and offset by the materials provisioned to the global community, such as gasoline or plastics from petroleum extraction Cech [13] provides evidence of a “culture of disengagement” that uses longitudinal survey data from four different engineering education programs She offers three main “ideological pillars” as the following beliefs: (i) an “ideology of depoliticization” wherein science and engineering are “pure” spaces free of social and cultural concerns, (ii) a “technical/social dualism” wherein technical knowledge and competencies have more value than social ones, and (iii) a “meritocratic ideology” wherein scientific systems are unbiased, with fair systems of advancement [13, p 45] Amy Slaton [14] builds upon these notions and offers additional evidence that links the premise that technological decisions are not influenced by political power and structural relationships with the ideals of individual achievement Slaton [14] argues that the neo-liberal ideals of meritocracy and technocracy reinforce bias against nonWhite and non-male persons, thus perpetuating historical and cultural inequities that track to racial and gender identities It is not surprising that those three scholars collaborated on a recent book chapter that nicely summarized and synthesized their research efforts [15] That work highlights the concept of grit or persistence as foundational to the experience of undergraduate engineers, whose educational journey is often described with analogies like “gauntlet” or “trial by fire” This enculturation process is underpinned by the ideals of meritocracy, yet yields inequitably outcomes for minority populations, as seen in the admissions and graduation data that are routinely cited in reports on the “leaky pipeline” in STEM education Together this critically reflexive scholarship on engineering education and culture offers a conceptual foundation of irresponsible innovation in engineering, which includes the following facets: Positivist epistemology and the myth of objectivity Commitment to problem solving based on reductionism (reductionist thinking) Persistence or “grit” as a personality trait Centrality of military and corporate organizations and decision-making that privileges command and control models of authority Technocratic or narrow technical focus regarding decision-making and design Uncritical acceptance of authority and rule-following behaviors Responsible Innovation: Foundations and Facets The roots of responsible innovation extend into the philosophies of deliberative democracy Richard Sclove’s book Technology and Democracy offers an entry to the relationship between deliberative democratic theory, technological designs and decision-making Attempts to apply deliberative democratic theories to the research and development of technology have been experimented with around the world with examples readily apparent in the Netherlands [16], France [17], the United States [18], Brazil [19], India [20] and Japan [21] Those experiments were seen as a means to address the negative consequences and perceptions that can accompany the emergence of novel scientific and engineering research associated with nuclear energy and waste [22], genetic engineering [23], and more recently nanotechnology [24] Recently, Rene von Schomberg’s book chapter [25] offers a conceptual definition of responsible innovation that many scholars reference in order to anchor (or critique) the notion The phrase most frequently pulled from that book chapter is, “A transparent, interactive process by which societal actors and innovators become mutually responsive to each other with a view to the (ethical) acceptability, sustainability and societal desirability of the innovation process and its marketable products (in order to allow a proper embedding of scientific and technological advances in our society)” [25] von Schomberg’s work is important as it expresses an explicitly normative position and, yet offers deliberative democratic theory as a means to arrive at the “good” outcomes desired from innovation Building upon that book chapter, Stilgoe and colleagues [26] offered four dimensions of responsible innovation The first dimension offered was anticipation that can be briefly defined as a predisposition towards the future that explores desired outcomes and associated uncertainties that are plausible through foresight methods [27,28] Second was reflexivity, a form of selfcritique that can attend to the individuals, organizations or societal structures that are supporting (or constraining) innovation activities [8, 29] This can be achieved through various methods that are internal to the individual or organization or facilitated by external means [30] Third is inclusion (or inclusive stakeholder engagement), which draws from Brian Wynnes [31] research that demonstrated that stakeholders possess different forms of knowledge and that a greater diversity of knowledge can avoid devastating outcomes Andy Stirling [32] and Jason Chilvers [33] scholarship articulates the demands for inclusive stakeholder engagement, while Frewer and Rowe [34] articulate criteria for assessing the inclusivity of stakeholder participation This framework is activated by the notion that stakeholders need to be responsive to the knowledge acquired from future-oriented, self-critical and diverse stakeholder perspectives They argue that it is not enough to conduct foresight activities and anticipate desired and undesired futures, but the important next step is to take actions, adopt rules or make other changes based upon that new knowledge Foley and Gibbs [35] review of prior literature aligned four additional dimensions of responsible innovation with engineering practice The first being systems thinking, which requires engineers to consider the future implications of their designs, as well as how a novel technology will create social and environmental implications from the extraction of resources to the ultimate disposal or recycling [36] By taking a systems approach, engineers need to work in coordination with diverse stakeholders and recognize and work to negotiate trade-offs between the technical designs and the demands of other stakeholders [37] The coordination and conflict negotiation needs to be conducted in a manner that is transparent to the involved stakeholders [38,39] The intended outcome is to design and process technology that contribute to human flourishing and social justice [40, 41] Research Design The concepts discussed in the literature review demonstrate a shift from individual responsibility to the collective responsibility Further, the facets of irresponsible innovation and dimensions of responsible innovation offer an analytical framing for this research Scholars faced with the challenge of conducting research on responsible innovation often look to discourse and content analysis as a means to glean insights about the perceptions and beliefs of individuals and discrete social groups [42] This informs the research question and methods of analysis for this research We ask: How is (ir-)responsible innovation represented in the writings of undergraduate engineers? Course context: Bringing science, technology and society into engineering education The University of Virginia, has structured course requirements to ensure that all engineering students will graduate with the technical knowledge of their field, accompanied by an understanding of the social context, engineering ethics and the contemporary issues in the field Students are required to take four courses in science, technology, and society (STS) The first course is taken during undergraduates’ first semester as an introduction to the relationship between engineering and society The second required course is at the 2XXX or 3XXX level (second or third year), and students can select from the course catalog that addresses a number of topics such as, data ethics, entrepreneurship, laboratory life, for example These courses use approaches aligned with the humanities and social sciences to further investigate the social and ethical issues related to engineering and engineered artifacts In their fourth-year all engineering students take a yearlong course sequence in both their fall and spring semesters This is where they learn about STS theories, consider various ethical frameworks and apply these concepts to their own research topics A graduation requirement is for all students to generate a written portfolio that includes a report on their technical capstone project and STS research paper that addresses aspects of engineering and the social and ethical context This curriculum has been previously recognized by the National Academies of Engineering as exemplary [43] The in-person course uses activity-based learning that is student-oriented, such activities include worksheets, concept mapping, and role playing, for example The students are introduced to core concepts from science, technology and society (STS) and then readings on responsible innovation are brought into the course The classroom activities involve stakeholder mapping, analysis of case studies and activities designed to demonstrate how the STS frameworks can be used to assess socio-technical systems For example, one activity involves using actor network theory to analyze the relationships between electronic healthcare records within the healthcare setting This activity is performed in small groups and then students share their analysis and co-construct a large network as a full class This activity demonstrates how a single technology is related to the broader network of human and non-human actors and the ways that power and authority are structured Another activity takes up Winner’s [44] paper and students consider how the designs of bridges in Long Island and the tomato harvester designed for the central Valley in California exerted power through the designs of the technology In small groups, the students visualize the design elements of the tomato harvester and then connect them to the designers, funders, land-owning farmers and labors A reading of Pinch and Bjiker’s [45] about the historical influence of stakeholders on the design of bicycles helps students understand how the physical infrastructure, values and preferences and cultural norms informed the design of the “safety bike.” Those lessons are then carried forward with an exercise that takes focuses on five different bicycle designs: mountain bikes, electric bikes, carbon fiber racing bike, beach cruiser bikes and children’s bicycles That secondary activity encourages the students to take the historical lessons from Pinch and Bjiker [45] and to apply them to contemporary designs of the same technology Those activities not inform the results directly, rather they serve to provide greater context for the educational intervention in question Data Collection and Thematic Coding Students were invited to participate in a research study to evaluate their representations of engineering and demonstrate their values of engineering and engineering education No incentives were offered to the participants and all students in the course were required to complete the same amount of work Per the requirements set forth by the School of Engineering, the instructor was not present during the recruitment for the research study The assignments that served as the data for this study took the form of two long-form essays At the start of the fall semester on the first day of class all students were assigned a long-form essay (1-page) that would be graded pass/fail The prompt for that essay was, “What is engineering?” At the conclusion of classes in December, the same assignment was given Those two essays comprise the pretest and posttest for this research study The participants two writing samples offer the evidence that is brought to bear on the research question The population of students solicited for participation was 72, of which 65 gave their consent and completed at least one of the requested tasks The remaining students either did not consent or did not complete both the pretest and posttest required tasks for full participation The pretest and posttest yielded approximately 130 pages of writing for analysis The writing samples were cataloged before being uploaded into Dedoose®, a software program that supports thematic content analysis The literature review above was converted into a thematic coding schema The thematic codes were clustered into two groups based upon their normative orientation: Irresponsible and Responsible Innovation, see left-hand column of Table below The analytical categories were then broken into two tiers based upon the theoretical framing suggested by the prior literature introduced earlier This coding schema was first proposed and explored by Araba Dennis, and presented at the Forum on the Philosophy of Engineering and Technology in 2018 After a process of iterative reflection and refinement, the research team settled on the themes offered in Table 1, below Once the analytical categories were finalized, the writing samples were analyzed by two graduate student researchers When coding the student writings with this analytical framework, the researchers looked for specific words, phrases or references to the thematic codes One researcher first went through each sample and highlighted relevant excerpts and assigned codes The second researcher reviewed these annotated documents, noting new excerpts and adding additional codes to those that fit into multiple frameworks Agreement, disagreement or uncertainly in regards to the previous notations were also marked After a secondary review of the thematic codes, both researchers went through each writing sample in Dedoose® to annotate the excerpts and codes for ease of analysis The final codes entered into the software were based upon the researchers’ previous analysis and the secondary review This process resulted in consensus coding of the data The lead author, Rider Foley, did not participate in the application of codes, but rather consulted when debates arose between the two researchers conducting the thematic coding The primary reason for this was due to unconscious bias of the lead author, who was the instructor of the course and recognized that they would read the essays with a more interpretive lens Secondarily this offered to two graduate research assistants an opportunity to practice and learn about this methodology Statistical analysis Data was collected from 65 participants in both a pretest and posttest for a total of 125 records Four participants completed only the pretest and one participant completed only the posttest This resulted in the two researchers creating 641 excerpts for the 60 pretest and posttest records Each individual record had between and 27 codes applied, and the mean number of codes applied per record was 14 with a standard deviation of 4.3 To investigate the difference in thematic codes applied to the pretests and posttests, the rate of codes applied across all excerpts was compared for individual codes as well as by two broader categories, as shown in Table Two different statistical tests were run to analyze the significance of the changes The first was a paired t-test that compared only the records of the 60 participants that completed both the pretest and posttest Secondarily, a two-sample t-test was used to test for significance among the aggregate pretests and posttest Since the number thematic codes applied were different in the pretest and posttest treatments both the total counts and occurrence––i.e., presence or absence–– of the codes applied were analyzed in both the paired and two-sample t-test For example, for the code reflexivity, tests were conducted based upon the total number of times the code was applied across all excerpts which is reported as mean count and tested with both the paired t-test and a two-sample t-test for significance A second statistical test was run on the binary variable of occurrence (presence or absence) of the thematic code within each record (meaning within each student writing sample) So, if a student’s writing was coded for reflexivity at least once, then it would count as and if no occurrences of reflexivity were identified by the researchers, then it would count as This binary representation of the data was also tested with the paired t-test and the two-sample t-test Tests were conducted on the total number of aggregate count of codes across all excerpts, as well as for a binary representation of the presence or absence of those code For both the paired and two-sample t-tests, an α = 05 threshold was used to determine significance of results The co-occurrence of thematic codes applied to the students’ writing was extracted from Dedoose® to give the research team an understanding of the relationship between the applied codes The results are then enriched and interpreted in a more qualitative manner by looking at specific excerpts and the thematic codes applied, as a means to interrogate the method and explain the results further Outcomes Normative Orientation Tier Codes (Parent Codes) Positivist Epistemology / Myth of Objectivity Commitment to Problem Solving via Reductionism Persistence/ Grit Irresponsible Centrality of Military and Corporate Organizations Innovation Technocracy Narrow Technical Focus Uncritical Acceptance of Authority Human Flourishing Ecological Integrity Responsible Innovation Dispute Resolution/ Negotiating Trade-Offs Livelihood Opportunity Procedural Transparency Socio-technical Connections Systems Thinking Coordination Stakeholder Engagement Anticipation Reflexivity Responsiveness Table Thematic codes applied to the writing samples Results This research offers two key findings from this analysis of the students’ discourse on engineering First, this exploratory project demonstrates that the reflective writing assignment coupled with a coding schema for both responsible and irresponsible innovation offers a method for assessing the change in discourse on the ethics and values of engineering Second, the quantitative results demonstrated a shift from discourse from irresponsible to responsible between the pretest and posttest writing samples The qualitative evidence makes transparent how the codes were applied and further suggests that some students layered knowledge from the course onto their existing understanding of engineering, while other excerpts articulate the nature of that shift in the discourse While the scope and boundaries of this study has limitations, this exploratory project generates productive questions and offers a path forward for future research Quantitative results The students’ writing samples demonstrated statistically significant shifts in both of the major categories when analyzed in in the aggregate and as binary (presence / absence) variables The mean aggregate score in the posttests for codes that were thematically aligned with irresponsible innovation were lower than in the pretests when analyzed with both the two-sample t-test and the paired t-test, see Table and below While there is no difference in the mean pretest or mean posttest between the two-sample t-test and paired t-test, there are subtle differences in the associated p-values This is due to the very small difference between total participants and paired participants Inversely, the mean aggregate score for thematic codes associated with responsible innovation were higher in the posttest On average, fewer statements associated with the core concepts of irresponsible innovation are present in the essays written at the end of the course, while more statements align with responsible innovation appear in the posttests When the application of codes was converted to a binary variable, the changes in pretests and posttests were still statistically significant for the two main clusters using both the paired t-test and two-sample t-test, as shown in Table and This suggests that the educational intervention induced a near-term changes in the discourse of engineering students about the values associated with their profession Aggregate Two-Sample T-test Mean Pretest Mean Posttest Significance P-Value Clusters – Irresponsible Innovation 2.867 1.700 Significant 6.16E-20 1a – Corporate Organizations 0.167 0.083 Significant 1.49E-02 – Responsible Innovation 2.400 5.983 Significant 5.86E-20 2a – Outcomes 0.217 0.200 Not 3.58E-01 2b – Procedural 1.667 4.567 Significant 2.16E-02 Table Two-sample t-test of clusters analyzed by aggregation Note: Cells colored red indicate a lower mean posttest, while green indicates higher mean posttest Aggregate Paired T-test Mean Pretest Mean Posttest Significance P-Value Clusters – Irresponsible Innovation 2.867 1.700 Significant 1.75E-05 1a – Corporate Organizations 0.167 0.083 Not 1.99E-01 – Responsible Innovation 2.400 5.983 Significant 2.55E-09 2a – Outcomes 0.217 0.200 Not 8.64E-01 2b – Procedural 1.667 4.567 Significant 1.70E-09 Table Paired t-test of clusters analyzed by aggregation Note: Cells colored red indicate a lower mean posttest, while green indicates higher mean posttest Binary Two-Sample T-test Mean Pretest Mean Posttest Significance P Value Clusters – Irresponsible Innovation 0.950 0.767 Significant 8.85E-03 1a – Corporate Organizations 0.117 0.050 Not 3.22E-01 – Responsible Innovation 0.783 0.983 Significant 1.76E-03 2a – Outcomes 0.167 0.133 Not 7.98E-01 2b – Procedural 0.733 0.967 Significant 8.89E-04 Table Two-sample t-test of clusters analyzed as binary Note: Cells colored red indicate a lower mean posttest, while green indicates higher mean posttest Binary Paired T-test Mean Pretest Mean Posttest Significance P Value Clusters – Irresponsible Innovation 0.950 0.767 Significant 1.69E-03 1a – Corporate Organizations 0.117 0.050 Not 1.59E-01 – Responsible Innovation 0.783 0.983 Significant 9.10E-04 2a – Outcomes 0.167 0.133 Not 5.68E-01 2b – Procedural 0.733 0.967 Significant 6.19E-04 Table Paired t-test of clusters analyzed as binary Note: Cells colored red indicate a lower mean posttest, while green indicates higher mean posttest The students’ writing samples were coded with every thematic code offered by the theoretical framing, see Table The most frequently applied code was the commitment to problem solving (via reductionism) and is consistent with many critiques of engineering culture Yet the mean posttest application of that code decreased from 1.62 per writing sample to 0.78 in the posttest and that change was significant And while a commitment to problem solving was still evident in the discourse, there was less of an emphasis on it as the primary defining characteristic The second most frequently applied thematic code was reflexivity, which should not be surprising, as the activity was intended to induce reflection on the question, “What is engineering?” The application of reflexivity changed from a mean pretest score of 0.5 to a mean posttest score of 1.45 and that change was significant as well The students’ writing samples explicitly discussed the reflections upon the practice of engineering with far greater frequency in the posttest Five other thematic codes associated with responsible innovation were more frequently coded in the posttest including anticipation, stakeholder engagement, human flourishing, dispute resolution and socio-technical connections These exploratory results are encouraging and suggest that this method of data collection and analysis yield significant results Further, this finding suggests that there was a demonstrable cognitive shift in the discourse on engineering practice The nature and size of the sample preclude further claims (see Limitations 5.1) The infrequent code application after the top ten and limited sample size contributes to the lack of significance for other codes, yet patterns are clearly evident and the aggregate scores above reinforce those trends 10 11 12 13 14 15 16 17 18 19 Code Commitment to Problem Solving Reflexivity Anticipation Stakeholder Engagement Human Flourishing Dispute Resolution /Negotiating Trade-Offs Narrow Technical Focus Socio-technical Connections Responsiveness Positivist Epistemology/ Myth of Objectivity Ecological Integrity Technocracy Coordination Persistence / Grit Centrality of Military and Corporate Organizations Systems Thinking Transparency Livelihood Opportunity Uncritical Acceptance of Authority Freq 148 123 73 70 54 Pretest Mean 1.62 0.5 0.22 0.18 0.28 Pre SD 1.15 0.65 0.56 0.43 0.61 Posttest Mean 0.78 1.45 0.97 0.58 Post SD 0.69 1.14 0.94 0.96 0.77 Significance Significant Significant Significant Significant Significant 53 44 42 30 0.23 0.33 0.15 0.22 0.59 0.63 0.44 0.49 0.63 0.33 0.52 0.28 0.78 0.63 0.91 0.58 Significant Not Significant Not 0.002 0.007 0.497 25 21 18 18 16 0.22 0.2 0.18 0.2 0.17 0.49 0.51 0.43 0.44 0.38 0.2 0.13 0.12 0.18 0.1 0.61 0.43 0.37 0.43 0.48 Not Not Not Not Not 0.859 0.437 0.35 0.837 0.398 12 0.08 0.1 0.02 0.02 0.02 0.33 0.3 0.13 0.13 0.13 0.08 0.03 0.12 0.07 0.38 0.18 0.56 0.31 Not Not Not Not Not 0.159 0.159 0.26 0.321 P Value 1.58E-06 2.73E-07 2.59E-07 2.59E-07 0.021 Table Individual Thematic Codes The thematic codes are listed from most frequently applied code to least frequently applied code The mean pretest and standard deviation and the mean posttest and standard deviation are listed, along with the p-value for this two-sample t-test Many of the excerpts from the students’ writing samples were coded for more than one theme and, thus we wanted to explore the co-occurrence of codes Co-occurrence is defined simply as a writing sample with more than one thematic code applied and the frequency of the co-occurrence is shown below in Table The codes with the highest levels of co-occurrence offer evidence for the concepts that were closely related in the students’ writing Reflexivity was coded with a high level of co-occurrence alongside of commitment to problem solving and stakeholder engagement This suggests that the students were explicitly reflecting on the commitment to problem solving and stakeholder engagement Another frequent co-occurrence is observed between anticipation and dispute resolution/ negotiating trade-off, which demonstrates a cognitive link between those two dimensions of responsible innovation Students were considering how anticipation might identify potential conflicts and negotiate future trade-offs earlier in the engineering process These results should not be overstated, due to the limited scope and boundaries of this exploratory research, but they illustrate a second-level of analysis that reveals aspects of the cognitive alignment of different concepts In the next section, we highlight some examples of the student writing and the codes applied to give a richer and more descriptive look into the student writing samples A A Centrality of Military & Corporate B B C D E F G H I J K L M N O P Q R S 0 0 0 1 0 0 0 0 20 0 1 0 1 1 0 0 0 0 1 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 10 1 4 2 2 16 0 1 0 0 19 11 1 1 2 0 20 1 Commitment to Problem Solving C Narrow Technical Focus D Persistence E Myth of Objectivity F Technocracy 0 1 G Uncritical Acceptance of Authority 0 0 0 H Dispute Resolution 0 0 I Ecological Integrity 0 0 0 J Human Flourishing 1 1 0 K Livelihood Opportunity 0 0 0 0 2 L Anticipation 0 16 M Socio-technical Connections 0 0 0 1 N Coordination 0 0 0 0 1 O Reflexivity 20 3 10 16 19 P Responsiveness 0 1 0 0 Q Stakeholder Engagement 1 0 9 11 20 R Systems Thinking 0 0 0 0 1 0 S Transparency 0 0 0 0 1 Totals 58 18 10 19 52 20 52 75 13 11 111 17 70 Table Code Co-Occurrence Excerpts that were coded with two or more thematic codes were analyzed for the co-occurrence of codes Please note that the name of the thematic code is written in the left-hand column, while an alphabetical letter is used in the column The codes are listed across the top in the same (mirrored) order as they are listed on the left-hand side Blue indicates low cooccurrence (1-5 co-occurrences), while green indicates medium (6-15 co- occurrences) and red signals the most frequent cooccurrence (>15 co-occurrences) Qualitative findings The most frequently applied thematic code was commitment to problem solving and that was associated with an approach to engineering that depends upon using math and science to break down complex problems into smaller and more controllable elements To illustrate how this code was applied some excerpts were selected “Engineering can briefly be described as the application of math, science, and technology to develop an understanding of the world around us and to design solutions to society’s problems, but this is not a complete definition.” Student 17 “Engineering is the application of math and science to solve problems of any magnitude in the real world.” Student 32 “In simple terms, engineering is an approach that applies mathematical and scientific principles to solve problems.” Student These excerpts are straightforward and articulate the relationship between math and science as a means to solve societal challenges through the application of that knowledge The definitions suggest that this approach can be universally applied from “any magnitude in the real world” to “design solutions to society’s problems.” In many instances, students expressed engineering and problem solving as inseparable “Engineering is an art of problem solving.” Student This suggests a high level of confidence in engineering to address problems of any scale and affect change in society Implicit in these excerpts is the linear model of science – engineering – societal change with science and math coming first and then engineering applying that knowledge to affect change in society Closely connected to this thematic code, but identified separately, and less frequently was Technocracy and the belief that persons with technical expertise are affecting societal change “I believe that without engineers, research would stop completely and society would barely function and no longer be productive.” Student 34 Thus, the definitions of engineering were not only informed by reductionism, but also by the linear model of technological innovation as discussed by Godin [46] Statements about anticipation addressed the future implications of technology and also considered alternative futures Many students expressed concern for the future and identified unintended consequences as something that warrants careful consideration in the design phase of engineering, prior to the final construction and implementation of engineering designs at scale Some students went as far as to state that a lack of foresight about the consequences is simple, “bad practice” and thus directly hints at irresponsible innovation “…it is incredibly important to consider in order to understand past, present, and future implications of technologies’ applications.” Student 14 “[Engineers]…in the design or production stages must keep in mind that their solution may have unintended consequences.” Student 23 “At the same time people solving these problems using engineering techniques must take into account the social implications As many solutions can have consequences beyond what was initially intended.” Student 26 “To solve a problem without thinking about the consequences of the proposed solution is bad engineering practice at best and potentially destructive in the worst case.” Student 34 Taken together, it is clear that students were grappling with how novel technologies and engineering analysis may shape alternative futures Further, these statements suggest a cognitive awareness of the mutual shaping forces between future social conditions and the functionality of technology These two examples showcase how the thematic codes were applied to explicit statements and offer insights to the students' cognitive representations of engineering practice Concept layering and code co-occurrence Certain codes are commonly expressed concurrently and in an intertwined fashion where the individual concepts were layered One frequent example of concept layering was observed in many statements that discussed human flourishing and dispute resolution For example, some essays addressed social aspects and concerns, while also weighting options in order to produce solutions that will lead to the best outcomes for different social groups (or stakeholders) Further, the consideration of stakeholders and potential outcomes inherently touches upon a number of the responsible framework codes This was also observed in the co-occurrence of dispute resolution and anticipation, as stated above In general, given the format of reflective writing, it was not surprising that students offered written responses that addressed a number of important concepts related to engineering practices at the same time Here are four extended quotes that demonstrates how the concepts were layered and was coded for human flourishing, dispute resolution, stakeholder engagement and anticipation: “With engineering, the designers have to take into account the ramifications of their design and acknowledge possible negative outcomes They must analyze every design and possible solutions to assure that they satisfy what society demands or needs Engineering is taking into account the different social groups that might use the technology or will be impacted by it and coming up with solutions that will satisfy most, if not all of the parties involved For example, from Winner’s article on artifacts having politics, I realize that to avoid future innovations like the racist bridge it is necessary for multiple social groups incorporate the needs and demands of multiple social groups.” Student 47 “For example, it may seem desirable to install a high-tech water filtration system in a third-world country to reduce water-borne diseases and deaths However, without proper communication with the locals, they may reject this system Perhaps they not understand the need for water treatment, or they not like the high-tech solution proposed Many times, it would be better to implement a lower tech system that is easier to understand and use on a daily basis Furthermore, low-tech options are often much cheaper than the fancier alternative.” Student 48 “For some social groups and stakeholders, the goal be low cost For others, the goal is longevity and efficiency of a solution, and still others hold timeliness in highest value There can be an overwhelming amount of metrics and goals to be balanced in any project or problem, but it is the job and responsibility of an engineer to consider and balance them when making decisions about design and application of scientific knowledge.” Student 23 “There isn’t always going to be a solution to a problem which satisfies everybody, which is why it is important for engineers to be aware of the different social groups and to be on the lookout for possible negative consequences on any of these groups that implementing a new technology could have.” Student 56 These statements demonstrate how students were wrestling with how different values align with different social groups and the trade-offs between those alternative solutions They also cede power to stakeholders, other than engineers, to influence the design and implementation of a given technology The practice of engineering design is discussed as a future-oriented activity that needs to serve the needs of different (and at times competing) interests Shifts in discourse: Pretest and posttest differences Reflexivity was the second most frequently applied code with quite a few writing samples explicitly speaking to this regard In the posttest, many students reflected upon their prior work and made comparative or evolutionary statements The long-form essay, as an educational activity clearly prompted many students to reconsider and to wrestle with their understanding of engineering What is also present in the excerpts below is the evidence of co-occurrence with stakeholder engagement These student writing samples show how the activity of reflection was bringing up other perspectives and different forms of expertise “After reviewing my first journal entry on the subject of engineering, I noticed that my writing focused more on the technical success of engineering as opposed to who are the stakeholder in the process.” Student 41 “Additionally, in August, I defined improvement as: faster, lighter, cheaper, more environmentally friendly, etc However, Pinch and Bijker’s SCOT apporach to technology has made me realize that all of these measures of improvement are relative to the person or social group that they affect.” Student 17 “In the previous journal entry, I stated that engineering was the application of not only the scientific or mathematical knowledge, but also the economic and social knowledge that brings inventions, structures, and materials [into] society.” Student 11 From an educational perspective, this form of self-reflection demonstrates what Carl Mitcham [47] wrote about in Issues in Science and Technology about the need to foster this competency within the engineering profession The statements make explicit the students own consideration for their prior definitions before demonstrating a shift in their discourse Discussion This research offers a clear conceptual contribution to the literature in two regards Substantively, the theories of responsible innovation have been framed in an aspirational manner, and deliberate processes and outcomes that are better than the status quo Yet, those aspirational theories rarely addressed the counternarrative, irresponsible innovation, and the processes that exclude diverse perspectives and contribute to negative unintended consequences Thus, by drawing upon the works of Riley, Cech and Slaton, this research brings critiques of engineering practice and culture into conversation with those more aspirational theories of responsible innovation This approach offers greater theoretical balance and analytical rigor Secondarily, this research suggests a methodological approach to discern the values of responsible and irresponsible innovation from writing samples This method offers insights about the professional formation of engineers and engineering culture by revealing the implicit assumptions about the profession held by students enrolled in an undergraduate engineering program The method also reveals how students integrate different concepts into their discourse and reflect upon their own cognitive shifts after taking a course focused on science, technology and society The cultures of disengagement and the reductionist approach of engineering to reduce society’s problems into scientific and mathematical formulae were quite evident in the writing samples turned in on the first day of class These students were in their fourth year of undergraduate education and, as theorized by Cech [13], had become “disengaged” by separating the social from the technical and viewing engineering as an apolitical activity Yet, the evidence is not so one-sided The students’ pretest writing samples also contained excerpts about different stakeholders, unintended consequences and potential value conflicts In analyzing her results, Cech concluded that a transition to a more socially engaged engineering that prioritizes responsible innovation must promote concern for public welfare and be “open to the perspectives of nonengineers” [13, p.67] What was encouraging in these results is observing the greater frequency and importance of different stakeholders and their values What the posttest revealed, statistically and in the qualitative statements, was a shift in their discourse about engineering That shift can be observed both the aggregate scores for irresponsible innovation and responsible innovation, as well as in the reflective writing samples offered This is encouraging, but it opens up more questions than it answers For one, how would these people define engineering after a few years on-the-job where the organization culture and professional experience will continue to shape and form their perspective? Or conversely, upon enrolling in an undergraduate engineering program, what did they understand engineering to be about? Were they just recruited because they were good at math and science, and engineering is what their high school guidance counselor suggested? Questions remain about the broader curriculum and what role that played in the influence on the students’ perception about the profession Are the students in this course distinct from other courses or distinct from other engineering colleges? Since this study was situated within an engineering school that sits within a larger university with ample Liberal Arts courses, what influence did those other courses and broader education environment have on the results? Or maybe the results are indicative of the students desire to please the professor Were the students just writing to the professor (of an STS course) and trying to tell the professor “what they wanted to hear”? What would their responses have been in a course on the robotics or computer coding? These questions point to many of the limitations and some of the future research associated with this project Limitations and future research This research project, like most, is flawed and imperfect The evidence from this exploratory project was gathered from two classrooms taught by one professor during one year at one university Also, the total population recruited for the study was not sufficient to yield statistically significant results for more than a few of the pre-post comparisons Those conditions alone weaken the findings Further, the pretest and posttest were not followed up with additional tests after graduation and entry to the profession The application of thematic codes by the two graduate research assistants was not tested for inter-coder reliability, since a consensus coding approach was taken To minimize potential bias from the professor, the coding was reviewed by the two graduate research assistants, who had not performed thematic coding in the past Persons with greater training and experience might have applied codes more judiciously or with greater frequency This project can be expanded in a number of ways that may offer greater theoretical, methodological and substantive results Temporally, the research could start earlier in the engineering education process by eliciting responses to the prompt, “What is engineering?” from students that were just enrolled in an engineering program Further, this prompt could be used to gather evidence from practicing engineers within the profession from to 30 years of experience Such a temporal dimension might yield findings that depict an evolutionary perspective on the profession and demonstrate the professional formation (and re-formation) of cultures of engineering Another opportunity to expand this research would be to look across disciplines and business sectors For example, civil engineers answer the question differently from electrical or chemical engineers? Do engineers working within consulting firm answer differently than engineers within a manufacturing firm? That cross-sectional view would identify shared attributes about the broader culture of engineering, while highlighting distinctive differences between the disciplines and businesses Finally, there is the context of this study and the opportunity to compare results between engineering schools to better understand how the institutional context including the curriculum, organizational culture and broader university environment may inform the research Prospective engineering students, might provide essays on why they chose engineering as their preferred degree program and the results of those essays could be compared against this essay, which was written near the end of the students’ undergraduate experience Another line of inquiry might be built around the prompt, “What is engineering?” The openness of this prompt and the variety of responses meant that many students did not address the components of irresponsible and responsible innovation Many students even quoted dictionaries and other seemingly neutral sources, rather than expressing their own thoughts This question could also be edited moving forward, “What is good engineering?” to get more normative response in regards to values, views and framework constructs Responses to “good engineering” might be compared to what is “bad engineering” To provoke more introspection and reflection, this assignment could be framed, more explicitly as a self-reflective assignment and ask students about their own growth over their engineering education Conclusion The critiques of engineering culture offered by Riley [9], Cech [13], and Slaton [14] bring the aspirations of responsible innovation into relief Together, the aspirational dimensions of responsible innovation offer clear contrast to the myth of objectivity and narrow technical focus that are associated with cultures of engineering This novel analytical framework offers advantages for measuring change, as seen in this paper, and for comparing different engineering designs or the discourse of engineering between different sectors or organizations As far as the student learning outcomes, many started off with a problem-solving orientation and a dependence on mathematics and science to inform the design After the course, mathematics and science and the desire to solve problems had not disappeared, rather there was a new emphasis on the stakeholders, social groups and social context as important for engineering design These results imply that the focus on problem solving via mathematics and science might inherently are cognitive inseparable from engineering design, but to practice responsible innovation requires considerations for the people, politics and places Engineering programs that focus solely on technical requirements and ignore the social and human dimensions of the profession will continue to graduate disengaged engineers poised to create the next wave of irresponsible innovation References [1] National Society of Professional Engineers Ethics reference guide Available at: https://www.nspe.org/sites/default/files/resources/pdfs/Ethics/EthicsReferenceGuide.pdf [2] J Herkert, “Ways of thinking about and teaching ethical problem solving: Microethics and macroethics in engineering,” Science and Engineering Ethics, vol 11, no 3, pp 373–385, 2001 doi:10.1007/s11948-005-0006-3 [3] I Van der Poel and L Royakkers, Ethics, technology, and engineering: An introduction West Sussex: John Wiley and Sons, 2012 [4] L Winner, The whale and the reactor: A search for limits in an age of high technology Chicago: University of Chicago Press, 1986 [5] S Jasanoff, States of Knowledge: The co-production of science and the social order New York: Routledge, 2004 [6] L Vinsel, “After innovation, turn to maintenance,” Technology and Culture, vol 59, no pp 1-25, 2018 [7] B Newberry, “Katrina: Macro-ethical issues for engineers,” Science and Engineering Ethics, vol 16, pp 535-571, 2010 [8] P.T Robbins, “The reflexive engineer: Perceptions of integrated development,” Journal of International Development, vol 19, no 1, pp 99-110, 2007 [9] D Riley, Engineering and Social Justice San Rafael: Morgan & Claypool Publishers 2005 [10] G Downey and J Lucena, “Engineering cultures,” in Science, Technology, and Society, S Restivo, Ed New York: Oxford University Press, 2005, pp 124-129 [11] A.M Weinberg, “Can technology replace social engineering?” Bulletin of the Atomic Scientists, vol 22, pp 4-8, 1966 [12] J Smith, Extracting accountability: Engineers and Corporate Social Responsibility Cambridge MA: MIT Press, 2020 [13] E Cech, “Culture of disengagement in engineering education?” Science, Technology, & Human Values, vol 39, no.1 pp 42–72, 2014 doi:10.1177/0162243913504305 .. .Exploring values and norms of engineering through responsible innovation and critiques of engineering cultures Abstract Engineers are often taught that ethics entail the adherence to a code of. .. works by Rene von Schomberg and David Guston However, the critiques of engineering cultures are often absent from concepts of responsible innovation For if responsible innovation is a normative... facets of irresponsible innovation and dimensions of responsible innovation offer an analytical framing for this research Scholars faced with the challenge of conducting research on responsible innovation