EFFECTS OF NETWORK CONNECTIVITY AND FUNCTIONAL DIVERSITY DISTRIBUTION ON HUMAN COLLECTIVE IDEATION

Kỹ Thuật - Công Nghệ - Công Nghệ Thông Tin, it, phầm mềm, website, web, mobile app, trí tuệ nhân tạo, blockchain, AI, machine learning - Quản trị kinh doanh 1 Effects of Network Connectivity and Functional Diversity Distribution on Human Collective Ideation Authors: Yiding Cao1†, Yingjun Dong1‡, Minjun Kim1,2, Neil G. MacLaren1,3,4, Sriniwas Pandey1, Shelley D. Dionne1,3, Francis J. Yammarino1,3, and Hiroki Sayama1,3,5 Affiliations: 1Binghamton Center of Complex Systems, Binghamton University, State University of New York, Binghamton, NY, USA. 2State University of New York at Plattsburgh, Plattsburgh, NY, USA. 3Bernard M. and Ruth R. Bass Center for Leadership Studies, Binghamton University, State University of New York, Binghamton, NY, USA. 4Department of Mathematics, State University of New York at Buffalo, Buffalo, NY, USA. 5Waseda Innovation Lab, Waseda University, Tokyo, Japan. Correspondence to: ycao20binghamton.edu and sayamabinghamton.edu. †Current address: Michigan Medicine, University of Michigan, Ann Arbor, MI, USA. ‡Current address: University of Texas Health Science Center, Houston, TX, USA. Abstract: Human collectives, e.g., teams and organizations, increasingly require participation of members with diverse backgrounds working in networked social environments. However, little is known about how network structure and the functional diversity of member backgrounds would affect collective processes. Here we conducted three sets of human-subject experiments which involved 617 participants who collaborated anonymously in a collective ideation task on a custom-made online social network platform. We found that spatially clustered collectives with clustered background distribution tended to explore more diverse ideas than in other conditions, whereas collectives with random background distribution consistently generated ideas with the highest 2 utility. We also found that higher network connectivity may improve individuals’ overall experience but may not improve the collective performance regarding idea generation, idea diversity, and final idea quality. One Sentence Summary: Performance of human collective ideation depends on network structure and background diversitydistribution of individuals. Main Text: Introduction Organizations are increasingly relying on diverse collectives for successful solution development for many real-world technical and business problems (1-5). In such collectives, multiple people with different backgrounds work together to achieve a greater goal than would be possible for individuals to accomplish alone (6-9). Sharing of expertise among collective members with diverse backgrounds and behaviors is recognized as an important factor in collective effectiveness (10-14). Researchers are hence interested in how to improve the quality and efficiency of human collective processes in organizational settings (15,16). Human collective processes involve interaction and interdependence among multiple individuals with task-related functional diversity in expertise, experience, knowledge, and skills (i.e., not demographic or identity diversity as often discussed in public media and literature) (17-20). Studies have shown that the multidisciplinary backgrounds within collectives are positively correlated with quantitative and qualitative task performance (7,10-12,19-23) and that social network structure influences collective performance (19-29), although results obtained so far were rather mixed. There are several difficulties in investigating human collective processes (27), including complex organizational structure (30,31), open-endedness of problemstasks (31), heterogeneity of participants (32), and long-term dynamic nature of the processes (33,34). Accordingly, previous studies were often computer simulation-based (7,14,15,17-20,26,29) or, if experimental, mostly limited in terms of collective size, duration, andor complexity of tasks (18,27,35,36). In this study, we experimentally investigated how the functional diversity of backgrounds of individual members and their social network structure affect human collective ideation 3 processes in realistic settings with a larger collective size, a longer collaboration period, and a more open-ended ideation task for which no simple solutions would exist. Three sets of experiments were conducted using a custom-made online social network platform. In each experimental session, 20~25 anonymous participants were arranged to form a social network according to their backgrounds and collaborated on text-based collective ideation tasks for two weeks. The performance of collective ideation was characterized using multiple metrics, including the number of generated ideas, the bestaverage quality score of final submitted ideas, semantic diversity of generated ideas quantified using machine learning-based word embedding methods, and post-experiment survey results on participants’ overall experience. Using these results, we aim to address the following research questions: (1) How does the participants’ background distribution within a collective affect the performance of collective ideation and the participants’ experience? (2) How does the network structure of a collective affect the performance of collective ideation and the participants’ experience? Experiments We conducted the experiments using a custom-made online social network platform with a Twitter-like user interface (see Supplementary Materials). This platform allows participants to submit ideas in response to the assigned collaboration task, and to discuss the task by reading, commenting on, and liking other participants’ ideas. The experiments involved a total of 617 participants who were undergraduate or graduate students majoring in Engineering, Management, or other disciplines at a mid-sized US public university. To participate in an experimental session, students were required to fill out an experimental registration form to provide information about their academic major and a (relatively long) written description of why they selected their major, as well as their academic knowledge, technical skills, career interest areas, hobbies, or extracurricular activities, andor any other information related to their background (this narrative information is called “background” in this study). The background information was used to characterize participants’ functional diversity and to determine their arrangement within the social network in each experimental session. Each online experimental session lasted for 10 working days (= 2 weeks), during which participants were assigned an anonymous username to log in to the experimental online platform and spend about 15 minutes each day to work on the collective ideation task by submitting ideas 4 and commenting on and liking their collaborators’ ideas. Participants were expected to continuously elaborate on and improve their ideas over time by utilizing their collaborators’ ideas and comments provided on the platform. After the experimental session was over, the participants were asked to submit an end-of-the-session survey form to provide their favorite final ideas, which were then evaluated by third-party experts on a 5-point Likert scale. This survey also included four questions about the participants’ overall experience in the experiment, level of learning and understanding from their collaborators, self-evaluation of own contribution to the collaborative process, and personal evaluation of the final ideas (see Supplementary Materials for details). The narrative information of participants’ background was converted into a 400- dimensional numerical vector using the Doc2Vec word embedding algorithm (37). A similar approach with Doc2Vec has been used in another recent study (38). These numerical vectors representing the background characteristics of the participants were used, together with the participants’ academic majors, to arrange the participants within the social network. The daily ideas and final ideas were converted to 100-dimensional numerical vectors using Doc2Vec. Analysis and results We first designed experiments with a high-collaboration task (laptop slogan design; see Supplementary Materials) and conducted four experimental sessions. The participants of each session were allocated into three collectives (networks), which were configured to be similar to each other in terms of the network size, network structure, and the amount of within-collective background variations (i.e., average distance of background between participants). The underlying social network structure was a spatially clustered regular network made of 20–25 members with a node degree of four (Fig. 1A, left). Participants connected to each other were able to observe each other’s posts and activities on the online platform but would not directly see activities of other nonadjacent participants. The three collectives in each session differed only regarding spatial distributions of participants’ background variations, which is called “background distribution” hereafter. We tested three different background distribution conditions: clustered (Condition CB: participants were connected to other participants with similar backgrounds), random (Condition RB: participants were connected randomly regardless of their backgrounds), and dispersed (Condition DB: participants were connected to other 5 participants with distant backgrounds) (Fig. 1B). We labeled each experimental session by the network structure, the type of the collaborative task, and the session number (e.g., “S-HC- Session 1” means Spatially clustered, High-Collaboration task, Session 1). We compared the outcome measures among the three background distribution conditions. The results show that the collectives in Condition CB made fewer daily posts than the collectives in Condition RB (Fig. 2A), but the distances among those posts were significantly greater than those generated by the collectives in Condition RB or DB (Fig. 2C). Moreover, in all the four S- HC sessions, the collectives with Condition RB always generated the best final idea with the highest utility value (Fig. 3A). Meanwhile, in all the four S-HC sessions, the collectives with Condition DB consistently achieved the highest average utility score of the final ideas (Fig. 3B). As there were only four sessions run in this experiment, we would not be able to derive a statistically significant conclusion from this result. However, the consistent patterns observed across the three background distribution conditions imply the possibility of such organizational arrangements to have impacts on collective ideation and innovation. Next, we designed experiments with a low-collaboration task that does not involve substantial interaction among participants (short story writing; see Supplementary Materials) and conducted three experimental sessions using the exact same network configurations and experimental conditions as above. We labeled these experimental sessions as the “S-LC” (Spatially clustered, Low-Collaboration) sessions. The results show that the collectives in Condition RB made fewer daily posts than the collectives in Condition CB or DB (Fig. 2B) but the distances among those posts were not statistically different among the three conditions (Fig. 2D). The latter finding was particularly interesting as it shows a stark contrast with the results of the S-HC experiment (Fig. 2C). The effect of background distributions on the idea diversity is clearly manifested in the high-collaboration tasks but may not be so if the collective process does not involve much collaborative interactions among participants. Finally, we designed the third set of experiments with a high-collaboration task on a fully connected network structure made of 20 participants each (Fig. 1A, right) and conducted two experimental sessions for a total of eight collectives on a fully connected network. We labeled these experimental sessions as the “F-HC” (Fully connected, High-Collaboration) sessions. The results show that there was no significant difference between spatially clustered and fully connected networks regarding the ideation activities (numbers of daily posts) (Fig. 4A), but the 6 average distances between generated ideas were significantly less in fully connected networks (Fig. 4B). Moreover, there appeared to be a moderate level of difference in the highest score (Fig. 4C) and the average score (Fig. 4D) of final ideas, indicating that fully connected network structure may reduce the quality of ideas. These findings suggest that high density of a social network is not necessarily a positive factor to improve idea generation, idea exploration or idea quality in collective collaboration. We also analyzed the participants’ answers regarding the end-of-the-session survey questions about the experiment experience. The results show that the difference in background distribution conditions did not affect the participants’ survey responses (see Supplementary Materials), but significant differences were found between spatially clustered and fully connected collectives regarding the self-evaluated overall quality of ideas and the learning experience (Fig. 5). This is probably because of the greater number of ideas each participant was exposed to in a fully connected social environment in the F-HC experiments. Note that these improved subjective experiences of the participants were not aligned with the objective performance metrics of the collectives shown in Fig. 4. Conclusion In this study, we conducted a series of controlled online human-subject experiments to investigate the effect of background distribution and network structure on collective ideation processes. The results showed the diversity of generated ideas was significantly greater when the network structure was of low density and spatially clustered and when the background distribution on the network was also spatially clustered. This observation was obtained only for high-collaboration tasks but not for low-collaboration tasks, indicating that the effects of spatial clustering were on the collaborative interactions among collective members. This result can be understood in that spatial clustering of participants’ backgrounds helps different parts of the collective explore possible solutions in different directions and thus diversify the results of collective exploration (20). This is analogous to biological diversity promoted and maintained in spatially structured evolutionary populations (19). It was also notable that our participant survey results indicated that the participants gained significantly better experience and more satisfaction when the network was dense and fully connected, even though the actual performance of the 7 collective ideation was actually lower. This presents an important lesson that the perceived and actual performances of collective ideation may not be correlated with each other. This study also indicated the potential difference of collective performance between different background distribution conditions in spatially clustered networks. When participants were randomly placed, the collective tended to find the best ideas most effectively. This seemingly puzzling observation may be understood by considering how much background diversity each generated idea was exposed to locally. Namely, in Condition CB, generated ideas are exposed to human participants that were relatively homogeneous, and thus those ideas only need to meet relatively simple, consistent criteria to be successful in spreading. In Condition DB, in contrast, ideas are exposed to and evaluated by very different human participants, and thus those ideas must satisfy a wide variety of (possibly inconsistent) criteria, necessarily making them conservative and mistake-proof. We hypothesize that collectives in Condition RB achieved the right balance in the middle of this “exploration vs. exploitation” spectrum (18-20,26) and thereby found the best ideas most frequently, and meanwhile, that collectives in Condition DB had a high ability to filter out potentially problematic ideas and generate ideas that can be commonly accepted by most participants, achieving the highest average score. These results and interpretations altogether paint a picture of the complex nature of human collectives; the structure and composition of a collective with regard to functional diversity of participating individuals should be considered and designed according to the objectives and success measures. References and Notes: 1. Horton, M., Rogers, P., Austin, L., McCormick, M. (1991). Exploring the impact of face-to- face collaborative technology on group writing. Journal of Management Information Systems, 8(3), 27-48. 2. Kerr, N. L., Tindale, R. S. (2004). Group performance and decision making. Annual Review of Psychology, 55(1), 623-655. 3. Weber, R. A. (2006). Managing growth to achieve efficient coordination in large groups. American Economic Review, 96(1), 114-126. 4. Lu, S. Y., ElMaraghy, W., Schuh, G., Wilhelm, R. (2007). A scientific foundation of collaborative engineering. CIRP Annals, 56(2), 605-634. 8 5. Stasser, G., Abele, S. (2020). Collective choice, collaboration, and communication. Annual Review of Psychology, 71, 589-612. 6. Hammond, J., Koubek, R. J., Harvey, C. M. (2001). Distributed collaboration for engineering design: a review and reappraisal. Human Factors and Ergonomics in Manufacturing Service Industries, 11(1), 35-52. 7. Hong, L., Page, S. E. (2004). Groups of diverse problem solvers can outperform groups of high-ability problem solvers. Proceedings of the National Academy of Sciences, 101(46), 16385-16389. 8. Lu, S. C., Jing, N. (2009). A socio-technical negotiation approach for collaborative design in software engineering. In International Journal of Collaborative Engineering, 1(1-2), 185- 209. 9. Woolley, A. W., Chabris, C. F., Pentland, A., Hashmi, N., Malone, T. W. (2010). Evidence for a collective intelligence factor in the performance of human groups. Science, 330(6004), 686-688. 10. Jehn, K. A., Northcraft, G. B., Neale, M. A. (1999). Why differences make a difference: a field study of diversity, conflict and performance in workgroups. Administrative Science Quarterly, 44(4), 741-763. 11. Van Knippenberg, D., De Dreu, C. K., Homan, A. C. (2004). Work group diversity and group performance: an integrative model and research agenda. Journal of Applied Psychology, 89(6), 1008. 12. Van Dijk, H., Van Engen, M. L. (2013). A status perspective on the consequences of work group diversity. Journal of Occupational and Organizational Psychology, 86(2), 223-241. 13. Moreland, R. L., Levine, J. M., Wingert, M. L. (2018). Creating the ideal group: composition effects at work. Understanding Group Behavior, 11-35. 14 Gómez-Zará, D., Das, A., Pawlow, B., Contractor, N. (2022). In search of diverse and connected teams: A computational approach to assemble diverse teams based on members’ social networks. PLOS ONE, 17(11), e0276061. 15. McHugh, K. A., Yammarino, F. J., Dionne, S. D., Serban, A., Sayama, H., Chatterjee, S. (2016). Collective decision making, leadership, and collective intelligence: tests with agent- based simulations and a field study. The Leadership Quarterly, 27(2), 218-241. 16. Dionne, S. D., Gooty, J., Yammarino, F. J., Sayama, H. (2018). Decision making in crisis: a multilevel model of the interplay between cognitions and emotions. Organizational Psychology Review, 8(2-3), 95-124. 9 17. Dionne, S. D., Sayama, H., Hao, C., Bush, B. J. (2010). The role of leadership in shared mental model convergence and team performance improvement: An agent-based computational model. The Leadership Quarterly, 21(6), 1035-1049. 18. Sayama, H., Dionne, S. D. (2015). Studying collective human decision making and creativity with evolutionary computation. Artificial Life, 21(3), 379-393. 19. Dionne, S. D., Sayama, H., Yammarino, F. J. (2019). Diversity and social network structure in collective decision making: evolutionary perspectives with agent-based simulations. Complexity, 2019, 7591072. 20. Gomez, C. J., Lazer, D. M. (2019). Clustering knowledge and dispersing abilities enhances collective problem solving in a network. Nature Communications, 10(1), 5146. 21. Horwitz, S. K., Horwitz, I. B. (2007). The effects of team diversity on team outcomes: a meta-analytic review of team demography. Journal of Management, 33(6), 987-1015. 22. Koh, W. T. (2008). Heterogeneous expertise and collective decision-making. Social Choice and Welfare, 30(3), 457-473. 23. Salas, E., Rosen, M. A., DiazGranados, D. (2010). Expertise-based intuition and decision making in organizations. Journal of Management, 36(4), 941-973. 24. Yang, H. L., Tang, J. H. (2004). Team structure and team performance in IS development: a social network perspective. Information management, 41(3), 335-349. 25. Wise, S. (2014). Can a team have too much cohesion? The dark side to network density. European Management Journal, 32(5), 703-711. 26. Lazer, D., Friedman, A. (2007). The network structure of exploration and exploitation. Administrative Science Quarterly, 52(4), 667-694. 27. Shore, J., Bernstein, E., Lazer, D. (2015). Facts and figuring: an experimental investigation of network structure and performance in information and solution spaces. Organization Science, 26(5), 1432-1446. 28. Yang, K., Fujisaki, I., Ueda, K. (2020). Interplay of network structure and neighbour performance in user innovation. Palgrave Communications, 6(1), 1-8. 29. Cao, S., MacLaren, N. G., Cao, Y., Marshall, J., Dong, Y., Yammarino, F. J., ... Ruark, G. A. (2022). Group size and group performance in small collaborative team settings: an agent- based simulation model of collaborative decision-making dynamics. Complexity, 2022, 1-16. 30. Dansereau, F., Yammarino, F. J., Kohles, J. C. (1999). Multiple levels of analysis from a longitudinal perspective: some implications for theory building. Academy of Management Review, 24(2), 346-357. 10 31. Cullen-Lester, K. L., Yammarino, F. J. (2016). Collective and network approaches to leadership: special issue introduction. The Leadership Quarterly, 27(2), 173-180. 32. Ely, R. J. (2004). A field study of group diversity, participation in diversity education programs, and performance. Journal of Organizational Behavior, 25(6), 755-780. 33. Grand, J. A., Braun, M. T., Kuljanin, G., Kozlowski, S. W., Chao, G. T. (2016). The dynamics of team cognition: a process-oriented theory of knowledge emergence in teams. Journal of Applied Psychology, 101(10), 1353. 34. Yammarino, F. J., Mumford, M. D., Connelly, M. S., Dionne, S. D. (2010). Leadership and team dynamics for dangerous military contexts. Military Psychology, 22(sup1), S15-S41. 35. Becker, J., Brackbill, D., Centola, D. (2017). Network dynamics of social influence in the wisdom of crowds. Proceedings of the National Academy of Sciences, 114(26), E5070- E5076. 36. Mason, W., Watts, D. J. (2012). Collaborative learning in networks. Proceedings of the National Academy of Sciences, 109(3), 764-769. 37. Le, Q., Mikolov, T. (2014). Distributed representations of sentences and documents. In International Conference on Machine Learning, 1188-1196. 38. Lungeanu, A., Whalen, R., Wu, Y. J., DeChurch, L. A., Contractor, N. S. (2023). Diversity, networks, and innovation: A text analytic approach to measuring expertise diversity. Network Science, 11(1), 36-64. Acknowledgments: Authors thank Ankita Kulkarni and Shun Cao for helpful discussions for this study. Funding: This work was supported in part by the US National Science Foundation under Grant 1734147 and the JSPS KAKENHI Grant 19H04220. Author contributions: Yiding Cao: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Data curation, Writing - original draft, Writing - review editing, Visualization, Project administration. Yingjun Dong: Methodology, Software, Validation, Investigation, Data curation. Minjun Kim: Software, Resources. Neil G. MacLaren: Investigation, Writing - review editing. Sriniwas Pandey: Software, Validation, Formal analysis, Data curation, Visualization. Shelley D. Dionne: Conceptualization, Writing - review editing, Supervision, Funding acquisition. Francis J. Yammarino: Conceptualization, Writing - review editing, Supervision, Funding acquisition. Hiroki Sayama: Conceptualization, 11 Methodology, Software, Resources, Writing - original draft, Writing - review editing, Supervision, Project administration, Funding acquisition. Competing interests: Authors declare no competing interests. Data and materials availability: Authors are currently preparing the anonymized experimental dataset for public release (properly following the experimental protocol approved by the IRB). All the software tools developed and used in this study are available from the corresponding authors upon request and will also be posted to a publicly accessible code repository shortly. 12 Fig. 1 Social network structures used in the experiments. (A) The layout of spatially clustered and fully connected networks. (B) Examples of three background distribution conditions. From left to right: Condition CB (clustered backgrounds) in which individuals with similar backgrounds are connected together; Condition RB (random backgrounds) in which individuals are connected randomly; and Condition DB (dispersed backgrounds) in which individuals with dissimilar backgrounds are connected together. 13 Fig. 2 Distribution of numbers of daily posts and average distances between ideas from Day 1 to Day 10. (A) Comparison of numbers of daily posts among three background distribution conditions in the S-HC experiment. (B) Comparison of numbers of daily posts among three background distribution conditions in the S-LC experiment. (C) Comparison of average distances between ideas among three background distribution conditions in the S-HC experiment. (D) Comparison of average distances between ideas among three background distribution conditions in the S-LC experiment. The p-value annotation legend is as follows. : 0.01 < p