The CoI framework was originally applied in distance and online learning contexts, and has been the subject of validation studies.26,27 The framework describes the important components for learning via the process of collabo- rative inquiry, during which learners develop personal meaning while engag- ing with each other to question ideas and revise their understanding of the course content.28 Garrison uses a broad definition of inquiry, encompassing a cyclical process of problem identification, exploration, integration, and resolution.
The CoI framework is based on three “presences” that interact when stu- dents engage in collaborative inquiry. Briefly, the three presences are defined by Garrison28 as follows: teaching presence directs the functioning of the learning community and involves the “design, facilitation, and direction of cognitive and social presences” (p. 61); social presence, according to Garri- son (2011), is the “ability of participants to identify with the group or course of study, communicate purposely in a trusting environment, and develop personal and affective relationships progressively by way of projecting their individual personalities”29 (p. 34); and cognitive presence is the process of
“learning and thinking collaboratively”28 (p. 34), and emerges based on the instructor’s selection of the course material, activities, and assessment tasks and is influenced by the explicit goals of the learning environment.
The CoI framework is a comprehensive tool to “help educators cope with the theoretical and practical challenges of constructive collaborative thinking and learning transactions”28 (p. 85). It is often invoked to inform the design of online courses, particularly those where collaborative learning occurs via asyn- chronous discussions. here, the framework has been adapted in Figure 23.1 to
Downloaded from http://books.rsc.org/books/edited-volume/chapter-pdf/1746495/bk9781839165238-00287.pdf by RMIT University user on 06 February 2024
Chapter 23 292
uncover the complex design decisions and interactions necessarily for online two-stage exams to be collaborative and inclusive. The adapted CoI framework provides guidance and justification for collaborative assessment design deci- sions and reveals areas where chemistry educators would benefit from addi- tional research to inform these decisions.
The presences dictate the importance of a supportive classroom climate, productive dialogue where students exchange ideas, and the involvement of all members in the group. Garrison’s framework does not explicitly address diversity, equity, and inclusion, yet these important considerations are influ- enced by, and influence, all three presences. The framework’s external ring represents situational and contextual factors that influence how the three presences emerge and interact. Inclusivity has been added to the external ring to highlight its over-arching influence on the learning process as a goal, mindset, instructional design consideration, contextual factor, and outcome of the learning process. The extent to which a group, course, or institution is inclusive of all students is an important contextual component to how the centre parts of the framework interact to foster collaborative learning during two-stage assessments. The CoI framework is centred on the collaborative inquiry taking place in a social context, thus the addition of inclusivity is a natural extension. In the following sections, the components of the CoI framework will be discussed in the context of the author’s experiences imple- menting two-stage exams in chemistry courses, providing specific sugges- tions of how to enhance collaboration and inclusion.
Figure 23.1 CoI framework emphasizing aspects of collaborative assessment.
Adapted from ref. 28 with permission from Taylor & Francis, Copy- right 2015.
Downloaded from http://books.rsc.org/books/edited-volume/chapter-pdf/1746495/bk9781839165238-00287.pdf by RMIT University user on 06 February 2024
293 The Community of Inquiry Framework
23.2.1 Cognitive Presence in Online Two-stage Exams
The cognitive presence depends on the nature of the assessment tasks and how students discuss their thinking, question each other, and refine their understanding during the group stage of the exam. higher-order cognitive tasks that require a decision with corresponding justification are ideal for collaborative assessments, such as multi-step syntheses in organic chemistry or conceptual questions that have common alternate conceptions as answer choices in general chemistry. In a study examining group dynamics in two- stage exams, researchers found that the group stage of the exam is positively influenced by an appropriate level of task challenge.30
Groups need to have sufficient time to discuss their reasoning and come to a consensus. A high-pressure environment, either through high-stakes grading associated with the assessment or unreasonable time constraints, will encourage behaviours that shut down collaboration, such as defaulting to the group member who seems to know the most or going with the major- ity without discussion. For this reason, many instructors who use two-stage exams reduce the number of test items for the group stage. There is greater learning potential when students discuss challenging course concepts rather than simple facts. Open-book two-stage exams encourage productive dis- course since students can find information that may be stalling their prog- ress. Open-book exams also reorient our courses away from memorizing facts toward the skilled use of existing knowledge.31 We have moved from forbid- ding to encouraging the use of course notes and other materials in two-stage exams (both stages or just the group stage). Since our assessments contain a range of types of tasks, including higher-order tasks that require students to apply their understanding, having access to course notes has not hindered the lively collaboration we observe in the group stage of our assessments.
In our first implementation of online two-stage assessments we conducted an action research study to collect survey data to inform the design and imple- mentation throughout the course. This organic chemistry course for students who were not chemistry or biochemistry majors was primarily lecture-based via Zoom software and had 424 students enrolled (zoom.us). The course used two- stage collaborative exams for both midterm exams, which also took place via Zoom. This work was approved by the University of British Columbia Behavioural research ethics Board (ID h20-01663) and will be reported in detail elsewhere.5
We included questionnaire items designed by physics colleagues who are investigating group interactions in two-stage exams.32 The items probed stu- dents’ perceptions of how their group arrived at their answers. respondents selected the proportion of the exam content that was decided on with each of the four options in Figure 23.2. Of the survey respondents in the online course, 47% reported their groups discussed and reached consensus on most questions, whereas 24% percent reported they voted and went with the majority agreement with some discussion. Almost 17% reported they went with majority agreement without discussion. The results from the online course were very similar to the results collected in a face-to-face course before
Downloaded from http://books.rsc.org/books/edited-volume/chapter-pdf/1746495/bk9781839165238-00287.pdf by RMIT University user on 06 February 2024
Chapter 23 294
the pandemic, which took place in 2018 (Figure 23.2). Interestingly, both the online and in-person organic chemistry course data showed slightly poorer collaboration than in physics courses reported in the literature.12
These preliminary results reassured us that the online format did not greatly negatively affect the nature of students’ collaborative inquiry process.
This is supported by qualitative research that compared discourse styles in online and in-person collaborative peer-led team learning chemistry courses.33 however, more research is needed to directly compare the process and outcomes of online two-stage exams to in-person exams.
23.2.2 Teaching Presence in Online Two-stage Exams
The teaching presence includes instructional design, instructor’s facilita- tion of the learning experience, and peer-to-peer regulation of the inquiry process. The teaching presence can be enhanced by including collaboration and metacognition as explicit course goals, both of which can be linked to workplace skills. Making connections to future learning or work can help students “buy-in” to the two-stage exam process. For example, online col- laborative assessment shares similarities with the day-to-day collaboration required in virtual teams in many workplaces.34
Figure 23.2 Comparison of face-to-face and online methods of responding to questions during the group exam, as self-reported on a course ques- tionnaire. error bars are the standard error of the mean.
Downloaded from http://books.rsc.org/books/edited-volume/chapter-pdf/1746495/bk9781839165238-00287.pdf by RMIT University user on 06 February 2024
295 The Community of Inquiry Framework
A course which minimizes competition by avoiding comparative grading (“scaling”) is likely to be much better at fostering effective collaboration than an environment that appears to be mostly focused on grades. Leading up to the exam, we communicate to students that the goal of the second stage of the exam is to learn chemistry concepts and problem-solving strategies from each other. explaining pedagogical choices is a category of “Instructor Talk”
that communicates support for student learning and is a form of teaching presence.35
Clearly communicated instructions about the test logistics are an import- ant part of teaching presence for online collaborative assessments. Instruc- tors need to determine and communicate how students will move into breakout rooms and how their work needs to be documented and submitted.
The questionnaires used in our action research study asked several questions about the logistics of the exam, for example about the clarity of instructions and whether students experiences issues with any of the technology. Three students reported a technical challenge so severe it prevented them from participating in the group stage of the exam. The vast majority (269 out of 375 questionnaire respondents) did not experience technical challenges.
We used the results to inform the next offering of the assessment, including clearly stating what students should do in the event of some common chal- lenges, such as being dropped from the Zoom call due to unstable internet.
For in-person collaborative assessments, those with accessibility accom- modations, such as individual spaces to work and/or extra time for the exam are typically offered three choices for the second stage of the exam. One option is that they can decline to write the second stage, just like any other student (since we have a grading option for 100% individual exam score). To acknowledge the reality that students’ grades are increased with the incor- poration of the group scores, we calculate the students’ exam grade using the current class average of the second stage scores for that exam. A second option is to write the second stage of the exam with other students who also have accommodations. A third option, and the most popular choice by far, is that we time the start of the second stage of the exam so that students can participate in the collaborative assessment with the rest of the class.
This means that for students with additional time as their accommodation, they start the individual portion of the exam early enough to complete it and travel to the location of the group exam if in person. For online two-stage exams, it was seamless to provide accommodations to individuals and have these students participate in the two-stage assessment with the rest of the class since travel time was not needed.
23.2.3 Social Presence in Online Two-stage Exams
Building identities, trust, and relationships is challenging in a short-term group experience. Two-stage exams can be done with spontaneous short- term groups or in longer-standing teams, the latter are likely to launch into their discussions more quickly as they have developed relationships with
Downloaded from http://books.rsc.org/books/edited-volume/chapter-pdf/1746495/bk9781839165238-00287.pdf by RMIT University user on 06 February 2024
Chapter 23 296
each other. Inclusive collaborative assessments facilitate equitable group interactions. Approaches that facilitate equitable group interactions and wel- come students to bring their whole selves to the classroom learning environ- ment will lead to a positive classroom climate.36
In our courses, we have found that helping students see the value in working together helps to create a collaborative classroom climate. When students understand that the purpose of the two-stage exams is for every- one to improve their understanding, they develop a shared purpose which helps foster social presence.37 If their discussions reveal that other students have similar chemistry challenges, the experience can normalize struggling and increase a sense of belonging.38 When students have a strong sense of belonging, they are more likely to experience wellbeing and academic persistence.39
A growing body of research has identified that not all students experience in-class group work in the same way.39 This likely also applies to collaborative assessments. Instructional strategies that aim to construct an inclusive class- room climate will lay the groundwork for more equitable participation and learning from two-stage exams. To do this, we lead by example and introduce ourselves with our social identities (e.g., gender pronouns) on the course syl- labus, learning management system, and in class sessions. Many instructors discuss why they decided to study chemistry to build instructor–student rela- tionships.35 To enhance students’ perceptions of the value of teamwork, we show evidence regarding teams outperforming individuals, and explain that diverse groups with a broad range of life experiences and perspectives tend to perform better than non-diverse groups. Typically, this is done by sharing charts from research studies and discussing them during synchronous class time. Valuing teamwork and understanding diversity as a driver of excellence in science are important learning goals instructors can consider making explicit in their course goals.