25.3 Research Goal and Setting
25.3.3 Research Tools and Data Analysis
the research used a mixed methods approach that included both qualitative and quantitative tools.20 the participants’ perceptions were collected using a questionnaire (quantitative tool), and their tpaCK was examined based on the online assignments they had developed (qualitative tool). Content analy- sis was conducted on their written reflections.
the questionnaire aimed to characterize teachers’ perceptions about their ability and willingness to teach and assess their chemistry students using online assignments, and their perceptions about the advantages and chal- lenges of teaching and assessing students using online assignments. the questionnaire consisted of three parts. part i contained 13 items related to demographic information. part ii contained 28 likert scale items (1 = strongly disagree, 5 = strongly agree) to identify teachers’ perceptions and willingness to implement assignments using Google Forms. the items were based on the teachers’ sense of efficacy scale questionnaire—tses.21 We adapted items related to classroom management and student engagement, and added items related to tK and aK.
part ii was divided into four categories: (1) student engagement—items related to teachers’ ability to motivate their students; (2) pCK—items related to teachers’ ability to teach chemistry using Google Forms; (3) tK—items related to teachers’ ability to incorporate technology in their teaching materi- als; and (4) aK—three levels of assessment using Google Forms—using/mod- ifying existing online assignments, developing new online assignments, and implementing online assignments.
Downloaded from http://books.rsc.org/books/edited-volume/chapter-pdf/1746532/bk9781839165238-00318.pdf by RMIT University user on 06 February 2024
323 Online Assignments
We used Cronbach’s alpha statistical test to assess the internal consis- tency of the items in these categories. the Cronbach’s alpha score for all items was 0.821.
examples of items in each of the categories and their Cronbach’s alpha scores are shown in table 25.2.
When analyzing teachers’ answers regarding the advantages and chal- lenges of integrating online assignments, we found three major themes:
the teacher aspect; the student aspect; and the technology aspect. We asked five researchers to categorize the teachers’ answers based on the different themes. We compared our analysis with the researchers’ analysis and found over 90% agreement. table 25.3 presents examples of teachers’ answers and how they were categorized.
We asked the teachers to work in groups to develop an online assignment using Google Forms that is related to one of the topics on the chemistry curricu- lum, such as energy, atom structure, chemical bond, etc. the assignment should include diverse types of questions (multiple choice, open-ended, etc.) as well as animations and images, to facilitate visualization of the chemistry content.
to analyze the assignments that the teachers developed, we used a rubric comprised of three main categories: pCK, tK, and aK (see Figure 25.1). Fig- ure 25.2 shows examples of parts of teachers’ assignments and their scores.
the pCK category (score range 0–10) comprised of three criteria: chemistry concepts, thinking level, and adjustments to the student’s level. each crite- rion was given a score of 1–3, and an extra point was given for including a question that involves values, opinion, or relevance. the tK category (score range 0–10) comprised of two criteria: Google Forms visual appeal and use of the various tools supported by the Google Forms platform. We gave an aK score of 0–10 for use of evaluation tools embedded in Google Forms, where 0 means no evaluation at all and 10 means exceptional use. Figure 25.1 Table 25.2 examples of items from the likert scale section of the questionnaire.
Category number of
items example Cronbach’s
alpha student
engagement 4 i predict that my students will be excited by the use of online assignments as a class activity.
0.534
pCK 8 i am in favor of using online tasks as a pedagogical tool and i will definitely use them in my classes.
0.741
tK 5 i can answer students’ technical ques- tions about Google Forms while they work on the online assignment.
0.612
tpaCK and aK 11 i plan to develop online assignments because i have acquired the proper tools for doing so. (level 2—develop)
0.742
i will evaluate my students based on their answers on Google Forms assignments. (level 3—assess)
Downloaded from http://books.rsc.org/books/edited-volume/chapter-pdf/1746532/bk9781839165238-00318.pdf by RMIT University user on 06 February 2024
Chapter 25 324
presents an example analysis of an assignment on covalent bonds (total score 26 out of 30). each online assignment was analyzed as a complete unit and each category could receive a maximum score of 10. Our rubric is a modification of the pCK and aK rubric,8 and the tK rubric.22
the participants were asked to write their personal reflections for the online tasks they developed, from four perspectives: their goals when choos- ing the topic of the assignment, advantages and challenges related to incor- porating the assignment in the learning process, and lessons learned for peers. the content of the reflections was analyzed from these four perspec- tives, and several categories were exposed for each perspective. For example, the categories exposed for goals were social and academic.
Figure 25.1 analysis of an assignment on covalent bonds.
Downloaded from http://books.rsc.org/books/edited-volume/chapter-pdf/1746532/bk9781839165238-00318.pdf by RMIT University user on 06 February 2024
325 Online Assignments
Table 25.3 examples of teachers’ answers regarding advantages and challenges, and a qualitative analysis of these answers.
teacher’s answers
theme teacher
aspect student
aspect technology aspect Advantages
Google Forms enables a break from the rou- tine classroom setting that mainly involves a teacher’s lecture, especially if the form includes illustrative images, videos, and simulations. In addition, Google Forms can be used for summary assignments and for assessing students’ performance (pst94897).
✓ ✓ ✓
Google Forms are easily accessible, editable, and saved on the drive, so that you can always add new participants and limit the time allocated for answering (ist39579).
✓
Challenges
It will be difficult for students to upload files, as there are not enough computers in class and the internet connection is unreliable (although they can connect using their mobile phones). Watching videos can cause noise in the classroom (ist92676).
✓ ✓ ✓
Internet availability, not enough computers, the teacher does not have extra time and must cover all the topics on the curriculum (ist75930).
✓ ✓
Figure 25.2 examples of assignments and their analysis.
Downloaded from http://books.rsc.org/books/edited-volume/chapter-pdf/1746532/bk9781839165238-00318.pdf by RMIT University user on 06 February 2024
Chapter 25 326
From the perspective of advantages and challenges, the categories were teacher, student, and technology, which coincides with the three themes we identified when analyzing the responses to the open-ended question on the perceptions questionnaire. From the perspective of lessons learned for peers, pedagogical and personal recommendations were mentioned. the catego- ries and perspectives that emerged from the content analysis are shown in Figure 25.3, and examples of the participants’ answers for the various catego- ries are shown in table 25.4. in the goals aspect, the difference between aca- demic and social categories is that the former concerns cognitive learning outcomes, while the latter is about teamwork and peer interactions.