interdisciplinary research and innovation in bilingual and second language teacher education

Thus, both students and teachers can benefit from tools that can help them monitor the writing process and improve students’ self- regulation and self-efficacy.Across a series of small s

Part II

Digital technology in L2

and bilingual education

Writing plays a crucial role in students’ language development and ment, yet it is a skill that needs to be better understood (Greer et al., 2016) Research suggests that solid writing abilities predict academic success, fos-ter the development of critical thinking skills, and hold significant value beyond formal education in personal and professional contexts (Johannes-sen, 2001) However, numerous students across different grade levels need help to produce coherent and extensive written texts, particularly those learning a second language (Ceylan, 2019; Farooq et al., 2020) Cummins’ extensive research on English language learners has revealed a substantial time gap between achieving conversational fluency in English and reach-ing grade-level expectations in academic aspects of the language (Cum-mins, 2008, p. 72) This disparity led to the differentiation of two distinct language skill sets: Basic Interpersonal Communicative Skills (BICS) for oral proficiency and Cognitive Academic Language Proficiency (CALP) for literacy skills in reading and writing While conversational skills tend to reach age-appropriate levels within approximately two years of exposure

assess-to English, acquiring CALP, including vocabulary knowledge and writing, can take immigrant students around five to seven years to approach grade-level standards (Cummins, 2008)

Consequently, when it comes to writing, both language learners and their peers must invest several years of effort to master this skill This is because mastering writing as a language skill is a complex process that involves various cognitive factors after acquiring other language skills (Dragomir & Niculescu, 2020) Effective writing seeks to fulfill the writer’s intention while addressing the reader’s needs, testing memory, thinking ability, and verbal command to express ideas successfully It is also a pro-ductive process that typically includes stages such as planning, drafting, revising, and editing (Abas & Aziz, 2016) Through the revision process,

3 A tool for enabling reflection

and exploration of bilingual and L2 writing behavior through revision history analytics

Selen Türkay, Bahar Otcu-Grillman, and Daniel Seaton

students refine their writing, ensuring coherence, clarity, and the effective conveyance of their message.

Revision has a central role in the writing process and involves refining

a text to better align with the writer’s goals However, it is a challenging cognitive and procedural aspect of writing (Fitzgerald, 1987; Hayes  & Flower, 1986) During revision, writers engage in metacognitive processes, evaluating and reconsidering ideas, organizing their writing, and identify-ing and correcting errors These processes relate to self-regulatory skills in writing, including observation/detection, judgment/identification, and reac-tion/correction (Williams, 2004) While challenging, the revision process can enhance metacognitive awareness and lead to the discovery of opportuni-ties for improving students’ writing (Roscoe et al., 2016) Developing strate-gies and tools that enhance students’ self-regulation and self-efficacy during the revision process would significantly contribute to their academic and language development (Pajares, 2003) The need for monitoring and self- regulation becomes even more crucial in virtual teamwork and collaborative writing for both first and second-language learners (Hedgcock & Lefkowitz, 1992; López & Guerrero, 2014; Robert, 2016) However, designing effective collaborative writing tools to support these core skills remains an open chal-lenge in computer-supported collaborative work (López & Guerrero, 2014).Recent studies have shown that analytics has affordances to help give writers meaning-making opportunities to take action (Durall  & Gros, 2014; Tabuenca et al., 2015; Vatrapu et al., 2013) The work exploring the relationship between writing analytics and self-regulatory processes

to improve self-efficacy is still in its infancy Studies, to date, explored designing dashboards to support reflection and self-monitoring (McNely

et al., 2012; Vatrapu et al., 2013) without an in-depth exploration of the relationship between user characteristics and patterns in analytics Writ-ing analytics can support writers’ information awareness and social group awareness in CW settings (Janssen & Bodemer, 2013); they can improve their self-efficacy by showing their mastery experience, displaying social comparisons, and giving writers feedback (Pajares, 2003) In the context of second language writers, this on-time mastery-oriented feedback on their writing process, rather than focusing on errors, could help improve their writing proficiency (Hyland & Hyland, 2006)

Over the last few decades, researchers have also used a wide variety

of computational approaches, such as keystroke logging, to study writing processes (Zhang & Deane, 2015) These virtual environments not only track keystrokes within a word processor but overall habits while work-ing from a computer Such collection, however, requires special attention concerning privacy and potentially invasive research Google’s Cloud Plat-form may strike a balance between potentially invasive keystroke logging applications and optimal data collection

Google Docs has an API that provides character-by-character revision data Using that data for a given user, we provide aggregate metrics and replay features for writing revisions in a novel application: Itero is an application we developed to visualize, analyze, and replay writing within

a Google doc Itero aims to help writers and instructors better understand the spatial and temporal nature of the writing process using analytics

We believe that Itero is timely as more students use Google Docs for their academic writing Over 30 million children in the United States, constitut-ing over half of the nation’s primary and secondary school students, use Google apps like Gmail and Docs in their education In addition, it was found that networked writing could motivate less proficient students, both first- and second-language writers, by providing a nonthreatening environ-ment for them to practice their writing skills and receive expert and peer feedback on their writing (Greenfield, 2003; Hyland  & Hyland, 2006) Thus, both students and teachers can benefit from tools that can help them monitor the writing process and improve students’ self- regulation and self-efficacy

Across a series of small studies in different contexts, we examine the effects of visualizations of temporal-spatial information about the text on writers’ subjective experiences; investigate the role of replaying the writ-ing process in supporting students’ awareness of their writing behaviors; explore how the temporal-spatial nature of revision patterns may relate to writing self-efficacy and self-regulation; and test a platform that enables users to take advantage of Itero without using their Google accounts In particular, our work provides the following:

• A working prototype of a writing revision analytics tool to address our research goals

• A working prototype of a platform to enable experimentation and usage without requiring users to own a Google account

• Evidence that writing revision patterns may give signals about writers’ self-efficacy and writing strategies

• Design implications for writing analytics tools that aim to improve efficacy and self-regulatory skills in second language writers

self-Background

Itero application

Itero is an application that shows detailed revision history for writers to observe their writing behavior We planned to achieve this by developing writing analytics and visualizations, building upon prior projects and stud-ies in the field (Wang et al., 2015) The main goal of this application was to

understand the temporal nature of revision patterns and how those relate

to activities within specific passages of a written text Such patterns can be used to identify writing strategies that could lead to computer-aided inter-ventions, for example, automatically asking a student to pause and reflect

on the theme of a paragraph Itero also aims to make the contributions among writers more transparent for writers

Google revision data

Google Docs is an online word processor offered by Google In addition

to basic word processing functionality, the Google Docs application also keeps a detailed and browsable record of a document’s revision history Document owners can see this history by selecting “File” -> “Version History” -> “See Version History” from the top toolbar menu In the result-ing view, the user can browse a series of revisions corresponding to one or more basic operations (e.g., insertion of a series of characters or deletion

of a series of characters) The revisions carry information about the time the change was made, and the user who made the change is indicated with color coding We note that the Google Doc web interface’s revision history features summarize revisions in aggregate We use the revision data on a character-by-character (insertion-by-insertion) basis

Retrieving users’ Google Docs revision history in Itero

Itero utilizes this detailed revision history from Google Docs to enable users

to analyze further and visualize the writing process After authenticating with their Google user account and giving permission for Itero to access their Google Drive files, users can browse a list of Docs in their Drive that they own They may select a particular document(s) to view in Itero – at which point Itero retrieves the detailed revision data for that particular document from Google Drive

Features of Itero

In its current design, Itero has three main affordances:

1 Individual’s writing analytics Visualizing temporal and spatial patterns

of the writing process that would potentially improve users’ mastery

2 Collaborative writing analytics Visualizing the contributions of ent co-writers to facilitate social comparison

differ-3 Replay Character-by-character replay of the writing process at different play speeds

Writing analytics for self-regulation and self-efficacy

Analytics has great potential to influence how students learn from the ing process They can be provided with automated feedback on writing Due to the topic’s interdisciplinary nature, many projects explored revision analytics in different academic disciplines They aimed to discover patterns

writ-of writing using visualizations and did not aim to use the visualizations as formative or summative tools to improve writing processes For example, Southavilay, Yacef, Reimann, and Calvo (Southavilay et al., 2013) devel-oped a tool to visualize collaboration taking place on Google Docs and proposed a topic-based collaboration network HistoryFlow visualizes the nature of contributions and how collaborative documents evolve on Wiki-pedia pages (Viégas et al., 2004) Whoviz was another tool developed to discover patterns of author participation on Wikipedia (Flöck & Acosta, 2015) Authors mainly looked at word-level authorship and disagreements One exception is Olson and colleagues’ work (2017) Using the application DocuViz, which visualizes the writing process, authors uncovered differ-ent participation patterns and their relation to the overall writing qual-ity, assessed by human raters They found that documents with evidence

of leadership and balanced participation from co-writers received higher ratings

There is a rich body of literature indicating the critical role of students’ beliefs and behaviors for success This includes monitoring and self-reflection

as mechanisms of self-regulation during the writing and revision cess, as well as writing self-efficacy Students’ writing self-efficacy posi-tively relates to increased effort and mastery goals and negatively relates

pro-to writing apprehension (Bandura, 2012; Pajares, 2003) Previous studies

of learning analytics show evidence of a relationship between students’ behaviors and self-efficacy (Liu et al., 2015; Yamada et al., 2015) Dash-boards and learning analytics can be used as formative assessment pieces

to foster metacognition and improve the final document Recent research shows that learning analytics can help students monitor their learning progress and self-reflection (McNely et al., 2012; Tabuenca et al., 2015; Yamada et al., 2017) Marzouk et al (2016) emphasize the importance

of labeling to facilitate monitoring and self-reflection They propose a set

of learning analytics designs that uses a framework of active learning to categorize data to support learners (Marzouk et al., 2016)

Most current writing revision tools, however, lack in-depth analytics support for writers One exception is WritingPal, which was primarily designed to provide feedback to student writers on their revisions (Roscoe

et  al., 2016) Authors showed evidence that automated tools can detect changes in students’ writing that can be used to give actionable feedback to students While there is prior work on autonomous feedback in the context

of second-language writing (Dikli  & Bleyle, 2014; Ranalli et  al., 2017), more studies are needed on how writing analytics might help these writers during their revision process This study investigates the extent to which writing analytics can be used to discover writers’ beliefs and behaviors, improve their self-regulation and writing self-efficacy, and provide implica-tions for bilingual and second language writing.

Collaborative writing

Collaborative writing (CW) is defined as writing accomplished by more than one author It is an iterative and social process involving co-writers coordi-nating, communicating, and negotiating while creating a shared document (Lowry et al., 2004) Successful CW, as in collaborative learning, allows co-writers to learn from each other The potential of CW has been recognized

by the continuous development of tools informed by Computer Supported Collaborative Work (CSCW) research Most recently, Cloud-based writing tools have become very popular in both educational and work settings It is

a common practice among teachers to use Google Apps or Microsoft Drive to assign students homework In a recent article, Singer reported that about 30 million children in the United States, more than half the nation’s primary and secondary school students, use Google apps like Gmail and Docs in their education One of the most exciting affordances of these cloud technologies is that they facilitate collaboration in distributed teams and make it easier even for teams who work in the same room

One-CW requires several skills and conditions to be a satisfactory experience for writers It also requires effective group dynamics, including coordina-tion and communication (Erkens et  al., 2005; Lowry et  al., 2004), and social dynamics revolve around authorship and trust (Baecker et al., 1993) People have expectations about roles, responsibilities, and timeliness This makes collaboration awareness an essential aspect of successful CW tasks (Dourish & Bellotti, 1992) Multiple projects have explored different ways

to visualize the evolution of documents, starting in the early 1990s (Baecker

et al., 1993) Addressing the findings from research in the 2000s, it was found that people prefer to work asynchronously when collaborating on writing (Noël & Robert, 2004) Commercial word processors also imple-mented ways to track changes in the document Microsoft Word assigns different colors to co-authors so it is clear who contributes or edits what (Microsoft Word 2016 | Document & Word Processing Software, n.d.), and Google Documents has revision histories to discover recently made revi-sions by co-authors (Google Docs – Create and Edit Documents Online,

for Free., n.d.) Many projects have used Google Doc API to visualize the

writing revision process in recent years Some of the ones that inspired our project were Draftback, a playback of Google Docs (Draftback, 2018), and DocuViz, which aims to show what collaboration patterns take place

in Google Docs (Wang et al., 2015) Itero combines these two alities, as we described earlier, and the rest of this chapter talks about the two studies we conducted with Itero Study 1 investigated potential users’ attitudes toward Itero, to discover design improvements, and to examine its role in writers’ awareness of their writing process Study 2 investigates

function-CW strategies (i.e., single-author, sequential, parallel writing by horizontal division, and parallel writing by stratification (Lowry et al., 2004))

Study 1

In the first study, we examined how Itero impacted individuals’ standing of their writing revision process during a writing task To gain insights into users’ experiences, we employed a mixed-method approach that involved surveys, semi-structured interviews, and writing analytics Our primary goal was to identify any barriers users encountered when utilizing Itero for individual writing tasks and to understand their specific needs This research was conducted before further refining and enhancing the Itero application

under-Participants

We recruited nine undergraduate students (five female and four male) from

a large private university on the east coast of the United States Students received US$15 for one hour of their time The study took place in a usabil-ity research laboratory of the university on a 1–1 basis All participants were familiar with Google Docs but needed to use the application more intensively in their academic work The main reasons they used Google Docs were to work on CW projects and take notes On average, partici-pants rated their writing fluency as “very fluent.”

Procedure and instruments

After signing the informed consent form, we conducted a five-minute view to learn about participants’ use of different writing tools and their perceived need to be better writers Once seated in front of the lab PC, participants completed a short survey They were then given the following writing prompt and were asked to write a two-page-long essay supporting one of the beliefs:

inter-Some people believe that it is imperative for individuals living in oped nations to reduce their energy consumption and lead a more sus-tainable lifestyle, given the evidence for global climate change Others believe such drastic lifestyle changes are unwarranted, based on the existing evidence for global climate change

devel-After 25 minutes of writing, participants were asked to look at the visuals

in Itero and study them for a couple of minutes After, they continued with their writing for ten more minutes After completing the writing, we con-ducted a short interview with participants Finally, they filled out a short survey (i.e., five 5-point Likert scale items) about their subjective experi-ences with Itero

We used the instrument developed by Bruning, Dempsey, Kauffman, and Zumbrunn to measure participants’ self-efficacy for writing (Bruning et al., 2013) Bruning et al developed their multifactor scale and tested it in two studies The factors are ideation (five items, e.g., “I  can think of many ideas for my writing”), conventions (five items, e.g., “I can write complete sentences”), and self-regulation (six items, e.g., “I can think of my writing goals before I write”)

Findings

User experiences with Itero

Overall, participants found Itero to be a novel application As one of the participants put it, they “had never seen anything like this before.” How-ever, novelty alone does not warrant the adoption When asked whether they would use Itero for their academic writing, all nine participants expressed doubt about using it for individual work, but they said they would use it for CW and group projects They proposed two main rea-sons for it: (1) to know how much each co-writer contributes (e.g., quan-tity, accountability), and (2) to understand who contributes which ideas (e.g., quality and content) In other words, participants wanted to see how engaged their co-writers were As one of the participants, P3, puts it, she would use Itero “to see where ideas came from and how engaged every single member of the group was in writing.” P4 indicated that he would use Itero if an assignment has a participation grade, because Itero would allow him to gauge how much he contributes compared to his teammates.Participants stated that Itero helps them understand their writing and revision process by helping them notice behavior they would not notice otherwise These included which parts they worked most on; what their revision process looked like However, P7 cautiously stated, “In the con-text of a normal assignment, you might not get much of a chance to use it

It would be useful to help develop writing skills.” This has positive tions for bilingual and second language writers as they would be develop-ing their writing skills in a new language

implica-The main reasons participants liked Itero were (1) objectivity (e.g., ing what their writing looks like as a process); (2) different visualizations

see-of their writing process All participants found the replay function to be the most helpful tool for their writing, resembling the performance of their documents.

Learn from others’ writing processes

Interestingly, participants brought up a possible use of Itero for writing instruction: using the replay feature of Itero to watch and learn from oth-ers’ writing processes P3 said if she knew a person who is good at writing, seeing their writing process would be beneficial to improve her writing Similarly, P9 said, “If I was trying to learn from someone else’s writing, it would help me see the thought process.” Both participants indicated their desire to learn from expert writers using Itero While we would need fur-ther design considerations to support such functionality, it certainly opens another line of exploration for the authors Using Itero to show example writing processes can also help second language writers to see multiple examples from experts and apply the strategies to their writing

Desired features

Some of the visualizations could have been more easily interpretable by the users Three participants wanted Itero to provide further information to writers on what different visualizations and analytics meant and facilitate their reflection In this way, they can discover what they need to do for the next iteration and how to improve Participants said they might not use Itero during the writing process (formative), but, instead, they would use it after they have a draft (summative) This is a quote from P8: “I think this

is good as a reflection after the essay is finished Not so much while it is being written While it is written it may be a bit overwhelming but after it

is done, I can learn for the next time.”

a readability test indicating how difficult a passage is to understand Both participants (P6, P7) with high self-efficacy scored thirteenth to fourteenth-grade level text standards (third highest among all participants) This

finding also has implications for instructors where it might be possible to identify writers with low self-efficacy early on to provide them with scaf-folding on their writing skills.

Discussion

In this first study, we tested Itero with participants from one of our tial target populations: college students We invited participants to observe their writing process using multiple features of Itero Overall, participants responded to Itero positively They became aware of their writing behav-iors, such as focusing too much on a single section or not paying attention

poten-to grammar

There were some usability issues with Itero Most participants found that character trace visualization could have been more intuitive Even though there were explanations of visualizations, participants had diffi-culty interpreting the data Later in this chapter, we present some of our planned improvements to the visuals

Implications for second language writing and instruction

There are some implications for bilingual and second language writing and instruction from this first study Participants mentioned that Itero helped them understand their writing and revision processes, enabling them to notice behaviors they would not have otherwise noticed This has positive implications for bilingual and second language writers as they can develop their writing skills in a new language by gaining insights into their writing processes and making improvements Visualizations and objective feed-back provided by Itero can further support second-language writers in self-assessment and understanding their writing progress

Participants also expressed interest in using Itero’s replay feature to observe and learn from expert writers’ processes This highlights the potential for using Itero to demonstrate example writing processes and strategies, benefiting second-language writers by exposing them to multiple examples from experts

We found that participants with higher self-efficacy showed different writing patterns compared to those with lower self-efficacy Identifying these patterns can assist instructors in identifying writers with low self-efficacy early on, allowing for targeted support and scaffolding to improve their writing skills

Previous studies showed the benefit of collaborative revision for language writers (Hedgcock & Lefkowitz, 1992) Itero could be used in the collaborative revision process to give writers further information on their writing patterns and a starting point for peers to comment on each other’s

second-writing and foster accountability within second-writing groups In addition, porating innovative tools like Itero can enhance student engagement and interest in second-language writing.

incor-Overall, findings from Study 1 suggest that incorporating writing sion analytics tools like Itero in second-language writing instruction can positively impact student engagement, collaborative writing, writing skill development, and self-assessment Additionally, the ability to observe and learn from others’ writing processes and identify writing patterns related

revi-to self-efficacy can further enhance instructional support and intervention strategies for second-language writers

Study 2

In the second study, we employed an experimental design to investigate the impact of Itero on participants’ awareness of their writing process during

a collaborative writing (CW) task Additionally, we explored the influence

of Itero on participants’ employment of CW strategies

Participants and research design

We recruited 26 participants through University’s Research study pool (13 female; 13 male) Participants reported being proficient in English Partici-pants’ average age was 26 (SD=5.46) Nineteen participants were current students Participants spent two hours on the study and were reimbursed

$25 for their time

Participants indicated the reasons to use Google Docs are ease of access (e.g., on the Cloud) and affordances for collaboration In particular, 12 participants indicated they use Google Docs mainly for collaborative purposes

Procedure

The study took place in a room with 12 computers, six on each side of a dividing wall Participants were randomly seated and assigned to groups There were ten groups of two and two groups of three due to the odd num-ber of participants we had in two sessions Once seated, participants com-pleted a survey on their demographic information, familiarity with Google Docs, writing fluency, and writing attitudes They were given the same writing prompt as in the first study

Because coordination is essential for CW (Dourish  & Bellotti, 1992; Lowry et al., 2004), participants were asked to coordinate and plan their writing strategies using the chat feature in Google Docs They were also asked to introduce themselves to each other before starting to work on the

essay After 40 minutes of writing, they were asked to review their ing analytics on Itero and fill out a short mid-writing task questionnaire about their experiences with Itero After filling out the questionnaire, they went back to writing their collaborative essays for 40 more minutes They were given a five-minute warning and asked to conclude their essays Par-ticipants interacted with Itero one last time, examining their writing revi-sion analytics and collaboration using the features of Itero They filled out a post-survey, which took about five minutes Finally, participants were asked to look at the writing revision analytics of another team for two to three minutes This task was designed based on the findings from Study 1, where participants highlighted the potential of using the tool

writ-to learn from others’ writings Our primary purpose was writ-to investigate whether participants could make sense of writing revision analytics for another team After the survey, the participants were briefed, reimbursed, and released

Instruments

We used the same writing self-efficacy instrument from Study 1 Because this study did not involve an interview, we asked open-ended questions about participants’ attitudes toward Itero and their self-regulatory expe-riences Specifically, we asked: “What did you notice about the writing process your team was involved in? Please explain briefly.” “What did you recognize about your writing process after looking at Itero?” “What are some of the challenges you had with Itero?” If participants observed another team’s writing process, we also asked, “What did you realize about your team’s writing compared to the other team’s writing?”

We administered a questionnaire on participants’ attitudes toward laborative writing (e.g., I  enjoy collaborative writing more than I  enjoy writing alone; I am more interested in writing when I write collaboratively) and a four-item 5-point Likert scale questionnaire on writing alone (e.g.,

col-I  enjoy writing alone; The process of writing alone is satisfying to me) Cronbach’s alpha were 89 and 88, respectively We also asked participants

to rate the perceived usefulness of the main features of Itero – replay and visualizations – on a 5-point Likert scale (1 – not at all; 2 – slightly; 3 – somewhat; 4 – very; 5 – extremely useful)

Findings

On average, participants found both replay (M=3.84, SD=1.11) and alizations (M=3.54, SD=1.25) somewhat helpful in understanding their writing processes Participants reported more positive attitudes toward writing alone (M=3.27; SD=1.00) than writing collaboratively (M=3.02;

visu-SD=.84) Participants’ perception of how useful the visualization was positively related to their attitudes toward collaborative writing (r=.53, p=.007) and negatively related to their attitudes toward individual writing (r=-.39, p=.06) Similar but nonsignificant patterns were found between the perceived usefulness of the replay function and attitudes toward individual and collaborative writing.

Awareness of collaborative writing

There were two different strands of awareness found: (1) participants became aware of how they worked with others in terms of communication and teamwork (e.g., social group awareness [11]); (2) participants became aware of the CW strategy they used [22] Following are example quotes on each point:

I realized the pattern of my writing It changed as I  kept on writing

I had to go back and edit the earlier written material to support the lowing material I made errors in framing the sentences which I changed too

fol-When writing in a team, communication is the key and my team municated very well We drafted our work first and then started writing Where the other person wrote first and I added points into it and mak-ing necessary changes like grammar and spelling

com-In the next section, we give examples of writing styles examining visuals from Itero and using participants’ answers to open-ended questions in the post-task survey

Writing patterns: parallel writing (Team 12)

We didn’t really work together so much as side by side.

– User 1

In his quote, User 1 describes a parallel writing strategy, that User 1 only focused on his writing and did not engage in any writing-related negotia-tion with his co-writer He did not edit any text from User 2 In the survey,

he says: “I feel uncomfortable changing the other person’s writing when

I feel like I have a better way of saying it I then feel like I am just writing the whole essay on my own, but I feel bad about it.” User 2 edited chiefly her work and very little of the text from User 1 We see that the authors started writing right away with no planning and little editing overall User

1 also commented that she recognized that she edits as she writes: finishes a

sentence, goes back and edits it rather than finishing the entire section and revision that afterward.

Writing patterns: Stratified-division writing (Team 5)

I see myself as more of an editor in the process, as opposed to lead writer.

– User 2

Here the author describes what Lowry et al call stratified-division writing, where there are specific roles among the co-writers When we replay this team’s document, we can see that writing and editing happen synchronously.User 2 further elaborates on her experience:

Individuals work very differently I understood there needed to be a quick outlining process, but the other person dove right in and started writing the content immediately I’m a slower writer, more careful, and found myself trailing behind to make edits and revisions more so than actually writing

User 1 reports himself as a lead writer but says they did start with a plan

in mind, contrary to what User 2 reported and what their analytics show Whatever plan he had was not communicated with User 2: “We started with a plan in mind to gather all the points I would start writing the key points and the other member would proofread it and make changes if any grammatical and spelling errors were there.”

Comparisons

Participants reported several observations they made between their own team’s writing and another team’s process by referring to Itero’s writing revision visualizations and replay feature The most observed difference or similarity was regarding the organization of the text or structure of the collaboration One participant noted, “We all tended to work independently.” Another said that compared to their team, the other team “had divided the work very well, and they did not repeat content, stuck to a few paragraphs and worked on them simultaneously producing and editing content.” Only one participant noted the content of the text and said, “They were more knowledgeable on the topic.” Similarly, only one participant talked about revisions, pointing out that the other team deleted more text in their essay than their team

Challenges

Participants reported some challenges during their writing process These can be categorized into (1) coordination and communication related and

(2) software related Regarding the first category, participants had culty balancing being too forceful or telling others what to do Everyone

diffi-in the study raised communication issues Followdiffi-ing is an example quote:

The most challenging aspect  .  dividing and organizing the writing Sometimes when your other writer goes off on a point for too long, it’s more difficult to tell them that they should continue onto another point

in order to meet the goals of the essay

Regarding this issue, writers can use Itero’s features to discover imbalances and communicate these imbalances to their co-writers objectively These findings also may differ based on context Previous studies found that trust and familiarity among co-authors impact their behavior: they feel they can make more changes to other people’s writing (Birnholtz et al., 2013; Birn-holtz & Ibara, 2012) Even though the setup in this experiment may reflect how CW may work in large classes, it is nevertheless in an artificial lab set-ting However, our removal of identification about participant accounts from the analytics created a barrier to identifying who is whom in visuals One of the participants said: “I found it difficult to analyze my writing process, as Itero did not identify who I was and who the other person was.”

There were some concerns about potentially making CW more tive when we added metrics and comparative visuals to it This is a related comment:

competi-I’m not a very mathematical/graphical inclined person, so the charts were kind of confusing to me It seems to make the writing process feel more competitive among the partners, which it shouldn’t be, as the pur-pose, I understand, is collaborative writing Assigning visuals, measures

to a writing collaboration makes me nervous, when I should be focusing

on the writing product

This is in line with the finding by Olson and Olson, who warned us about potential problematic aspects of monitoring for trust (Olson  & Olson, 2003) Thus, we have to be careful in our design to inform partici-pants about their writing process and highlight opportunities to improve

it without lowering their writing self-efficacy by giving them a sense that it

is a competition

Discussion

Study 2 showed that Itero may help participants’ social group awareness (e.g., how they communicate with their team) Even though most of the par-ticipants were familiar with Google Docs and were comfortable working

simultaneously on their essays, everyone needed help communicating and negotiating their ideas with their co-writers This is a well-known issue

in the CSCW field and distributed CW settings (Boellstorff et al., 2013) Future studies may investigate adding extra communication channels (e.g., voice chat) and introducing planning tools (e.g., mind mapping tools) in the writing process

Similar to the work by Olson et al (Olson et al., 2017), we show dence that we may detect writing patterns using Itero We showed examples

evi-of two teams’ work: one parallel writing and the other stratified-division writing These could be used to inform writers about their collaborative writing strategies It is, however, an open question whether one’s collabora-tive writing strategy depends on context and co-writer

Lastly, we asked participants to compare their CW with another team’s CW, using Itero’s writing revision analytic tools Overall, partici-pants could make sense of Itero’s visualization and replay tools enough

to compare them across different texts Considering that we only allowed participants to study another team’s analytics for a short time, two to three minutes, these findings show promising direction for further studies in writing instruction and writing revision analytics (e.g., improving writing through a combination of cognitive apprenticeship (Collins et al., 1991) and self-directed learning)

In summary, a combination of visualizations and a replay of the writing process gave participants a perceived objective view of their writing pro-cess Itero can provide instructors with valuable information about their students and affordances for students to self-reflect

Implications for bilingual and second language writing and instruction

The findings from Study 2 on using revision analytics in collaborative ing have several implications for bilingual and second language writing and instruction:

writ-• Enhanced understanding of writing processes The use of revision

ana-lytics provides second language learners with a deeper understanding of their writing processes By utilizing features such as replay and visuali-zations, learners can observe and analyze their writing strategies, pat-terns, and areas for improvement This increased awareness can help them become more proficient writers in their second language

• Facilitating collaborative writing The study revealed that participants

had more positive attitudes toward writing alone than writing ratively However, the perceived usefulness of the visualization feature was positively related to attitudes toward collaborative writing This suggests that revision analytics can be a valuable tool in promoting

collabo-effective collaboration among second-language learners It can help them understand their roles within a collaborative writing task, improve communication and teamwork skills, and facilitate negotiation and coordination of ideas.

• Individualized feedback and self-reflection Revision analytics platforms

can provide individualized feedback to second language learners The visualizations and replay features allow learners to review their writing processes and identify areas of strength and weakness This feedback can guide them in making targeted revisions and improvements in their writing Additionally, the opportunity for self-reflection offered by the analytics platform empowers learners to take ownership of their writing development and monitor their progress over time

• Comparative analysis and cultural awareness The ability to compare

their writing process with that of other teams using the revision lytics platform can broaden second language learners’ perspectives They can observe different approaches to collaborative writing, under-stand cultural influences on writing styles, and learn from the strengths and strategies of their peers This comparative analysis fosters cultural awareness and promotes cross-cultural communication skills in writing

ana-• Addressing challenges in collaborative writing The study identified

challenges related to coordination, communication, and software issues

in collaborative writing Revision analytics platforms can help mitigate these challenges by providing objective insights into writing imbalances, facilitating effective communication among co-writers, and promoting

a clearer understanding of individual roles and responsibilities within a collaborative writing task

• Integrating cognitive apprenticeship and self-directed learning The

findings suggest combining cognitive apprenticeship and self-directed learning approaches can be beneficial in second-language writing instruction Revision analytics platforms, such as Itero, can serve as tools for cognitive apprenticeship by providing guidance, feedback, and modeling of effective writing processes Simultaneously, they promote self-directed learning by encouraging learners to take an active role in analyzing their writing and making informed decisions about revisions and improvements

Overall, the use of revision analytics in second language writing and instruction has the potential to enhance learners’ understanding of writing processes, facilitate collaborative writing, provide individualized feedback and self-reflection, promote comparative analysis and cultural awareness, address challenges in collaborative writing, and integrate cognitive appren-ticeship and self-directed learning approaches These implications highlight the value of incorporating revision analytics platforms into bilingual and

second language writing classrooms to support learners’ development and proficiency in writing.

Study 3

This study aimed to pilot a platform that aims to minimize data privacy issues with Itero The platform allows participants to use Itero without their Google accounts

Participants and research design

We recruited 15 participants through Amazon’s Mechanical Turk (Mturk) platform to take place in the study (eight female; seven male) All partici-pants reported being native English speakers with an average age of 39.23 Participation took about two hours across two days We asked participants

to work on their essays for about 30 minutes on day one and another 30 minutes on day two The reason was that writing in multiple sessions may

be a more ecologically valid process than writing at one sitting as was done

in Study 1 and Study 2 Participants were reimbursed $15 for their time

Platform

Itero allows researchers to create customized “experiment portal” pages that research participants are linked to and can visit without requiring Google account authentication The experiment portal contains links to surveys, a writing prompt, an input form for providing their unique partici-pant ID (for example, a mTurk ID), and a button to initiate the creation of

a new Google Doc assigned to the participant The Doc is generated using the Google Drive API It is associated with a Google Service Account that allows the participant to access the Doc and closes access at the end of the experiment Users initiate closing the Doc by accessing Itero to view analyt-ics related to their writing

Procedure

Participants were asked to follow each step outlined in the experiment tal to complete the study The first step was to fill out a pre-survey that was the same survey in Study 2 In Step 2, they were asked to read the writing prompt (see Section 3.2) Step 3 took them to their individual Google Docs When the participant was ready to view the Itero analytics, they clicked on the “done” button Links to each analytics are presented on the page in Step

por-4 In Step 5, participants were given a survey link that asked them multiple questions about their attitudes toward different features of Itero (e.g., To what extent did Itero help you become aware of your writing behaviors?)

Overall, the platform worked well to experiment online without requiring participants to use a Google account All participants had unique docu-ments and could view Itero’s writing revision analytics tools Participants said they had no challenges with the procedure and found it straightfor-ward Similar to Studies 1 and 2, participants liked the replay feature the most (M=3.87, SD=.99), followed by overview (M=2.87, SD=1.30) and character trace (M=2.80, SD=1.32) Participants reported that Itero helped them become more aware of their writing process to a moderate extent (M=3.27, SD=1.22) One participant (P3) said,

ITERO was useful in seeing how my writing style was flowing On the second day, when I went to revise my writing assignment and then stud-ied the charts as well as the replay, I found that I made more revisions

on it than I thought I had It was fascinating to see this charted out in an analytical way to examine my writing style from a technical standpoint.Another participant (P10) stated, “I saw that I did not think some ideas through and had to go back, delete them, and then write them in a better way.”

Discussion

In this small study, we tested Itero over two days in an online setting using

a platform that does not require participants to have/use a Google account This was closest to an in-the-wild use of the application, as researchers did not need to monitor the use in a lab setting Participants reported no chal-lenges with the platform and had similar subjective experiences with Itero,

as reported in Studies 1 and 2 The platform provides multiple ties for researchers and instructors For instance, instructors can create an assignment for students where students would enter their student ID to create unique documents for that assignment On the backend, the instruc-tor has access to Itero analytics to keep track of students’ writing process While the current version of the platform does not allow for collaborative work, we plan to implement it in the future

opportuni-Global discussion and conclusion

Individual writing is a complex nonlinear process; the collaboration aspect makes the writing process even more complex Tools that inform writers about their writing process can improve writers’ self-regulation and self-efficacy Building and extending the early work on awareness information in

CW environments (Janssen et al., 2010; Janssen & Bodemer, 2013; Olson & Olson, 2003), we used surveys/reflections and trace analysis to investigate

the extent to which Itero may make writers aware of their writing process

We devised three small studies in three contexts: (1) individual writing with college students, (2) collaborative writing with mostly graduate-level stu-dents, and (3) online writing to test an account-free platform

While previous research highlighted the potential of API-supported replay features as a forensics tool (Roussev & McCulley, 2016), our study showed that replay features can also be a powerful tool for students Participants

in Study 1 were interested in observing expert writers’ writing processes Expert modeling may be an approach to pursue in future experiments There

is an immense opportunity that through expert modeling, we may provide

a “vicarious experience of comparing written work with that of models and comparable peers” (Bandura, 1997) Bandura further states that “com-petent models transmit knowledge and teach observers effective skills and strategies” (Bandura, 1997, p. 88) One of the unexpected findings in all three studies was that participants perceived Itero’s analytics visualizations

as “unbiased.” This raises questions about data ethics, researchers’ sibilities, and everyday people’s data literacy (Fairfield & Shtein, 2014) As researchers, we need to be conscious of communicating to users what data

respon-we are collecting and what respon-we choose to display

Multiple groups of people can benefit from Itero Educators already use Cloud-based writing tools for individual and collaborative work Itero can allow them to detect participation and writing styles to create an anchor for interventions and scaffolds for students if needed Google Docs are part

of many workplace writing practices The authors themselves use Google Docs to collaborate on multiple projects Itero can help track progress by participation and keep each member accountable Managers can use Itero

to monitor their teams’ progress on writing-related tasks

As we discussed earlier, using the platform we tested in Study 2, ers can collect writing data more efficiently to train machine learning algorithms to detect self-efficacy and writing style-related patterns auto-matically Last but not least, Itero can benefit writers in both solo and CW settings For individuals, they can monitor their writing process over time For co-writers, Itero provides data on their participation Following we also outline some ways that Itero and similar writing analytics applications may benefit second-language writers and their instructors

research-Further implications and design recommendations

• Supporting bilingual and second language writers Itero and similar

writ-ing analytics applications can benefit second language writers and their instructors These tools can provide valuable insights into the writing process, allowing learners to track their progress over time and identify areas for improvement Instructors can utilize Itero’s analytics to under-stand students’ writing behaviors better and provide targeted feedback

and support Explore ways to integrate Itero into second language ing instruction, considering the specific needs and challenges of second language learners.

writ-• Enhancing self-regulation and self-efficacy Itero, with its replay feature

and analytics visualizations, has the potential to improve writers’ regulation and self-efficacy By making writers aware of their writing processes, it allows them to reflect on their strategies, identify strengths and weaknesses, and make informed decisions about revisions Incor-porate Itero or similar tools in second language writing instruction to empower learners to monitor and regulate their writing processes

self-• Leveraging expert modeling The participants in Study 1 were interested

in observing expert writers’ writing processes Consider incorporating expert modeling as an approach in second language writing instruction Provide learners with opportunities to compare their writing work with that of expert models and peers, allowing them to gain insights and learn practical skills and strategies This can be facilitated through the replay feature of Itero or similar platforms

• Addressing data ethics and data literacy The finding that participants

perceived Itero’s analytics visualizations as “unbiased” raises questions about data ethics and users’ data literacy As researchers and instruc-tors, it is crucial to communicate clearly to users what data is being col-lected and how it is being used Promote data literacy among learners, ensuring they understand the data being collected, the purpose of the analytics, and their rights and privacy concerns Ethical considerations should be at the forefront when implementing writing analytics tools

• Supporting educators Itero can benefit educators and professionals in

workplace writing settings Instructors can use Itero to track students’ writing process, detect participation and writing styles, and provide tar-geted interventions and scaffolding when needed

By considering these implications and design recommendations, second language writing instruction can leverage the potential of Itero and similar writing analytics tools to enhance learners’ self-regulation, provide mode-ling opportunities, address data ethics and literacy, support educators, and provide valuable support for second language writers and their instructors

Limitations and future directions

Similar to any experimental study, this study had to deal with the trade-offs between external validity and experimental control Our participant sam-ple was small in all three studies, which makes it challenging to generalize findings For human computer interaction (HCI) researchers interested in developing writing revision visualizations, we found several potential addi-tions that participants highlighted that can improve the impact of writing

analytics visualizations on awareness and learning in general We found that Itero can make writers aware of their writing styles, and participants did notice similarities and differences in patterns and writing styles of their own and other teams’ However, this study did not involve a control group

In a future study, we will compare Itero with the original Google Doc ing revision history to examine the role of Itero visualizations and replay above and beyond, looking at revision history We also aim to investi-gate the impact of using Itero in a classroom longitudinal study and online experimental studies

writ-Participants also talked about the challenges of CW, emphasizing the communication gap Even though we encouraged them to use Google chat, participants did not benefit from it as much as we hoped As Olson et al emphasize, the CSCW field now has access to multiple tools to expand the theories and practices conceptualized in lab studies starting in the 1990s Google documents and tools that afford discovering patterns are ripe to try outside of laboratories (Olson et al., 2017) We needed to explore the potential contribution of Itero and found satisfactory results and fruitful design direction to improve it

Refining writing analytics design

We discovered the need for more visualizations to inform writers about co-participation and content contributions Two of the visuals we plan to implement are (1) displaying the maturity of ideas (e.g., when some of the ideas were introduced to the document) While the replay feature can give information about this visualization, this feature would make it easier to differentiate (2) In addition to showing the age of ideas on a timeline and document index axis, we will investigate the effectiveness of showing it on the actual document using heat-map-like visualizations

Participants wanted to know more about co-authors’ participation We will investigate whether visualizing participation on a paragraph or sentence level would be meaningful to users to understand co-author contribution

Privacy

We were highly concerned about users’ privacy when designing Itero We discovered that our concerns created a barrier for users to identify their work, even in two-people teams In the second study, we asked participants

to what extent they were concerned about privacy issues around Itero (1 – not at all, 2 – somewhat, 3 – a lot) The average was 1.64 (SD=0.7)

We will explore design solutions to present enough information to the ing about the co-authors Currently, users can only visualize writing revi-sion history if they own the document Another method is to implement a permission system

writ-In addition, the platform we tested in Study 3 that we are continuing

to develop further allows users to take advantage of Itero without giving third-party access to their Google Drive, which may help moderate privacy concerns The platform can also help instructors to take advantage of the application in large online or offline classrooms

(pp. 399–405) http://dl.acm.org/citation.cfm?id=169312

Bandura, A (1997) Self-efficacy: The exercise of control Freeman.

Bandura, A (2012) On the functional properties of perceived self-efficacy revisited

Journal of Management, 38(1), 9–44 https://doi.org/10.1177/0149206311410606

Birnholtz, J., & Ibara, S (2012) Tracking changes in collaborative writing: Edits, visibility, and group maintenance In Proceedings of the ACM 2012 conference

on computer supported cooperative work (pp.  809–818) http://dl.acm.org/

citation.cfm?id=2145325

Birnholtz, J., Steinhardt, S., & Pavese, A (2013) Write here, write now! An mental study of group maintenance in collaborative writing In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 961–970) Asso-

experi-ciation for Computing Machinery https://doi.org/10.1145/2470654.2466123

Boellstorff, T., Nardi, B., Pearce, C., & Taylor, T L (2013) Words with friends: Writing collaboratively online Interactions, 20(5), 58–61.

Bruning, R., Dempsey, M., Kauffman, D F., McKim, C., & Zumbrunn, S (2013) Examining dimensions of self-efficacy for writing Journal of Educational Psy- chology, 105(1), 25–38 https://doi.org/10.1037/a0029692

Ceylan, N O (2019) Student perceptions of difficulties in second language ing Journal of Language and Linguistic Studies, 15(1), Article 1 https://doi.

(pp. 487–499) Springer US https://doi.org/10.1007/978-0-387-30424-3_36

Dikli, S., & Bleyle, S (2014) Automated essay scoring feedback for second guage writers: How does it compare to instructor feedback? Assessing Writing,

lan-22, 1–17 https://doi.org/10.1016/j.asw.2014.03.006

Dourish, P.,  & Bellotti, V (1992) Awareness and coordination in shared spaces In Proceedings of the 1992 ACM conference on computer-supported cooperative work (pp. 107–114) http://dl.acm.org/citation.cfm?id=143468

Interna-Erkens, G., Jaspers, J., Prangsma, M.,  & Kanselaar, G (2005) Coordination processes in computer-supported collaborative writing Computers in Human Behavior, 21(3), 463–486 https://doi.org/10.1016/j.chb.2004.10.038

Fairfield, J., & Shtein, H (2014) Big data, big problems: Emerging issues in the ethics of data science and journalism Journal of Mass Media Ethics, 29(1),

38–51 https://doi.org/10.1080/08900523.2014.863126

Farooq, M S., Uzair-Ul-Hassan, M., & Wahid, S (2020) Opinion of second guage learners about writing difficulties in English language South Asian Stud- ies, 27(1), Article 1 http://journals.pu.edu.pk/journals/index.php/IJSAS/article/

dynam-Machinery https://doi.org/10.1145/2740908.2742846

Google Docs – Create and Edit Documents Online for Free (n.d.) Retrieved tember 15, 2017, from https://www.google.com.au/docs/about/

Sep-Greenfield, R (2003) Collaborative e-mail exchange for teaching secondary ESL:

A case study in Hong Kong Language Learning, 25.

Greer, N., Teevan, J.,  & Iqbal, S T (2016) An introduction to technological support for writing Technical Report Microsoft Research Tech Report MSR-

Hyland, K., & Hyland, F (2006) Feedback on second language students’ writing

Language Teaching, 39(2), 83–101 https://doi.org/10.1017/S0261444806003399

Janssen, J.,  & Bodemer, D (2013) Coordinated computer-supported tive learning: Awareness and awareness tools Educational Psychologist, 48(1),

collabora-40–55 https://doi.org/10.1080/00461520.2012.749153

Janssen, J., Kirschner, F., Erkens, G., Kirschner, P A., & Paas, F (2010) Making the black box of collaborative learning transparent: Combining process-oriented and cognitive load approaches Educational Psychology Review, 22(2), 139–154

stu-López, G.,  & Guerrero, L A (2014) Notifications for collaborative document editing Ubiquitous Computing and Ambient Intelligence Personalization and User Adapted Services, 80–87 https://doi.org/10.1007/978-3-319-13102-3_16

Lowry, P B., Curtis, A.,  & Lowry, M R (2004) Building a taxonomy and nomenclature of collaborative writing to improve interdisciplinary research

and practice Journal of Business Communication, 41(1), 66–99 https://doi.

org/10.1177/0021943603259363

Marzouk, Z., Rakovic, M., & Winne, P H (2016) Generating learning analytics

to improve learners’ metacognitive skills using study trace data and the ICAP framework LAL@ LAK, 11–16.

McNely, B J., Gestwicki, P., Hill, J H., Parli-Horne, P., & Johnson, E (2012) Learning analytics for collaborative writing: A  prototype and case study

In Proceedings of the 2nd international conference on learning analytics and knowledge (pp.  222–225) Association for Computing Machinery https://doi.

Olson, G M., & Olson, J S (2003) Human-computer interaction: Psychological aspects of the human use of computing Annual Review of Psychology, 54(1),

491–516 https://doi.org/10.1146/annurev.psych.54.101601.145044

Olson, J S., Wang, D., Olson, G M.,  & Zhang, J (2017) How people write together now: Beginning the investigation with advanced undergraduates in a project course ACM Transactions on Computer-Human Interaction (TOCHI), 24(1), 1–40 https://doi.org/10.1145/3038919

Pajares, F (2003) Self-efficacy beliefs, motivation, and achievement in writing:

A review of the literature Reading & Writing Quarterly, 19(2), 139–158 https://

doi.org/10.1080/10573560308222

Prat-Sala, M.,  & Redford, P (2012) Writing essays: Does self-efficacy matter? The relationship between self-efficacy in reading and writing and undergraduate students’ performance in essay writing Educational Psychology, 32(1), 9–20

https://doi.org/10.1080/01443410.2011.621411

Ranalli, J., Link, S.,  & Chukharev-Hudilainen, E (2017) Automated writing evaluation for formative assessment of second language writing: Investigating the accuracy and usefulness of feedback as part of argument-based validation

Educational Psychology, 37(1), 8–25 https://doi.org/10.1080/01443410.2015.

1136407

Robert, L P (2016) Monitoring and trust in virtual teams In Proceedings of the 19th ACM conference on computer-supported cooperative work and social computing (pp.  244–258) Association for Computing Machinery https://doi.

org/10.1145/2818048.2820076

Roscoe, R D., Jacovina, M E., Allen, L K., Johnson, A C., & McNamara, D S (2016) Toward revision-sensitive feedback in automated writing evaluation In

EDM (pp. 628–629) Association for Computing Machinery.

Roussev, V.,  & McCulley, S (2016) Forensic analysis of cloud-native artifacts

Digital Investigation, 16, S104–S113 https://doi.org/10.1016/j.diin.2016.01.013

Singer, N (2017, May 13) How google took over the classroom The New York Times https://www.nytimes.com/2017/05/13/technology/google-education-

chromebooks-schools.html

Southavilay, V., Yacef, K., Reimann, P.,  & Calvo, R A (2013) Analysis of collaborative writing processes using revision maps and probabilistic topic mod- els In Proceedings of the 3rd international conference on learning analytics and knowledge (pp.  38–47) Association for Computing Machinery https://doi.

org/10.1145/2460296.2460307

Tabuenca, B., Kalz, M., Drachsler, H., & Specht, M (2015) Time will tell: The role

of mobile learning analytics in self-regulated learning Computers & Education,

89, 53–74.

Vatrapu, R., Reimann, P., Bull, S., & Johnson, M (2013) An eye-tracking study of notational, informational, and emotional aspects of learning analytics represen- tations In Proceedings of the 3rd international conference on learning analytics and knowledge (pp. 125–134) Association for Computing Machinery https://

doi.org/10.1145/2460296.2460321

Viégas, F B., Wattenberg, M., & Dave, K (2004) Studying cooperation and conflict between authors with history flow visualizations In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 575–582) Association

for Computing Machinery https://doi.org/10.1145/985692.985765

Wang, D., Olson, J S., Zhang, J., Nguyen, T., & Olson, G M (2015) DocuViz: Visualizing collaborative writing In Proceedings of the 33rd annual ACM con- ference on human factors in computing systems (pp. 1865–1874) Association

for Computing Machinery https://doi.org/10.1145/2702123.2702517

Williams, J (2004) Tutoring and revision: Second language writers in the ing center Journal of Second Language Writing, 13(3), 173–201 https://doi.

writ-org/10.1016/j.jslw.2004.04.009

Yamada, M., Shimada, A., Okubo, F., Oi, M., Kojima, K., & Ogata, H (2017) Learning analytics of the relationships among self-regulated learning, learn- ing behaviors, and learning performance Research and Practice in Technology Enhanced Learning, 12(1), 13 https://doi.org/10.1186/s41039-017-0053-9

Yamada, M., Yin, C., Shimada, A., Kojima, K., Okubo, F., & Ogata, H (2015) Preliminary research on self-regulated learning and learning logs in a ubiquitous learning environment In 15th IEEE international conference on advanced learn- ing technologies (p. 3) https://doi.org/10.1109/ICALT.2015.74

Zhang, M., & Deane, P (2015) Process features in writing: Internal structure and incremental value over product features ETS Research Report Series, 2015(2),

1–12 https://doi.org/10.1002/ets2.12075

As Covid-19 necessitated the rapid transition to remote teaching, tions of higher education around the world found themselves in varying stages of readiness In the United States, for example, where many institu-tions provide every course, even for those that meet in-person, with access

institu-to its own learning management system (LMS), the technical infrastructure was already in place for remote teaching to occur This is not true for many other countries In a conversation with two English language professors

in Japan, where their institutions did not have their own LMS in place, the professors had to improvise ingenious solutions to reach their students (Hung, 2020) In their case, they built on Cope and Kalantzis’ (2015, p. 1) pedagogies of multiliteracies and Ito et al.’s (2020, p. 4) Connected Learn-ing, leveraging technologies familiar to them such as games and applica-tions such as Discord – a platform commonly used by gamers in which users can interact via text or video – to cope with the lack of institutional infrastructure

The challenge with taking an in-person course online is that the way

we present content and support classroom interaction is fundamentally changed by the tools we decide to use Some tools may be used based

on familiarity of use or resemblance to in-person interaction instead of sound pedagogical reasons This is further exacerbated if educators have little time to train or rethink their course design, as was the case in the early stages of emergency remote learning during Covid-19 (Barbour et al.,

2020, p. 3)

Theories for online learning

Although the contexts of learning differ in different modalities (i.e., online, face-to-face, blended, asynchronous, synchronous, hyflex, etc.), well-designed

4 Digital tools and theories to

support English language learners applications to bilingual teacher education

To-Ken Lee and Aaron Chia Yuan Hung

courses tend to build on decades of evidence-based research from the ing sciences A good overview of these theories as they apply to online learn-ing can be found in Nilson and Goodson’s (2018, p. 79) Online Teaching

learn-at Its Best, in which they present 25 principles drawn from the cognitive

sciences

Some of the cognitive principles relevant to the case to be presented are:

• Active learning Involve students in the learning process using activities

that involve collaboration, groupwork, and content creation

• Targeted feedback Give students specific feedback so they know how to

improve their understanding through further practice

• Prior knowledge Build on students’ prior knowledge so they can fit

new information into existing frameworks Teachers can use pretests to assess where their prior knowledge is and then create opportunities in the lesson to address any misconceptions

• Attention attractors Find ways to keep students’ attention Nilson and

Goodson (2018, p. 86) suggest keeping videos relatively short – six to eight minutes seems to be the ideal Text-heavy presentations are not particularly engaging so they should be replaced with more visual pres-entations (see following Multimedia Theory section)

• Cognitive load Minimize cognitive overload (see following Multimedia

Theory)

• Stories and cases Use case studies and narratives to help students

under-stand how abstract ideas and theories manifest in the real world

• Varied examples Use a variety of examples, contexts, and levels of

abstractions to help students understand and apply their learning

• Practice Provide students with opportunities to practice their learning

regularly, requiring them to retrieve both the same content and applying what they learned to new content

• Self-regulation Support students in self-monitoring so they develop a

habit of engaging in metacognitive reflection

• Error correction Create an environment in which errors are used and

appreciated as learning opportunities and not negative criticism

• Prompt feedback Provide students with feedback on assessments as

quickly as possible Sometimes this can be done efficiently through graded assignments in LMS

auto-The reliance of multimedia presentations – using presentation applications such as PowerPoint or videos – in teaching today makes it important for educators to develop a good understanding of how multimedia learning works The following section elaborates on multimedia theory in more depth

Multimedia theory

Multimedia theory, developed by Richard Mayer (Mayer, 2021), builds

on the knowledge construction view of learning (Clark & Mayer, 2016) According to this view, we process auditory (i.e., narration, sound, music) and visual (i.e., text, graphics, images, video) information through separate channels There is a limit to how much we can process in each channel simultaneously For example, driving while listening to music is a lot less distracting than driving and texting because, in the former, the driver can focus on the road (visual) while still listening to music (audio), while in the latter, the driver experiences cognitive overload (Sweller, 1988, p. 261) when trying to focus on the road and phone (both visual) at the same time The distribution may vary according to the situation For example, to stay with the driving analogy, we may find ourselves turning of the music or asking a passenger to stop talking if we have to park in a tight spot When content is presented to learners in a balanced way, then they can better engage in active processing to reflect on prior knowledge and integrate it with existing understandings (Donovan & Bransford, 2005, p. 1) When one or both channels are overwhelmed, then the viewer experiences cogni-tive overload, which can result in distraction and ineffective learning

When presenting a multimedia lesson, which typically involves tions of on-screen text, narration, graphics, images, video, and audio, it is crucial to make sure that the combination does not create cognitive over-load To that end, Clark and Mayer assembled a list of design principles for multimedia intended to help content creators strike a balance to cognitive load All these design principles were derived from numerous empirical studies in which a control and experimental group were shown different combinations of text, images, and narration, and then given a post-test to see which combinations were most effective in supporting learning Some

combina-of these principles are briefly summarized below:

• Multimedia principle Content should be presented with words and

images, instead of words alone (Sun et al., 2019, p. 10) Here, graphics

do not refer to any imagery, but specifically to information-rich and relevant graphics such as graphs, charts, maps, images that illustrate procedures, time-lapse animations, and so on Educators should avoid

decorative graphics that may add humor or fill up space but otherwise

offer no relevant information to the topic discussed Imagery and sound are often used in language learning and the Multimedia Principle recom-mends educators carefully consider what types of images they choose

to use (Mayer et al., 2014, p. 658; Lee & Mayer, 2015, p. 451) For example, the stylized emoji of a heart might not be the best image to pair with the word “heart” if the idea is to teach about the human organ

The overuse of stylized, decorative graphics ought to be minimized and replaced with better, more relevant images.

• Contiguity principle This principle encompasses two concepts The tial Contiguity Principle recommends captions and labels to be arranged

Spa-close to the corresponding image For example, instead of placing the labels to an image separately at the bottom, side or, worse, on another page, labels should be aligned close to the matching image The Tem-

poral Contiguity Principle recommends that spoken word, such as

nar-ration, be timed according to the corresponding image For example, if

a video presentation involves showing an image, the spoken narration should be timed to occur when the image is introduced The Contiguity Principle is relevant for all learners, but especially for English language learners (Türk & Erçetin, 2014, p. 20) Educators should break down complex images into simpler parts and present them in time with spo-ken narration in order to reduce cognitive load (Kanellopoulou et al.,

2019, p. 9)

• Modality principle Narration is better than on-screen text when used

in conjunction with complex graphics This is again due to the need to balance the auditory and visual channels When content is presented as both on-screen text and images, then the visual channel is overloaded When content is presented visually, and accompanied by narration instead of text, then both channels are used, and cognitive overload is reduced This is not to suggest that on-screen text should never be used Clark and Mayer recommend using it when introducing technical terms

p. 658) show that subtitles do not help non-native English speakers in performing better, though they add that it might help if the pacing of the video is slowed and/or if the learner has control over the pace

• Coherence principle Extraneous content – including words, images,

and audio – can hinder learning Here, extraneous content is defined not only as irrelevant content but also content that might be interest-ing but not germane to the topic being presented Clark and Mayer also argue that, in some cases, using a simpler graphic is better than a complex one if that complexity is not part of the topic being discussed Likewise, extraneous audio, such as background music or sound effects, can potentially cause cognitive overload, especially when the material is new and/or complex

• Personalization principle When presenting information, use

conversa-tional language (in text or narration) instead of formal, polite language (Mayer et al., 2004, p. 389) For example, if you are using multimedia

to convey information, narrate the presentation as if you were ing with the learner directly (Moreno  & Mayer, 2000, p.  730) This makes the content more engaging and easier to understand Perhaps the best example of this is the popularity of multimedia used for educa-tion, ranging from children’s educational television programs to You-Tube educational videos to TED Talks, all of which engage learners by being conversational, informal, and friendly, instead of being formal, academic, and serious

interact-• Image principle Having a talking head (i.e., when the narrator speaks

directly to the learner) does not add to learning (Mayer, 2005) In other words, visuals are best used if they are relevant to the topic being dis-cussed, which typically does not involve the narrator’s face

• Segmenting principle Complex lessons should be broken into smaller,

simpler lessons (Mayer & Chandler, 2001, p. 396; Mayer & Pilegard,

2014, p.  316) This principle seems straightforward but can often be underappreciated, especially when the instructor is an expert on a topic and does not appreciate how something that seems simple to her may be complex to novice learners (Bransford et al., 2000, p. 44), especially if they are also English language learners

• Pretraining principle Complex concepts/words should be presented

upfront to learners before the lesson commences This should be familiar

to instructors of English language learners, who may frontload lary words to ensure that students understand the key words they will encounter in the lesson

vocabu-Applying multimedia theory

While the principles just outlined may seem straightforward and reasonable

in theory, implementing them is far more challenging in practice Many educators who are used to text-centric lectures may have to redo them to include more graphics Even if graphics can be found online, they may not

be ideal if they are too complex or include extraneous content Finding proper images would involve a more thoughtful process and even require educators to have some basic image editing skills to crop out any extrane-ous content Many educators may also use videos created by individuals who are not familiar with Multimedia Theory Content creators who are focused on drawing attention and clicks will likely create videos that violate these principles because they want to keep viewers watching This means viewers will keep watching the video because they are entertained, but not stop to reflect and engage in metacognitive processes necessary to learning

Another challenge, especially in the case of videos, is that it can be ficult to find relevant images to display throughout the video Having a static image on-screen for too long can be boring, even distracting, which may be why many content creators include large sections of themselves speaking to the audience Otherwise, content creators will have to fill the visual space with stock photos or videos that might not be crucial to the topic being discussed.

dif-Applying multimedia theory to English language learners adds another layer of challenge Multimedia Theory was not derived with English lan-guage learners in mind, and many of the original studies focused on native English speakers Native and non-native English speakers may have dif-ferent criteria for what text and visuals are considered irrelevant or dis-tracting (Liu et  al., 2018) For example, Shadiev et  al (2017, p.  1239) suggest that verbatim speech-to-text may help university-level ESL stu-dents learn more effectively Similarly, Liu et al (2018, p. 198) suggest that the Modality Principle and Redundancy Principle did not apply to university-level ESL students (though it should be noted that their sample sizes were small)

The Multimedia Theory principles are further softened when users have more control over the videos In theory, if students have the opportunity to pause and rewind videos, they can always go back and re-watch anything they missed The problem is that students may not always do that, or even realize there was something they missed As the following case study will show, this is often what happens

Case study

Introduction to higher education and online learning in Taiwan

The spread of Covid-19 brought unprecedented and complicated lenges to educational institutions, impacting approximately 70% of stu-dents around the world (Giovannella, 2020, p.  3) Although the highly contagious nature of Covid-19 made its spread difficult to contain, Tai-wan was one of the few countries in which most universities were able to successfully maintain regular lessons in the spring semester of 2020 (Shih

chal-et al., 2021, p. 104) Even with the social distancing and facial-mask dates, students could maintain their normal campus lives uninterrupted However, with the burgeoning number of confirmed cases since the May of

man-2021, many colleges and universities started to cancel all on-campus ings and move courses online (Shih et al., 2021, p. 104)

meet-This case study describes the use of online technologies to facilitate dents’ English learning at a university in Taiwan Under the continuous

stu-ebb and flow of the pandemic, the benefits and drawbacks of using online learning for an extended period of time warrants further investigation and evaluation The shift to online has raised several issues For instance, how can teachers ensure that students engage in learning? What are the best assessments to use for students who learn online? What can teachers do

to improve their interactions with students online? Although numerous studies have covered these questions (Shih et al., 2021, p. 110), it was dif-ficult for teachers who had to make an abrupt transition to know all the research The concerns, accompanied by moving online, posed challenges

on teachers’ instructional effectiveness

Since 2019, the Language Center at the targeted school has been working on ways to motivate students’ English learning during their leisure time One of the missions of the Language Center is to enhance its students’ and faculty members’ English proficiency, global vision and mobility, and multicultural awareness Accordingly, the faculty at the Language Center have already embarked on using technologies such

as livestreaming or online meeting to make students’ English ing more flexible After the outbreak of the pandemic, the vital need

learn-of online teaching and learning further stimulated teachers to master the necessary tools for online education and attempted to connect tra-ditional instruction with innovative technologies (Mohmmed et  al.,

2020, p. 2)

In response to the Taiwan government’s policy of becoming a bilingual country, the targeted university aims to become a bilingual campus by the year of 2030 In addition to the Language Center, the school also built the Bilingual Teaching Center (BTC) in 2021 The BTC provides training programs in English as a Medium of Instruction to instructors and teach-ing assistants During the pandemic, the BTC also provided guidance for adapting conventional course design into an EMI (English as Medium of Instruction) course Both the Language Center and the BTC work together toward guiding the instructors with limited English competency and online teaching experience to exploit the school online resources and tools to make the transition smoother

Several studies have highlighted that the epidemic opened up more sibilities and interests for online learning (Adedoyin  & Soykan, 2020,

pos-p. 1; Shih et al., 2021, p. 108) Due to the fact that almost all types of classes were transferred to online mode during the pandemic, teachers were encouraged to try out different ways to deliver their instruction and investi-gate what instructional methods or pedagogical approaches would be most effective for online learning For the targeted school, the Language Center has already taken initiatives to promote students to learn English online prior to the pandemic