Factors that influence learner engagement and completion rate in an xMOOC on energy and sustainability

In Mexico, there has been a rise in Massive Open Online Course (MOOC) enrollments through platforms such as MexicoX. However, this rise in interest has not been accompanied by a corresponding increase in completion rates. This article examines the factors that influence Mexican learners’ retention rates and learner engagement to determine the extent to which a student´s profile can predict his or her ability to engage with and complete an xMOOC on energy and sustainability. Correlation and multiple regression analysis methods were employed to analyze a sample dataset (n = 844) of participants who had completed the xMOOC.

Knowledge Management & E-Learning, Vol.12, No.2 Jun 2020 Factors that influence learner engagement and completion rate in an xMOOC on energy and sustainability José Antonio Canchola González Leonardo David Glasserman-Morales Tecnologico de Monterrey, México Knowledge Management & E-Learning: An International Journal (KM&EL) ISSN 2073-7904 Recommended citation: Canchola González, J A., & Glasserman-Morales, L D (2020) Factors that influence learner engagement and completion rate in an xMOOC on energy and sustainability Knowledge Management & E-Learning, 12(2), 129–146 https://doi.org/10.34105/j.kmel.2020.12.007 Knowledge Management & E-Learning, 12(2), 129–146 Factors that influence learner engagement and completion rate in an xMOOC on energy and sustainability José Antonio Canchola González* School of Humanities and Education Tecnologico de Monterrey, México E-mail: antonio.canchola@hotmail.com Leonardo David Glasserman-Morales School of Humanities and Education Tecnologico de Monterrey, México E-mail: glasserman@tec.mx *Corresponding author Abstract: In Mexico, there has been a rise in Massive Open Online Course (MOOC) enrollments through platforms such as MexicoX However, this rise in interest has not been accompanied by a corresponding increase in completion rates This article examines the factors that influence Mexican learners’ retention rates and learner engagement to determine the extent to which a student´s profile can predict his or her ability to engage with and complete an xMOOC on energy and sustainability Correlation and multiple regression analysis methods were employed to analyze a sample dataset (n = 844) of participants who had completed the xMOOC It was found that the critical factors affecting completion rate were age, education level, and primary occupation and that participants who were most likely to complete an xMOOC were 34 years of age or older, had a bachelor’s degree or higher, and were in a full-time job Keywords: Learner engagement; Completion rate; xMOOC; Distance education Biographical notes: Antonio Canchola is a PhD student in the Educational Innovation program at Tecnologico de Monterrey His research interests center around Digital literacy, Adult education, and Lifelong learning He carried out a research stay on the Graduate School of Education at the University of California, Berkeley He is a member of the Research Group on Innovation Education in the School of Humanities and Education at Tecnologico de Monterrey Dr Leonardo Glasserman is a Professor and Researcher in the School of Humanities and Education at Tecnologico de Monterrey He is a member of the Research Group on Innovation in Education He has been technical manager and partner in research projects funded by the National Council of Science and Technology (CONACYT) and CITRIS-ITESM Seed Funding He is a member of the National System of Researchers in Mexico Currently, he serves as director of the M A in Entrepreneurship in Education and the M.A program in Management of Educational Institutions at Tecnologico de Monterrey 130 J A Canchola González & L D Glasserman-Morales (2020) Introduction There are now many options for adults to improve their professional skills and competencies without the need to enroll at traditional educational establishments In particular, distance education, which is a flexible educational system where there is no physical contact between the instructor and the learners (Akhter, 2015), has become increasingly popular in the last twenty years While distance learning has been around for at least 50 years, nowadays, online courses are available, and in particular, Massive Open Online Courses (MOOCs) have grown in popularity Since its inception in 2008, the MOOC can be considered as a pedagogical strategy, a multi-domain knowledge base, and a technological tool able to stimulate creativity, autonomy, and social-networked learning (Cirulli, Elia, Lorenzo, Margherita, & Solazzo, 2016) MOOCs allow hundreds or thousands of students to enroll and study because access to the lectures is free for everyone, regardless of education level, geographic location, language, or time zone; however, some have certification fees MOOCs are completely online based, require an internet connection, and have definitive course plans that include learning objectives and activities based on specific instructional designs Many MOOCs also have a prefix based on their particular purpose; for instance, the letter c (for connectivist) in cMOOCs that prioritize the connection between the students using diverse tools such as social media for collaborative creation, with a teacher’s guide to assist students when necessary (Fontana & Leffa, 2018), and x (for eXtended MOOC) in xMOOCs, which have concise, targeted short videos rather than full-length lectures, and use automated testing to check students’ understanding as they work through the content (Xu & Yang, 2015) The retention and completion rates of students taking MOOCs are under 10% (Liyanagunawardena, Adams, & Williams, 2013; Rai & Chunrao, 2016) These figures are a cause for concern among educators (Koller, Ng, Do, & Chen, 2013) as, in a traditional context, the completion rates in courses, and graduation rates in colleges have long been important metrics for measuring college success (Reich & Ho, 2014) However, researchers have found it misleading to consider course completion as the sole indicator of success in MOOCs (Pursel, Zhang, Jablokow, Choi & Velegol, 2016) While trends and statistics demonstrate the overall experience of students in these types of courses, they fail to provide more nuanced insights from the learners themselves (Loizzo, Ertmer, Watson, & Watson, 2017) Therefore, recent research has shifted from an outcome-related perspective to a more individual one (Henderikx, Kreijns, & Kalz, 2017) This paper explores the extent to which personal factors (e.g., gender, age, education level, primary occupation, and previous experience with online courses) affect the level of learner engagement and course completion Correlations and multiple regression analyses are then employed to assess the relationships between the participants’ profiles and their final grades, to develop a model that is able to predict the most critical factors related to the highest end-of-course grade The objective of this paper is to determine the relationship between learner engagement and completion rates in online distance education, and specifically in xMOOC courses A quantitative approach, using descriptive and inferential statistics, was used to analyze an initial survey on interests, motivations, and prior xMOOC knowledge, after which the results were compared with the achievement databases for 11,944 participants The surveys were completed by 8,124 participants from three xMOOCs on energy and sustainability that were designed and developed by Tecnologico de Monterrey Knowledge Management & E-Learning, 12(2), 129–146 131 as a part of a project called “Binational Laboratory on Smart Sustainable Energy Management and Technology Training” A sample of 844 users, or 10.38% of those who completed their course, was then extracted to examine the specific factors that could possibly predict completion and course grading Literature review 2.1 Completion rate A MOOC takes advantage of technology that promotes access to thousands of people wishing to educate themselves within a “knowledge society” However, the massive entry of participants at the start of a MOOC course is rarely reflected in the completion results Several researchers (Jordan, 2014; Kruchinin, 2019; Liu, He, & Cai, 2018; Liyanagunawardena et al., 2013; Lyu, Chan, & Yeung, 2019; Rai & Chunrao, 2016; Reich & Ruipérez-Valiente, 2019; Romero-Rodriguez, Ramirez-Montoya, & Gonzalez, 2019) have reported low completion rates of participants in MOOCs This has led instructional designers, educational institutions, teachers, and investors to question the efficiency of MOOCs The term “completion rate” is defined as the percentage of students who pass the exam from the total number of students who register on the course (Liu et al., 2018) However, it has been observed that many students enroll without even starting the course, and Jordan (2014) suggests that the completion rate would be better characterized as the proportion of active students completing the course Therefore, assessing the success of MOOCs merely on its completion figures ignores other key factors such as the individual characteristics of the students (Henderikx et al., 2017), which are important in a virtual learning ecosystem, and different, in many aspects, from the context of traditional education (face-to-face) Since the inception of the concept, MOOCs have largely reported completion rates below 10% (Liyanagunawardena et al., 2013; Rai & Chunrao, 2016) For example, a 14-week course called “6.002x: Circuits and Electronics” offered in 2012 by the Massachusetts Institute of Technology (MITx) registered an enrollment of 154,763 students and only 7,157 (4.62%) of them fully completed the course (Romero-Rodriguez et al., 2019) Another example is the course “Information Theory” designed by the Chinese University of Hong Kong that registered 10,953 participants in 2014 and only 0.137% of the total completed it (Lyu et al., 2019) Another university in China offered a course called “Ancient Chinese Architectural Art” between 2015 and 2016 with a total of 29,099 participants, of which only 678 students (2.33%) successfully completed (Liu et al., 2018) From the above information, it is apparent that the completion rate has remained constant over time, with a range of between 0.1% and 10% of termination efficiency This is confirmed by Jordan (2014), who studied 39 MOOCs between 2012 and 2013, reporting that the typical completion rate was 5%; and Kruchinin (2019), who, in 2017, analyzed 132 MOOCs from different platforms (Coursera, edX, Udacity, and FutureLearn) and determined that, generally, 4.5 students from each hundred completed the course More recently, an analysis by Reich and Ruipérez-Valiente (2019) that analyzed 261 courses on the edX platform, with a total of 5.63 million participants, between 2012 and 2018, revealed that completion rates are still between 6% and 10% 132 J A Canchola González & L D Glasserman-Morales (2020) In attempting to explain this trend in low completion rates for MOOCs, researchers have focused on different aspects; for example, the characteristics and behavior of the participants (Kizilcec, Piech, & Schneider, 2013; Rai & Chunrao, 2016); the types of MOOCs that attract students (Kruchinin, 2019); the technological or instructional design (Cirulli et al., 2016; Jordan, 2015); and the factors that influence an individual's intention to use or complete a MOOC (Alraimi, Zo, & Ciganek, 2015; Canchola González & Glasserman-Morales, 2019; Daneji, Ayub & Khambari, 2019) Among the main characteristics of the participants, it has been observed that most of those who complete MOOCs already have one or two years of undergraduate education, have completed a Master’s degree, or even have a doctorate (Chernova, 2013; Kilgore, Bartoletti, & Freih, 2015; Loizzo et al., 2017); that is, the most educated participants (Emanuel, 2013; Greene, Oswald, & Pomerantz, 2015) are the most likely to successfully conclude a MOOC 2.2 “Student engagement” or “learner engagement”? “Student engagement” has various definitions (Steele & Fullagar, 2009; Deng, Benckendorff, & Gannaway, 2020) Academic literature reflects little consensus on the way engagement is operationalized and measured (Appleton, Christenson, & Furlong, 2008) Astin (1999) defined it as the amount of physical and psychological energy that the student devotes to the academic experience While there has been significant research focused on traditional learning (face-toface); the analysis of participant engagement in online courses and/or distance education has attracted more recent research attention Given this, researchers such as Deng et al (2020), make a valuable distinction between “student engagement”, understood as a term widely adopted in the traditional educational field; and “learner engagement”, a concept used to refer to course engagement in MOOCs (distance education), since not all learners behave or assume themselves as traditional students Therefore, for this study researchers have adopted the concept of “learner engagement” as it better fits the context Different conceptual approaches have been given for learner engagement For example, Arghode, Brieger, and Wang (2018) denominated it as the ability of students to actively interact and critically examine the course content at cognitive, behavioral, and emotional levels On the other hand, Chakraborty and Nafukho (2014) described the concept of learner engagement as the levels of interest exhibited by students and interaction with the content, the instructor, and/or peers In turn, Ballard and Butler (2016), identified it as an active and spontaneous process carried out by the learner in response to directed activities aimed at developing higher-order thinking skills It seems that although there is a lack of agreement on a unique definition of the concept, the importance of learner engagement is underscored, and linked to positive outcomes such as student success and development (Leslie, 2019) Primarily, it has been student intentions and motivations that have been associated with completion rates (Engle, Mankoff, & Carbrey, 2015; Petronzi & Hadi, 2016) In the context of MOOCs, engaging participants is more challenging due to the large and diverse group of individuals that are attracted to such courses (Hew, 2016) In addition to this, they are engaged remotely and from a wide range of backgrounds (Rai, Yue, Yang, Shadiev, & Sun, 2017) There is also evidence of people joining a MOOC just to follow a class or simply to experience the MOOC format (Sunar, White, Abdullah, & Davis, 2017) Knowledge Management & E-Learning, 12(2), 129–146 133 Researchers have identified factors that influence learner engagement For example, the quality and content of video (Kim, Guo, Seaton, Mitros, Gajos, & Miller, 2014), course materials in general (Wong, Khalil, Baars, de Koning, & Paas, 2019), teacher–student interaction (Callahan, 2010), the sociocultural context (Arghode et al., 2018), student characteristics (Engle et al., 2015; Gil-Jaurena, Callejo-Gallego, & Agudo, 2017), and demography (Arslan, Bagchi, & Ryu, 2015; Shalem, Bachrach, Guiver, & Bishop, 2014) Therefore, for this study of MOOCs in Mexico, the following research questions were developed RQ1: Amongst Mexican students on an xMOOC, what is the relationship between their profiles and their final grades? RQ2: Which demographic factors have a greater effect on the final grade? RQ3: Is there a specific profile for Mexican participants that determines the final grade of their online courses? Method 3.1 Study overview Tecnologico de Monterrey (Tec de Monterrey), a private, nonsectarian, co-educational multi-campus university based in Monterrey, Mexico, has offered twelve different energy and sustainability international xMOOCs since 2016; Energy past, present, and future; Mexico´s energy reform; Conventional and clean sources of energy; Mexico´s power industry; Carbon markets; Energy markets; Electric power; and five others The courses were promoted through different open international online hosting platforms, such as edX and its Mexican adaptation, MexicoX It was recommended that these xMOOC courses be taken sequentially even though the content was independent Each course was six weeks long, delivered in Spanish, and was free and open to the general public The courses were created as part of a project initiative called the Binational Laboratory for Smart Sustainable Energy Management and Technology Training (http://energialab.tec.mx/en) and were part of the highest funded project given to a private institution in Mexico by the Mexican Federal Government through the Ministry of Energy and the Mexican Council on Science and Technology (CONACYT, in Spanish) The objectives of the Binational Laboratory are: a) to train national and international technicians and professionals; b) to create global research networks; c) to strengthen the infrastructure for the development of applied scientific teaching and research; and d) to develop physical and virtual laboratories for learning and research (Nava, 2016) The MOOC sub-project forms part of the objective to train people in energy and sustainability This study took a quantitative approach to analyzing a dataset of participants using a correlation matrix and multiple regression statistical methods for which Quantitative Minitab 18 software was used to process and analyze the student data set 134 J A Canchola González & L D Glasserman-Morales (2020) 3.2 Participants These xMOOC courses attract new enrollment from all over the world, even from countries in which Spanish is not the native language such as the United States and France; however, the majority are from Latin American countries For this study, only users from Mexico were examined The raw data contains 11,944 records from the Mexican learners who answered the initial MOOC survey between 2017 and 2018 Table shows the number of participants enrolled in each xMOOC Table Types of xMOOC and numbers of users xMOOC Users Electric power: concepts and basic principles 3,790 Energy: past, present, and future 3,876 Conventional, clean energies, and their technology 4,278 Country Dates México 2017–2018 Around 8,000 Mexican learners answered the initial survey question: “What is your level of commitment to this course?” for which there were six response options Only learners who responded and chose options one and two were selected for this study Below are the option responses for the question: “What is your level of commitment to this course?” I plan to complete all the activities and exams to finish the course, even if I not get the certificate I plan to carry out all the activities and exams as I am interested in the certificate I plan to see all sessions, specific tests, and some activities, but I am not interested in completing the course I am only interested in consulting some videos and course materials I am interested in knowing what the course is about, but I not plan to see the sessions or complete the activities Other (specify) As can be seen, options three to six not commit the learner to complete the xMOOC and only involve reviewing the materials or understanding the focus of the course (general information) For this study, the term “participant” was taken to be equivalent to a “learner” in an xMOOC Table shows the number of the participants who completed the initial survey and chose options one or two for one of the three xMOOCs examined in this study Table Participants who completed the initial survey xMOOC Option Option Total Electric power: concepts and basic principles 444 1,939 2,383 Energy: past, present, and future 499 2,340 2,839 Conventional, clean energies, and their technology 470 2,436 2,906 Total 8,128 Knowledge Management & E-Learning, 12(2), 129–146 135 3.3 Instruments The initial survey on interests, motivations, and prior MOOC knowledge (Vázquez, Ramirez-Montoya, & Gónzalez, 2018) had a mixed format and was answered by participants through the online Survey Monkey tool (https://bit.ly/2Z7muli) There were 28 items, which were multiple-choice or four-level Likert-scale questions, across three sections, as described in the following paragraph Part had 13 general background questions; name, gender, date of birth, country of origin, maximum education attained, primary occupation, and previous MOOC experience Part contained nine questions focused on interest, motivation to study, and reasons for selecting the MOOC; and Part had six questions about computer skills and competency, and general awareness of energy issues The instrument validity and reliability were analyzed using a Vázquez et al (2018) exploratory factorial analysis, from which a Cronbach's alpha of 898 was reported, indicating that the survey results were stable The validity and reliability were confirmed through the analysis of two xMOOCs that were launched in January 2017 Data analysis Descriptive statistical and inferential statistical data analyses were conducted using Minitab software version 18 4.1 Descriptive statistics As shown in Table 3, of the 8,128 participants, 66.58% were male, and the ages ranged from 15 to 74 years old, with a mean of 32, with nearly one third being between 25 and 34 years old (32.86%) and around 30% being between 15 and 24 years old Most learners had a bachelor’s level of education (53.19%), less than 1% were retired, and 42.43% were full-time employees For almost half (47.35%), this was the first time they had enrolled in a MOOC, and only 7.73% had completed more than two MOOCs The study focused on learners with the following profile; they completed the course, chose options one or two to the question on commitment in the initial survey, and received a final grade between and 10 Of the 8,128 participants enrolled in one of three energy xMOOC courses, only 844 (10.87%) completed their course (all the activities and quizzes), and received a final grade Table gives the details of these participants In this final learner group, 70.97% were male, around one third (33.41%) were between 15 and 24 years old, seven were up to 65 years old, and the mean age was 33.6 years Almost half (48.34%) had a bachelor’s degree, just over 15% had a master’s degree, and 3.31% had a doctorate degree Nearly half (42.89%) were employed full-time, and nearly 30% were undergraduate students Around 14% had previously completed three or more MOOCs, 9.12% had completed two or more MOOCs, and over 20% had completed at least one MOOC; therefore, less than half (44.66%) were taking a MOOC for the first time 136 J A Canchola González & L D Glasserman-Morales (2020) Table Participant profiles for three xMOOCs Gender Age Education Main occupation Previous experience whit a MOOC Male Female 15–24 25–34 35–44 45–54 55–64 65–74 Bachelor´s degree High school Master´s degree Associate degree Other Doctorate Full-time employee Undergraduate student Part-time employee Other Own business Unemployed Postgraduate student Retired It is the first time that I signed up for a MOOC I have participated in and completed two MOOCs I have participated in and completed three or more MOOCs I have participated in and completed one MOOC I had already registered for at least one MOOC, but I did not complete it Frequency 5,412 2,716 2,410 2,671 1,624 1,017 358 48 4,324 1,186 1,102 1,075 288 153 3,449 2,053 911 594 419 356 282 64 % 66.58 33.42 29.65 32.86 19.98 12.51 4.40 0.59 53.19 14.59 13.55 13.22 3.54 1.88 42.43 25.25 11.20 7.30 5.15 4.37 3.46 0.78 3,849 47.35 629 7.73 1,045 12.85 1,561 19.20 1,044 12.84 Table Participant profiles for those who had completed one of the three xMOOCs and achieved a grade between and 10 Gender Male Female Frequency 599 245 % 70.97 29.03 Knowledge Management & E-Learning, 12(2), 129–146 Age Education Main occupation Previous experience with a MOOC 15–24 25–34 35–44 45–54 55–64 65–74 Bachelor´s degree Associate degree Master´s degree High school Doctorate Other Full-time employee Undergraduate student Other Part-time employee Own business Postgraduate student Unemployed Retired It is the first time that I sign up for a MOOC I have participated in and completed two MOOCs I have participated in and completed three or more MOOCs I have participated in and completed one MOOC I had already registered for at least one MOOC, but I did not complete it 137 282 216 158 120 61 408 133 127 126 28 22 362 244 85 67 32 28 23 377 33.41 25.59 18.72 14.22 7.23 0.83 48.34 15.75 15.04 14.92 3.31 2.60 42.89 28.90 10.07 7.93 3.79 3.31 2.72 0.35 44.66 77 9.12 122 14.45 179 21.20 89 10.54 4.2 Inferential statistics Only those participants (n = 844) who had completed the course and received a grade between and 10 out of 10 were analyzed in this study to identify the possible variable correlations that predicted xMOOC completion Table shows the correlation matrix for the grades (6–10) and the independent variables; gender, age, education level, main occupation, and previous experience with an xMOOC 4.2.1 Correlation analysis The results showed that using Pearson’s correlation coefficient between grade vs gender, age, education level, main occupation, and previous experience with an xMOOC, the strength of the relationships between the variables were less than moderate; for example, the highest correlation coefficients were 0.200 between grade and education level and 0.168 between grade and age The Pearson’s correlation between grade vs gender was 138 J A Canchola González & L D Glasserman-Morales (2020) 0.033, between grade and main occupation was -0.076, and between grade vs previous experience with an xMOOC was -0.047, which indicated that as gender, main occupation, and previous experience increased, the grade decreased If p-values are used to demonstrate that the correlation grade and the other variables are statistically significant, they must be less than, or equal to, the significance level; that is, p-value ≤ α (0.05) In this case, the p-value for both the correlation between grade and age, and grade and education were 0.000 and between grade and the main occupation was about 0.027, which were less than the significance level of 0.05, which indicated that the correlation coefficients were significant Table Pearson’s correlation coefficient Gender Age Age Education Main Occ Previous Exp 0.105* 0.002 Education 0.029 0.420* 0.394 0.000 −0.028 −0.109* −0.204* 0.412 0.002 0.000 −0.073* 0.091* 0.147* 0.015 0.033 0.008 0.000 0.670 −0.033 0.168* 0.200* −0.076* −0.047 0.336 *p-value is significant 0.000 0.000 0.027 0.172 Main Occ Previous Exp Grade 4.2.2 Multiple regression The sample of 844 participants was considered large enough to obtain a precise estimate of the strength of the relationship between grade and gender, age, education level, main occupation, and previous experience with an xMOOC Therefore, multiple regression analyses were conducted using the response optimizer assistant in Minitab 18 The results are shown in Fig 1, in which it can be seen that the relationships between grade and the independent variables were statistically significant (p < 0.10) Therefore, the regression model explained 6.36% of the variations in grade (Y) Fig Relationship between Y (Grade) and X (independent variables) Knowledge Management & E-Learning, 12(2), 129–146 139 The final model equation for grade is shown in Fig The variables were X1 Age, X2 Education, X3 Main Occupation, X4 Previous experience with an xMOOC, and X5: Gender Fig Final model equation When the model equation was interpreted as if the variables were one unit, the grade increased 0.01174 because of age, 0.2855 because of education level, and 0.0094 because of main occupation; however, it decreased -0.1909 where there was previous experience with an xMOOC Therefore, the participants’ grades were affected by each independent variable, the details for which are shown in Fig Grades were found to increase if the participants were between 40 and 60 years old; however, if they were male, the grade decreased (1 = Female, = Male) If learners had a master’s degree (5), they achieved the highest grade, followed by a bachelor’s degree (4) and other levels of education (6); nevertheless, if they had an associate degree (2) or high school degree (1), the grade decreased Surprisingly, if participants had a doctorate degree (3), they had the lowest grades Fig Main effects plot for grade The analysis of the critical effects of grade and main occupation found that if learners were unemployed (1), full-time employees (2) or retired (6), they had the highest grades at around 8.5, but if they were part-time employees (3), an undergraduate student (4), had their own business (7) or other occupation (8), they had grades around 8.0 However, the postgraduate students (5) had the lowest grades The analysis of the critical effects of grade and participants with xMOOC experience found that if it was the first time that they had signed up for an xMOOC (1), participants had the highest grade if they had completed one (4) or two xMOOCs (2) (with a grade around 8.2), and if they had previously registered, but not completed, at least one xMOOC (5), they had the lowest grade The software Minitab 18 provided five alternative options to predict the highest grade related to the variables; X1 age, X2 education, X3 main occupation, and X4 previous experience with an xMOOC, as detailed in Table 140 J A Canchola González & L D Glasserman-Morales (2020) Table Top five alternative solutions for the highest grade X1 X2 X3 X4 Predicted Y 59 9.15431 56 9.11909 65 2 9.10072 43 1 9.08296 34 9.08264 Note: X1: age, X2: education, X3: main occupation, X4: previous experience with an xMOOC The data showed that the learners’ profiles exhibiting the highest grades were: aged between 43 and 65 years old (X1-age), had a Master’s degree (5) (X2-education), were a full-time employee (2) (X3-main occupation), and, had signed up for an xMOOC for the first time (1) (X4-previous experience with an xMOOC) Discussion Although the concern of the different actors that promote, develop and support MOOCs is evident, some researchers see that completion rates not suffice, or at best, are an incomplete measure, for evaluating MOOCs; and at worst, are a measure that threatens the goal of educational access that originally motivated the creation of MOOCs (Henderikx et al., 2017; Reich & Ho, 2014) Previous research sought to measure the impact of demographic factors such as gender, age, geographic location, and education level on MOOC success (Arslan et al., 2015; Shalem et al., 2014), and proved that these variables could be assessed in this context In that sense, different research approaches to understanding the completion rate have been taken For example, Coursera founder Daphne Koller proposed moving away from completion rate as the standard measurement of success in MOOC to one that focuses on learner intentions (Loizzo et al., 2017) On the other hand, researchers such as Henderikx et al (2017), suggest taking individual intention as a starting point for the discussion about dropout and success in MOOC Thus, this paper explored the relationships between various variables, specifically the sociodemographic (educational level, age, main occupation, etc.), previous experience with a MOOC, and the final grade for an xMOOC in Mexican learners registered for three different MOOCs Surprisingly, the final completion rate of the three energy and sustainability xMOOCs studied was 10.38%, showing a tendency superior to those reported in previous research (Jordan, 2014; Kruchinin, 2019; Liu et al., 2018; Reich & Ruipérez-Valiente, 2019) from international contexts, where an average of 5% was observed This may be due to the support of an elite institution such as Tec de Monterrey who, in coordination with the Mexican government, have promoted the project widely; and also, due to the particular interest that the Mexican population has in free online training opportunities on issues surrounding energy and sustainability This investigation found that there was a correlation between the grade and age, education level, and main occupation, but not between gender and previous xMOOC experiences This was consistent with Liang, Jia, Wu, Miao, and Wang (2014) who noticed in their investigation that there was no statistically significant impact on the behavior or outcomes if a learner had previously taken a MOOC Similarly, Pursel et al Knowledge Management & E-Learning, 12(2), 129–146 141 (2016), found that prior learning online experience in MOOC (or other online formats) had no impact on participant completion Moreover, Gil-Jaurena et al (2017) observed that 70.5% of learners who participated in the initial questionnaire stated that this was their first MOOC experience It is generally believed that the more educated the participants, the better their academic performance Arslan et al (2015) stated that there was evidence that education level and age were significant for achieving certification from a MOOC platform in developed countries, which was also confirmed by Shalem et al (2014) who found substantial evidence for a positive correlation between student educational level and performance Additionally, Pursel et al (2016), found that the majority of the students were in their 30s, with a Bachelor´s degree or greater, and this led them to believe that participants who complete a MOOC are well educated and comfortable in situations that require a high degree of self-directed learning In the same way, Engle et al (2015) and Loizzo et al (2017), discovered that more than half of the learners who participated in the MOOC had Bachelor´s or Master´s degrees In this study, however, it was shown that the relationship between final MOOC grades and performance was influenced not only by educational level, but also by primary occupation, and age However, Engle et al (2015) noticed no significant difference with respect to learners’ ages and their achievement levels, it is possible that since so few learners intend to complete MOOCs and there is no penalty for withdrawing, the effects of age are not evident No gender differences were found for course completion or previous xMOOC experience and achieving a final grade between and 10 Notwithstanding, as the xMOOC course content was focused on energy, in Mexican and Latin American contexts, there is a greater proportion of men in that field This research was the first to quantitatively describe and characterize a Mexican sample of participants learning in an energy and sustainability xMOOC Conclusions MOOCs allow people with internet access to learn anything, anywhere, at any time However, generally, MOOC completion rates are low, the reasons for which remain unclear In that sense, researchers are looking for an acceptable and generalizable response to understanding the low completion rates However, MOOC designers worldwide have differing objectives based on the instructional design, the educational objectives, the necessary digital competencies of the participant or the type of audience the MOOC attracts; therefore, measuring all MOOCs through a single indicator is inappropriate Thus, there is a need for metrics that take initial participants’ intentions and educational backgrounds into account (Pursel et al., 2016; Reich & Ho, 2014), as in this study From a sample of 844 participants who had completed one of three energy and sustainability xMOOCs, this study sought to determine the relationship between a Mexican MOOC learner’s profile and their final grade (between and 10); the demographic factors that had a greater effect on the final grade; and whether there was a specific profile for Mexican participants that predicted the highest grades at the end of the online course In conclusion, it was found that the Mexican participants who were most likely to complete an xMOOC and get a final grade (between and 10) were 34 years old or older, 142 J A Canchola González & L D Glasserman-Morales (2020) had a bachelor’s degree or higher, and were in a full-time job These results indicate that learners who are seeking professional development are more likely to complete an online MOOC Gender was not found to be a significant predictor for course completion; however, it was found that even though males made up 70% of the participant sample for the three MOOCs, the females achieved the highest end-of-course grades This investigation, as well as investigations from others (Arghode et al., 2018; Arslan et al., 2015; Engle et al., 2015; Gil-Jaurena et al., 2017; Shalem et al., 2014) have shown that learners’ characteristics or sociodemographic profiles can influence success and achievement in MOOCs Limitations and future work For all the above, it is considered that, in the future, the investigation of MOOCs from the perspective of the participants will bring new evidence confirming the link between the participants' profiles (preferences, intentions, and educational objectives) and whether or not they conclude the course Although further research is needed to confirm these results, it is a first step in the inquiry into Mexican participants studying in a distance education context, specifically the xMOOCs, and it provides evidence explaining the results of completion rates from the individual perspective of the Mexican participant and, by extension, also the Latin American learner As this study only focused on three energy and sustainability MOOCs to confirm the research results, it may be useful to compare the same xMOOC at different times As a limitation, this study did not analyze or evaluate the instructional design of the threeenergy sustainability xMOOCs to see how it also might influence student learning and completion rates Researchers Cirulli et al (2016) affirmed that developing a MOOC platform with a flexible instructional design that allows learners to access, and effectively use, digital content can influence their level of engagement and motivation, and as a consequence, the completion rate of the course Another limitation which needs to be taken into consideration is the course subject, since energy and sustainability topics generally attract men from an engineering background This clearly limits the research to a narrow group which, despite being comprised of people with widely differing sociodemographic characteristics (age, gender, educational level) nevertheless covers a relatively small set of academic profiles Therefore, the findings could not be generalized to an audience with a more heterogeneous profile, attracted by other MOOCs with diverse topics such as culture, music, or videogames It is recommended that complementary questions be added to the initial survey to examine the gender variable in more depth, as, for example, a high proportion of the learners with the lowest grades were male In addition, this research was based on the students who completed xMOOCs; therefore, it is suggested that students who take the courses but not complete them might be included in a future analysis, the relationship between their learner's profile and their final grade be determined, and for there to be a clarification of which aspect of their profile (age, gender, main occupation) influences their decision to drop out Knowledge Management & E-Learning, 12(2), 129–146 143 Acknowledgements This research is a product of the Project 266632 - Laboratorio Binacional para la Gestión Inteligente de la Sustentabilidad Energética y la Formación Tecnológica [Binational Laboratory on Smart Sustainable Energy Management and Technology Training], funded by the CONACYT SENER Fund for Energy Sustainability (Agreement: S0019¬2014¬01) ORCID José Antonio Canchola González https://orcid.org/0000-0002-2964-6097 Leonardo David Glasserman-Morales https://orcid.org/0000-0001-7960-9537 References Akhter, N (2015) Distance education at a glance Journal of Educational Research, 18(2), 1–13 Alraimi, K M., Zo, H., & Ciganek, A P (2015) Understanding the MOOCs continuance: The role of openness and reputation Computers & Education, 80, 28–38 Appleton, J J., Christenson, S L., & Furlong, M J (2008) Student engagement with school: Critical conceptual and methodological issues of the construct Psychology in the Schools, 45(5), 369–386 Arghode, V., Brieger, E., & Wang, J (2018) Engaging instructional design and instructor role in online learning environment European Journal of Training and Development, 42(7/8), 366–380 Arslan, F., Bagchi, K., & Ryu, S (2015) A preliminary evaluation of the determinants of certification success in MOOCs: A multi-level study In Proceedings of the TwentyFirst Americas Conference on Information Systems (AMCIS) Astin, A W (1999) Student involvement: A developmental theory for higher education Journal of College Student Personnel, 40(5), 518–529 Ballard, J., & Butler, P I (2016) Learner enhanced technology: Can activity analytics support understanding engagement a measurable process? Journal of Applied Research in Higher Education, 8(1), 18–43 Callahan, J (2010) The online oxymoron: teaching HRD through an impersonal medium Journal of European Industrial Training, 34(8/9), 869–874 Canchola González, J A., & Glasserman-Morales, L D (2019) Factors that adults attribute for finishing a xMOOC on energy sustainability In Proceedings of the Seventh International Conference on Technological Ecosystems for Enhancing Multiculturality (pp 751–755) ACM Chakraborty, M., & Nafukho, F M (2014) Strengthening student engagement: What students want in online courses? European Journal of Training and Development, 38(9), 782–802 Chernova, Y (2013, July) New study sheds light on free online courses The Wall Street Journal Retrieved from https://blogs.wsj.com/venturecapital/2013/07/31/new-studysheds-light-on-free-online-courses/ Cirulli, F., Elia, G., Lorenzo, G., Margherita, A., & Solazzo, G (2016) The use of 144 J A Canchola González & L D Glasserman-Morales (2020) MOOCs to support personalized learning: An application in the technology entrepreneurship field Knowledge Management & E-Learning, 8(1), 109–123 Daneji, A A., Ayub, A F M., & Khambari, M N M (2019) The effects of perceived usefulness, confirmation and satisfaction on continuance intention in using massive open online course (MOOC) Knowledge Management & E-Learning, 11(2), 201–214 Deng, R., Benckendorff, P., & Gannaway, D (2020) Learner engagement in MOOCs: Scale development and validation British Journal of Educational Technology, 51(1), 245–262 Emanuel, E J (2013) MOOCs taken by educated few Nature, 503(7476), 342–342 Engle, D., Mankoff, C., & Carbrey, J (2015) Coursera's introductory human physiology course: Factors that characterize successful completion of a MOOC International Review of Research in Open and Distance Learning, 16(2), 46–68 Fontana, M V L., & Leffa, V J (2018) MOOCs for language teaching: A study on call from the connectivist perspective Alfa: Revista De Lingüística, 62(1), 75–89 Gil-Jaurena, I., Callejo-Gallego, J., & Agudo, Y (2017) Evaluation of the UNED MOOCs implementation: Demographics, learners' opinions and completion rates International Review of Research in Open and Distance Learning, 18(7), 141–168 Greene, J A., Oswald, C A., & Pomerantz, J (2015) Predictors of retention and achievement in a massive open online course American Educational Research Journal, 52(5), 925–955 Henderikx, M A., Kreijns, K., & Kalz, M (2017) Refining success and dropout in massive open online courses based on the intention–behavior gap Distance Education, 38(3), 353–368 Hew, K F (2016) Promoting engagement in online courses: What strategies can we learn from three highly rated MOOCS British Journal of Educational Technology, 47(2), 320–341 Jordan, K (2014) Initial trends in enrolment and completion of massive open online courses International Review of Research in Open and Distributed Learning, 15(1), 133–160 Jordan, K (2015) Massive open online course completion rates revisited: Assessment, length and attrition International Review of Research in Open and Distributed Learning, 16(3), 341–358 Kilgore, W., Bartoletti, R., & Freih, M A (2015) Design intent and iteration: The #HumanMOOC In Proceedings of the European MOOC Stakeholder Summit (pp 7– 12) Kim, J., Guo, P J., Seaton, D T., Mitros, P., Gajos, K Z., & Miller, R C (2014) Understanding in-video dropouts and interaction peaks in online lecture videos (pp.31–40) In Proceedings of the First ACM Conference on Learning @ Scale Conference (pp 31–40) ACM Kizilcec, R F., Piech, C., & Schneider, E (2013) Deconstructing disengagement: Analyzing learner subpopulations in massive open online courses In Proceedings of the Third International Conference on Learning Analytics and Knowledge (pp 170– 179) ACM Koller, D., Ng, A., Do, C., & Chen, Z (2013) Retention and intention in massive open online courses: In depth Educause Review Retrieved from https://er.educause.edu/articles/2013/6/retention-and-intention-in-massive-openonline-courses-in-depth Kruchinin, S (2019) An investigation into the attraction and completion rates of MOOCs Knowledge Management & E-Learning, 11(1), 38–58 Leslie, H J (2019) Trifecta of student engagement Journal of Research in Innovative Teaching & Learning doi: 10.1108/JRIT-10-2018-0024 Liang, D., Jia, J., Wu, X., Miao, J., & Wang, A (2014) Analysis of learners’ behaviors Knowledge Management & E-Learning, 12(2), 129–146 145 and learning outcomes in a massive open online course Knowledge Management & E-Learning, 6(3), 281–298 Liu, L., He, R., & Cai, M (2018) The exploration and analysis of completion rate and efficiency rate of MOOC based on data analysis – Taking ancient chinese architectural art as an example In Proceedings of the 4th International Symposium on Social Science (ISSS) Liyanagunawardena, T R., Adams, A A., & Williams, S A (2013) MOOCs: A systematic study of the published literature 2008-2012 The International Review of Research in Open and Distributed Learning, 14(3), 202–227 Loizzo, J., Ertmer, P A., Watson, W R., & Watson, S L (2017) Adult MOOC learners as self-directed: Perceptions of motivation, success, and completion Online Learning, 21(2) doi: 10.24059/olj.v21i2.889 Lyu, C X., Chan, R Y Y., & Yeung, R W (2019) Promoting student completion in a MOOC on information theory In Proceedings of the IEEE Frontiers in Education Conference (FIE) IEEE Nava, P V A (2016) Invierten sener y conacyt en laboratorio de sustentabilidad energética Retrieved from http://www.conacytprensa.mx/index.php/tecnologia/energia/6622-laboratorio-deenergia-conacyt-sener Petronzi, D., & Hadi, M (2016) Exploring the factors associated with MOOC engagement, retention and the wider benefits for learners European Journal of Open, Distance and e-Learning, 19(2), 129–146 Pursel, B K., Zhang, L., Jablokow, K W., Choi, G W., & Velegol, D (2016) Understanding MOOC students: Motivations and behaviours indicative of MOOC completion Journal of Computer Assisted Learning, 32(3), 202–217 Rai, L., & Chunrao, D (2016) Influencing factors of success and failure in MOOC and general analysis of learner behavior International Journal of Information and Education Technology, 6(4), 262–268 Rai, L., Yue, Z., Yang, T., Shadiev, R., & Sun, N (2017) General impact of MOOC assessment methods on learner engagement and performance In Proceedings of the 10th International Conference on Ubi-media Computing and Workshops (Ubi-Media) IEEE Reich, J., & Ho, A (2014) The tricky task of figuring out what makes a MOOC successful The Atlantic Retrieved from https://www.theatlantic.com/education/archive/2014/01/the-tricky-task-of-figuringout-what-makes-a-mooc-successful/283274/ Reich, J., & Ruipérez-Valiente, J A (2019) The MOOC pivot Science, 363(6423), 130– 131 Romero-Rodriguez, L M., Ramirez-Montoya, M S., & Gonzalez, J R V (2019) Gamification in MOOCs: Engagement application test in energy sustainability courses IEEE Access, 7, 32093–32101 Shalem, B., Bachrach, Y., Guiver, J., & Bishop, C M (2014) Students, teachers, exams and MOOCs: Predicting and optimizing attainment in web-based education using a probabilistic graphical model Lecture Notes in Computer Science, 8726, 82–97 Springer Verlag Steele, J P., & Fullagar, C J (2009) Facilitators and outcomes of student engagement in a college setting Journal of Psychology: Interdisciplinary and Applied, 143(1), 5–27 Sunar, A S., White, S., Abdullah, N A., & Davis, H C (2017) How learners’ interactions sustain engagement: A MOOC case study IEEE Transactions on Learning Technologies, 10(4), 475–487 Vázquez, J A V., Ramirez-Montoya, M S., & Gónzalez, J R V (2018) Motivation and 146 J A Canchola González & L D Glasserman-Morales (2020) knowledge: Pre-assessment and post-assessment of MOOC participants from an energy and sustainability project The International Review of Research in Open and Distributed Learning, 19(4), 116–132 Wong, J., Khalil, M., Baars, M., de Koning, B B., & Paas, F (2019) Exploring sequences of learner activities in relation to self-regulated learning in a massive open online course Computers & Education, 140: 103595 Xu, B., & Yang, D (2015) Study partners recommendation for xMOOCs learners Computational Intelligence and Neuroscience, 2015: 832093  ...Knowledge Management & E-Learning, 12(2), 129–146 Factors that influence learner engagement and completion rate in an xMOOC on energy and sustainability José Antonio Canchola González* School of Humanities... determine the relationship between learner engagement and completion rates in online distance education, and specifically in xMOOC courses A quantitative approach, using descriptive and inferential... course completion or previous xMOOC experience and achieving a final grade between and 10 Notwithstanding, as the xMOOC course content was focused on energy, in Mexican and Latin American contexts,