Engagement and desertion in MOOCs: Systematic review
Abstract
Massive and open online courses (MOOCs) satisfy learning needs from the particularities of their typologies (xMOOC, tMOOC, cMOOC, iMOOC, among others) even though their high dropout rate is still latent. Recent studies reaffirm engagement as an alternative to reduce dropout rates. The literature analyzed has not yet been able to systematize responses as to how to guarantee engagement in MOOCs and thus reduce their attrition rate. And, consistent with that question, are there still challenges for teachers in this area of educational technology? These answers motivated us to carry out this systematic review to determine how engagement has been studied to help reduce the attrition rate in MOOCs. Articles from journals indexed in Scopus or WoS were reviewed applying the PRISMA protocol. At the end of the protocol, it was defined to analyze 40 studies. The results reflect that the main variables are: the design of e-activities, intrinsic and extrinsic motivation, and communication between students. This paper confirms that the main challenges to guarantee engagement in MOOCs are individualized tutoring, interactivity, and feedback. Due to the scarcity of studies that analyze the variables in an integrated way, it is proposed as future work to determine what relationships exist between these variables that interfere with engagement and dropout in MOOCs.
Keywords
Engagement, MOOC, sMOOC, tMOOC, xMOOC, learning
Palabras clave
Compromiso, MOOC, sMOOC, tMOOC, xMOOC, aprendizaje
Resumen
Los cursos en línea masivos y abiertos (MOOCs) permiten satisfacer necesidades de aprendizaje desde las particularidades de sus tipologías (xMOOC, tMOOC, cMOOC, iMOOC, entre otras), sin embargo, es aún latente su alta tasa de deserción. Estudios recientes reafirman el engagement como una alternativa para disminuir los índices de deserción. La literatura analizada aún no logra sistematizar respuestas a ¿cómo garantizar el engagement en los MOOCs y disminuir así su tasa de deserción? Y, en coherencia con esa pregunta, ¿existen aún retos del profesorado en este ámbito de la tecnología educativa? Ello motivó a realizar esta revisión sistemática para determinar cómo se ha trabajado el engagement para contribuir a disminuir la tasa de deserción en los MOOCs. Se revisaron artículos de revistas indexadas en Scopus o en WoS aplicando el protocolo PRISMA. Al finalizar el protocolo se definió analizar 40 estudios. Los resultados reflejan que las principales variables son: el diseño e-actividades; la motivación intrínseca y extrínseca y; la comunicación entre los estudiantes. Se ratifica que los principales retos para garantizar el engagement en los MOOCs son: la tutoría individualizada; la interactividad; y la retroalimentación. Debido a la escasez de estudios que analicen de forma integrada las variables antes mencionadas, se propone como trabajo futuro, determinar qué relaciones existen entre estas variables que intervienen en el engagement y la deserción en los MOOCs.
Keywords
Engagement, MOOC, sMOOC, tMOOC, xMOOC, learning
Palabras clave
Compromiso, MOOC, sMOOC, tMOOC, xMOOC, aprendizaje
Introduction and state of the art
The study of massive and open online courses (MOOCs), their evolution, design, and assessment has been the subject of analysis since the last century. However, in the last 15 years focus on this topic has increased due, fundamentally, to the rise in educational offers and the increasing demands and learning needs of society (Palacios-Hidalgo et al., 2020). In this context of virtual education, there are different didactic and psycho-pedagogical foundations according to the different types of MOOCs. Several authors (Mellati & Khademi, 2020; Osuna-Acedo et al., 2018; Romero-Frías et al., 2020; Teixeira et al., 2019) classify them as: (1) cMOOC or cMOOCs (Connectivist MOOCs) in which collaborative work and cooperation are promoted through connectivism; (2) xMOOCs or xMOOCs (eXtendedMOOCs) where the interaction is strongly linked to the student-teacher relationship and the assessment process focuses on closed questions; (3) madeMOOCs, encouraging the use of videos, interactivity and co-evaluation; (4) synchMOOCs, establishing time limits; (5) adaptiveMOOCs, developing dynamic assessments using adaptive algorithms and methods; (6) gMOOC, including to a greater extent gamification; (7) sMOOC (Social Massive Open Online Course) which promote greater interaction in learning and the constant accessibility and ubiquity of its educational resources; (8) tMOOC (transferMOOC) contributing to higher levels of learning transfer and pedagogical transformation; and, (9) iMOOC (intelligent MOOC) promoting the personalization of training.
Therefore, nowadays, to carry out a theoretical study of MOOCs would imply either selecting a certain typology or analyzing them "in their unit" from an interdisciplinary perspective. There are different platforms used to design MOOCs, highlighting edX, Udacity, Moodle, and Coursera. Annually, as an initial statistic, more than 40,000 people enroll in different "MOOCs" (Deshpande & Chukhlomin, 2017; Zainuddin et al., 2020), a representation of their importance and relevance even when there is a high dropout rate among those enrolled (Zhu et al., 2020a). There are several explicit causes in the literature, from which a few stand out: motivation, time availability, attitude, interest, tutoring, interactivity and feedback, the accessibility of educational resources, engagement, among other causes (Alturkistani et al., 2020; Fırat et al., 2018; Palacios-Hidalgo et al., 2020).
Previous studies declare that one of the most debated variables in the scientific community is engagement. This term refers to the participation, school commitment, passion, interest in the study, enthusiasm, energy, and dedication that the student demonstrates. This has been the object of pedagogical analysis from the field of learning, academic performance, and the permanence/dropout of a student in a course (Doo et al., 2020; Er et al., 2020; Gallego-Romero et al., 2020). Engagement has its beginnings in the 1980s. However, in the context of MOOCs, it has been fundamentally studied in the last ten years, related to dropout, interactivity, motivation, quality of digital educational resources, e-activities and, virtual tutoring (Deng et al., 2020). It is interesting that in 2020, due to the existing theoretical shortcomings (Deng et al., 2020), an exhaustive analysis of the literature is carried out, scientifically validating a scale to measure engagement in MOOCs, updating the following dimensions: social engagement, emotional engagement, cognitive engagement, and behavioral engagement.
Regarding the trends of MOOCs, various meta-analyses, reviews, and systematic mappings have been published in the last five years, highlighting those of journals indexed in the Web of Science (WoS) or Scopus (Table 1). Table 1 reflects the main topics analyzed in these articles. Of these, only eight articles study engagement and its relationship with MOOCs (Almatrafi & Johri, 2019; Alturkistani et al., 2020; Joksimović et al., 2018a; Khalid et al., 2020; Nortvig et al., 2018; Paton et al., 2018; Wong et al., 2019; Zainuddin et al., 2020; Zhu et al., 2018). Recent studies evidence that the correlation between engagement and MOOCs is not a new trend (Monique & Chiappe, 2020). However, these studies do not systematize and group the variables of engagement and dropout in MOOCs (Galikyan et al., 2021). In this sense, in virtual education, it is essential to study and identify the current challenges of teachers to promote and ensure engagement in MOOCs. We consider that these studies (Table 1) do not answer the following question: how can we ensure or encourage engagement in MOOCs and reduce their dropout rate? And, consistent with that question, are there still challenges for teachers in this area of educational technology and digital teaching? These questions motivated us to carry out this systematic review.
Material and methods
The PRISMA protocol was applied (Urrútia & Bonfill, 2010) and the considerations of how to carry out a systematic review (Moher et al., 2016). This protocol provides a checklist and a four-phase process that guides the proper design of systematic reviews.
Step 1: Purpose of the study. The objective is to carry out a systematic review to analyze how to reduce the attrition rate in MOOCs from engagement. The scientific questions developed to fulfill the aim of the research were:
(1) What are the platforms and study modalities most used in research studying engagement in MOOCs?
(2) What are the most studied variables in engagement to reduce the dropout rate in MOOCs?
(3) What are the main challenges associated with engagement in MOOCs?
Threat validity criteria
Step 2: Review protocol.
-
Internal validity. Each study was analyzed using a protocol that involved: (1) keywords, (2) description, (3) type of research, (4) research design, (5) analysis of results, and (6) argumentation of the conclusions.
-
External validity. Articles (case studies or experimental studies) that do not validate their results are highlighted.
-
Conclusion validity. A form was applied using the keywording technique (Petersen et al., 2008), the assessment criteria for systematic reviews proposed by the Joanna Briggs Institute (Lockwood et al., 2015), and the guidelines for quality, transparency, and replicability (Díaz-Iso et al., 2020).
Selection process and inclusion and exclusion criteria
-
Selection and classification process. The keywording technique (Petersen et al., 2008) allowed the researchers to classify the variables and the psycho-pedagogical foundations of constructivism framed the analysis of these studies. Mendeley was used to identify duplicate papers. In any discrepancies between the authors, we analyzed the opinions of three guest researchers.
-
Inclusion criteria. (1) Papers published between 2017 to February 2021; (2) articles in journals indexed in Scopus or WoS; (3) case studies or experimental studies; (4) research that studies engagement and its relationship with dropping out of MOOCs; (5) articles written in English or Spanish and peer-reviewed.
-
Exclusion criteria. Level of the description of the research, type of research (essays, tutorials, meta-analyses, reviews, and systematic mappings), relationship with the object of study (engagement and desertion in MOOCs), and publication period.
Search strategy
-
The scientific literature search was carried out in Scopus and WoS.
-
Combinations between the logical AND / OR operators were used. The keywords were: engagement, MOOC, MOOCs, xMOOCs, cMOOC, iMOOC, sMOOC, tMOOC, experimental studies, case studies, pre-experiment, quasi-experiment, empirical experiences, and studies.
-
Several terms associated to MOOCs (MOOC, MOOCs; xMOOCs; iMOOC; tMOOC and sMOOC) were used and similar terminologies were examined (cMOOC or cMOOCs; xMOOC or xMOOCs).
-
General search strings -in Spanish and English-: KEY ((MOOCs OR xMOOC OR MOOC OR iMOOC OR sMOOC OR tMOOC OR cMOOC) AND (engagement) AND (experimental studies OR pre-experiment OR case studies OR quasi-experiment OR study) OR TITLE (MOOCs OR xMOOC OR MOOC OR iMOOC OR sMOOC OR tMOOC OR cMOOC) AND (engagement) AND (experimental studies OR pre-experiment OR case studies OR quasi-experiment OR study). In the case of WoS, only the following indices were searched: Social Sciences Citation Index (SSCI) and Science Citation Index Expanded (SCIE).
Quality criteria
To reduce research biases, all articles were assessed on a scale from 1 to 5. The value 5 is the maximum score conferred on the basis of each researcher’s criteria. Among the criteria used, the following questions stand out: Are the instruments and the research process described? Are the results argued? Is there coherence between the type of study and the methodology used?
Step 3: Data extraction. Through in-depth analysis of evidence content, their information and relevant knowledge were stored in a data matrix to analyze, synthesize and group the information (Díaz-Iso et al., 2020; Lockwood et al., 2015; Petersen et al., 2008). In the stored information, the following factors stand out: the authors, publication date, study variables, type of research, and education level.
Step 4: Data analysis. The process included the grouping of variables, trend analysis, and statistics. Cohen's Kappa coefficient (k=0.826) was applied, obtaining 96% of «agreements» achieving a match in the researchers (Tang et al., 2015).
Analysis and results
Overview of the systematic review
Of the 40 selected studies (Figure 1), 77.5% (Figure 2) are from the last three years, highlighting the researches of case studies with pre and post-test (65%) and experimental studies (27.5%).
RQ1: What are the platforms and study modalities most used in research studying engagement in MOOCs?
Educational experiences are mainly focused (Figure 3) on edX platforms (n=14), Coursera (n=8), FutureLearn (n=4), and Moodle (n=4).
E-learning is the most used modality with an emphasis on online learning (n=34). Therefore, in this modality, a greater diversity of learning management platforms is used.
Some papers (b-learning and e-learning modalities) use MOOCs but do not classify them. In the case of e-learning, the xMOOC and sMOOC are mainly used. The least used typologies are cMOOCs and iMOOCs. The research relationship, study modality, and platforms are presented in Table 2.
RQ2: What are the most studied variables related to engagement in order to reduce the dropout rate in MOOCs?
Three scenarios stand out in the research (Figure 4):
-
General education –students of different ages. The most studied variables are the following: data privacy, forum design, education democratization, gamification, satisfaction, and perceived quality.
-
University education. In this scenario, the following variables stand out: design of electronic learning activities (e-activities), intrinsic and extrinsic motivation, personal learning networks, and peer review.
-
Postgraduate education. The following variables stand out: communication and social media, design of e-activities, motivation, and intrinsic communication.
Table 2 exposes the research-variables relationship. When grouping and analyzing the studies, it is highlighted that the variables most used from engagement to reduce the dropout rate in MOOCs are: e-activity design, intrinsic and extrinsic motivation, and communication between students.
RQ3: What are the main challenges associated with engagement in MOOCs? The analysis of each paper allowed to identify the following challenges (pair research-challenges) (Table 2).
-
I1: Validity of the educational offer; mentoring according to individualization and diversity.
-
I2: Makeup of the learning community and interaction.
-
I3: Tutoring according to individualization and diversity.
-
I4: Money, infrastructure, and internet access.
-
I5: Makeup of the learning community and interaction.
-
I6: Tutoring; previous preparation of how to use the MOOCs.
-
I7: Interactivity and feedback
-
I8: Quality of digital educational resources.
-
I9: Interactivity and feedback.
-
I10: Tutoring according to individualization and diversity.
-
I11: Generation of collaborative activities from group work.
-
I12: Quality of digital educational resources.
-
I13: Accessibility and reusability of content.
-
I14: User interface and interactivity.
-
I15: Interactivity.
-
I16: Interactivity and feedback.
-
I17: Interactivity and collaboration.
-
I18: Data privacy.
-
I19: Interactivity and collaboration.
-
I20: Tutoring according to individualization and diversity.
-
I21: Educational information policy.
-
I22: Quality of digital educational resources.
-
I23: Diversity of activities in various MOOCs.
-
I24: Personalization of the training itinerary.
-
I25: Technological literacy, control of learning materials, availability of teaching materials, and assessment criteria.
-
I26: Tutoring according to individualization and diversity.
-
I27: Tutoring and little familiarity that some teachers have with technology.
-
I28: Quality of digital educational resources.
-
I29: Interactivity and feedback.
-
I30: Teaching methods in MOOCs.
-
I31: Quality of digital educational resources.
-
I32: Curiosity.
-
I33: Interactivity and collaboration.
-
I34: Interactivity and collaboration.
-
I35: Activity-based learning designs.
-
I36: Autonomy.
-
I37: Interactivity and collaboration.
-
I38: Quality of digital educational resources.
-
I39: Tutoring according to individualization and diversity.
-
I40: Interactivity and feedback.
In summary, when analyzing the papers and grouping them by year, the most recurrent challenges are:
-
2017: Individualized tutoring and the training of learning communities.
-
2018: The development of learning communities and interactivity.
-
2019: Interactivity, feedback, accessibility and user interface, and individualized tutoring.
-
Articles published from 2020 to February 2021: interactivity, feedback, quality of digital educational resources, and individualized tutoring.
In essence, the analysis, selection, and grouping of the variables made it possible to determine that the main challenges, among others particular to each study, are: (1) individualized tutoring; (2) interactivity; and (3) feedback.
Discussion and conclusions
Desertion or abandonment in MOOCs concerns the community of teachers because there are differences between initial enrollment and the number of students who complete the courses. Its causes are diverse, highlighting interest, previous academic experience, repeated absences, systematic self-learning, tutoring, interactivity and feedback, accessibility of educational resources, and engagement (Martinez-Navarro, 2021; Alturkistani et al., 2020; Fırat et al., 2018; Palacios-Hidalgo et al., 2020). For this reason, the scientific community studies for alternatives to reduce the dropout rate in MOOCs. In this sense, engagement is one of the theoretical foundations applied to achieve this goal. That is why this systematic review identifies those engagement variables' that reduce the dropout rate in MOOCs.
Regarding the first question (RQ1), MOOCs are frequently designed using the EdX, Coursera, FutureLearn, and Moodle platforms. We all need constant professional improvement and therefore, the open alternative of MOOCs is reaffirmed as the main route of virtual education. The analyzed articles focus on university and postgraduate training due to the age of the participants, their employment situation, and economic expectations.
The above justifies that e-learning is the most widely used modality due to its potentialities related to online education. In this modality, the MOOCs developed in university education were fundamentally designed in the typology cMOOCs and "MOOCs" –the authors declare this, without arguments– while, in postgraduate education, they were referred to as xMOOCs. Although innovative experiences are expressed through the sMOOC and the tMOOC –as the latest trends in MOOCs, there is still a lack of studies to ratify the achievement of collaborative work, the transfer of learning, pedagogical transformation and, the “generating interest towards the professional action and interaction” (Osuna-Acedo et al., 2018). Regarding the second question (RQ2), several variables assure or provide engagement in MOOCs. In the analysis of the results, the following variables stand out:
-
In the design of e-activities in MOOCs, the following are recurrent: (1) promoting the cognitive freedom of the student and their involvement in the learning activity; (2) autonomy; (3) foster collaborative learning and interaction between the student and a digital educational resources system and; (4) the orientation and development of skills that allow students to search, interact, analyze, select and manipulate the information present in the learning environment (Cabero-Almenara & Palacios-Rodríguez, 2021; Gros Salvat, 2018). At the same time, the assessment of the e-activities requires the interweaving between the "appropriate" learning rubrics, self-management of learning, learning strategies, tutoring, and personal learning environments. Therefore, the assessment is according to the typologies of the e-activities: 1) analysis and synthesis, 2) problem solving, 3) interaction and communication, 4) collaborative knowledge construction, and 5) reflection activities (Maina, 2020).
-
The relationship between intrinsic and extrinsic motivation and demotivation is known. This study does not analyze these theories from conductive, cognitivist, or constructivist psychology. However, in the analyzed studies (Table 2), it is declared that these variables are essential to promote or assure engagement in MOOCs. The scientific literature reinforces the hypothesis that in the development of the psyche, motivation (intrinsic and extrinsic) and demotivation are complex and dynamic processes conditioned by internal and external situations, thus reinforcing their biological, psychological, and social character. Therefore, that extrinsic educational rewards (congratulations, accreditations, certificates, among others) and intrinsic rewards (self-esteem, among others) contribute to learning, academic performance, and the student's permanence in the course, which contrasts with what has been declared by Acosta et al. (2014).
-
The third most used variable is “communication between students”, highlighting interaction and interactivity. Most of the studies focus on interactivity in MOOCs. However, educational communication transcends these limits, as it includes the pedagogical labor of the teacher, the style and form of educational communication, and individualized and group educational communication. Therefore, the design, development, and assessment of e-activities should develop social interaction, collaboration, and social inclusion in an environment of educational communication, constant feedback, and pedagogical direction. The aspects described above evidence a possible relationship between these three variables. However, we consider that the literature lacks studies that demonstrate the relationship between these variables through experiments; and how, in its unit, it affects engagement in MOOCs.
Finally, consistent with the results obtained in the two previous questions, the main challenges (RQ3) to guarantee engagement in MOOCs are grouped into the following aspects:
-
First challenge: individualized tutoring. It is known that this educational activity is carried out personally and directly. However, e-learning and b-learning tutoring have become more complex due to the diversity of interaction scenarios. In effect, the introduction of MOOCs increased the complexity of the educational process with the massiveness of tuition. Therefore, trends have emerged to promote new tutorials even when teachers are not always prepared. In this sense, the peer support process is characteristic of xMOOCs and the cMOOCs, tutoring is transformed and supported by the relationships, nodes, and interactions present in the virtual environment, whether cognitive, didactic, or social. This development of cMOOCs evolved until the creation of recommender systems, but there is a lack of emerging pedagogies for their use.
In recent years, tMOOCs have accentuated two actors in the pedagogical process: the "tutor" and the instructor or teacher (s) of the course. From a humanistic and educational perspective, personalization of learning, concern for student performance and motivation, learning outcomes, and course engagement are aspects that involve these two actors. For this reason, this unresolved and poorly approached challenge is sometimes a product of the commercialization of education and the individualization of the teaching staff which affects student engagement (Maré & Mutezo, 2020).
-
Second challenge: interactivity. This “well-known” aspect is vital in e-learning and b-learning modalities. However, its presence in current challenges to cause engagement in MOOCs is reiterated. Therefore, if there are already several studies (theoretical and empirical) to promote interactivity, why is it a current challenge? This systematic review confirms an increase in the learning demands of the student (person of any age, mainly adults), implying the need for new MOOCs courses. The instructional design of some courses lacks fundamentals because teachers often lack pedagogical, didactic, and communication skills.
There are various e-activities carried out in MOOCs, with forums being one of the most widely used. The literature reiterates the need to train teachers and tutors in how to assure or promote engagement in MOOCs and achieve interaction in discussion forums, collaborative learning, MOOC teaching methods, and MOOC assessment methods (Wu, 2021). This challenge is summarized in that the teacher, tutor, and instructor must "know" and "know how to do" the interaction in the discussion forums and integrate them with the online reviews of the MOOCs.
-
Third challenge. Feedback. Feedback is conceived from three perspectives: (1) centered on the teacher, (2) dialogue centered on the process, and (3) sustainable action (Quezada-Cáceres & Salinas-Tapia, 2021). However, the massification of MOOCs does not allow the correct educational orientation and individual monitoring of the student (Gordillo et al., 2019). In this sense, it is a current need for the teacher to design and produce digital educational resources with a high level of accessibility, ensuring that they adapt to the student and provide feedback according to the learning and performance needs. This challenge requires that feedback transcends the communicative limits of the teacher-student, including educational tools and resources designed to provide feedback to the student. This challenge, therefore, circumscribes teacher training in the use of the author's tools. In summary, there are coincidences and concerns about implementing feedback in MOOCs, involving the pedagogical actions of the teacher, the design and production of digital educational resources, and the training of teachers. The teacher must use, interpret, and analyze the tools and functionalities of the learning platforms to determine the current and prospective state of student learning. It is concluded that the design of MOOCs solves various learning needs, however, although its effectiveness and relevance is undoubted, the high dropout rate is its main Achilles heal. Along these lines, various studies have explored how to mitigate this limitation, highlighting the line of engagement.
In the last decade, studies of engagement and dropout in MOOCs identify their main variables (design of e-activities; intrinsic and extrinsic motivation; and communication between students) but, it is still a pending challenge. This systematic review identifies, ratifies, and groups the main challenges to ensure engagement in MOOCs. These challenges are individualized tutoring, interactivity, and feedback (Almatrafi & Johri, 2019; Nortvig et al., 2018).
Extension limitations make it impossible to delve into the results and their discussion. We consider that this study has several shortcomings. First, only papers indexed in Scopus and WoS written in English or Spanish were analyzed, therefore, other studies that may diversify the results obtained were omitted. Second, the alternative solutions to the challenges present in the literature are not determined. Therefore, this weakness encourages carrying out theoretical and empirical research to solve these challenges. Finally, it would be important to refine the search criteria in terms such as madeMOOCs, synchMOOCs, adaptiveMOOCs and gMOOC as they were not intended in our search strings. Conclusively, it would be interesting to determine what relationship or relationships exist between the variables linked with engagement (e-activity design, intrinsic and extrinsic motivation, and communication between students) and the dropout rate in MOOCs.
Author Contribution
Idea, O.E., D.R.F.C.; Literature review (state of the art), O.E., D.R.F.C.; Methodology, O.E., D.R.F.C.; Data analysis, O.E., D.R.F.C.; Results, O.E; Discussion and conclusions, O.E., D.R.F.C.; Writing (original draft), O.E; Final revisions, O.E., D.R.F.C.; Project design and sponsorship, O.E. (1)