Introduction

The paths used by researchers to approach ‘reality’ are the methods. They set the relationships, strategies and techniques that will be used, and which will be established through a particular methods design. In this sense, an approach to ‘reality’ in social environments can be provided through mixed methods. These last are usually defined as the combination of multiple methodological strategies to study and answer questions on a particular topic. Among these definitions of mixed methods, we find that of Plano-Clark & Ivankova (2016: 57) who have conceptualized it as “the intentional integration of quantitative and qualitative research approaches to better address a research problem”. Meanwhile, others have defined it as the ability to conduct balanced study analyses that increase the validity of a rationale (Edmonds & Kennedy, 2017) and its scope (Onwuegbuzie & Teddlie, 2003). Similarly, various authors have argued about philosophical assumptions that guide data collection and analysis, mixing quantitative data and general qualitative approaches incorporated into the various phases of the research process (Creswell, 2007; Yu, 2009; Tashakkori and Teddlie, 2010).

More recently, Valenzuela-Gonzalez (2019) has indicated that the merger of both data forms in the same research design or method is something new as the idea of mixing data, specific research designs, the note-taking process, terminology, procedures, and difficulties in using different designs are recent features that emerged at the dawn of the 21st century. This, as previous definitions, indicates that mixed-methods studies are much more than the sum of quantitative with qualitative data as they encompass a strategic combination, triangulation and integration of both types of data that is based on the particular research design.

Creswell (2003) classifies research designs into six types: sequential explanatory design, sequential exploratory design, sequential transformative design, concurrent triangulation design, concurrent nested design, and concurrent transformative design. Another classification by Johnson and Onwuegbuzie (2004) suggests there are nine designs represented in a four-quadrant matrix where the researcher must decide between the paradigm (dominant or not) and the time to carry out the study (concurrent or sequential). For their part, Teddlie and Tashakkori (2006), analyzed the usefulness of various design typologies, as well as the dimensions used by the authors and proposed a Method-Strands Matrix, which presents research designs, especially four families: sequential, concurrent, conversion, and fully integrated. Based on these classifications, other authors have contributed in terms of the possible questions and analyses (Onwuegbuzie & Leech, 2006). As part of this classification, Harwell (2014) links these designs to research questions that they provide in the process of carrying out studies, including examples for each design. Moreover, DeCuir-Gunby and Schutz (2017) propose five basic designs: explanatory sequential design, exploratory sequential design, convergent parallel design, embedded design and multiphase design. Having said that, it is important to consider that mixed methods also have their challenges for researchers, mainly in relation to the incompatibility of qualitative and quantitative approaches (Creamer, 2018). Each of these designs has its own benefits and its own difficulties and challenges, so the choice depends on the research questions and the purpose of the research study.

In the field of social and educational research, the digital imprint and technological developments have provided opportunities to design studies with mixed methods that have made important contributions to innovation in the sector. In a systematic literature mapping, González-Pérez et al. (2019) located the emerging themes of educational technology: digital education, technological models, adaptive technologies, open technologies, smart technologies and disruptive technologies. These themes have provided opportunities for innovations in different fields. Rogers (2003: 11) has defined innovation as “an idea, practice or project that is perceived as new by an individual or other adoption unit", where collaboration is substantial (Corbo et al., 2016); it has also been defined as the process of coming up with new products that can be adopted or redesigned for use and transformation (Rikkerink et al., 2016), and even in an open way (Ramírez-Montoya, 2018). Innovation, accordingly, can promote a new process (organization, method, strategy, development, procedure, training, technique), a new product (technology, article, instrument, material, device, application, manufacture, result, object, prototype), a new service (attention, provision, assistance, action, function, dependence, benefit) or new knowledge (transformation, impact, evolution, cognition, dissent, knowledge, talent, patent, model, system).

From this we can derive that new processes, products, services and knowledge are the engines of change in the field of education, where innovation often contributes to address problems and situations arising from teaching practices and delivery. In this respect, Sein-Echaluce et al., (2019) argued that educational innovation means making changes in learning/training in order to improve learning outcomes. In order to achieve this, educational innovation must be embraced holistically and inclusively. Consequently, companies, students, educational providers, communities and political organizations need to integrate the objects of innovation at all levels (Baumann et al., 2016).

Equally, it is important to understand how these research foci are classified. In this sense, some scholars at the Educational Innovation Research Group in Mexico (Ramírez-Montoya & Valenzuela-González, 2019) have proposed a comprehensive classification. It is one that includes: psycho-pedagogical (related to general learning and teaching); use and development of technology in education (application and impact of technology in education, both face-to-face and at a distance); educational management (administration processes: planning, organization, management, and evaluation of human, material, and financial resources of educational institutions); and, socio-cultural (emphasis on the sociocultural context in which the educational process takes place). In addition, these scholars also highlight a ‘disciplinary’ category, which relates to the curriculum and the teaching-learning process in disciplines such as mathematics, medicine, natural sciences, engineering, accounting, business, and English, among others (Rodríguez et al., 2015). Moreover, these scholars go on to say it is important to ask what types of contributions can be made with studies that use mixed methods within the particular research spectrum.

Among scholarly works that have undertaken systematic literature reviews, meta-analyses and/or studies based on mixed methods, we find contributions to a variety of disciplines. Among them are those related to computers and the use of software in small and medium enterprises (Sharma & Sangal, 2018). We should also mention those in the field of health and gender (Mabweazara et al., 2019) as well as those that deal with autism spectrum disorder (Frantzen & Fetters, 2016) and the safety of elderly drivers (Classen & Lopez 2006). Scholars such as Pluye and Hong (2014), on the other hand, have contributed with reviews providing guidelines for planning, conducting, and evaluating mixed-method research. Specifically, in the field of education, the work of Imanuel-Noy and Wagner (2016; 2014) who have looked at teacher training in the clinical area, is worth mentioning. In turn, Levin and Wagner (2009) produced a theoretical and practical vision of education, which argued about its importance for the advancement of knowledge and public policy. Overall, these contributions provide a valuable basis from which to argue that mixed methods have in fact contributed to advance our knowledge in these areas.

However, important questions still need to be addressed and in some cases remain unanswered. Particularly, around challenges and opportunities that arise in the combination of the digital era and education. Indeed, in the existing literature there are still important gaps in knowledge in relation to the contributions of mixed methods, specifically in terms of their potential for educational innovation. One particular example of this is the gap in areas such as the representation of culturally and linguistically diverse students (Klingner & Boardman, 2011). In this sense, this article aims to analyze recent studies (2010-20) that have integrated digital and technological components in social and educational research, where mixed-method designs were applied. Therefore, the aim would be to understand the characteristics of the studies, the research topics that are often undertaken, the types of research designs found, and the nature of the innovation contributions made in the field. It is necessary to fill this gap in order to dissect the original knowledge that can in turn allow us to develop a theoretical framework to provide further guidance for trainers, researchers and decision-makers with a vision of improvement and change in education in general.

Method

Our central approach involved a systematic literature review (SLR) due to its ability to help identify, assess and interpret available research related to a topic area (Kitchenham and Charters, 2007). The process is broadly based on guidelines established by Verner et al. (2012) and the University of York (2009). In order to analyze the articles in a fair, rigorous and transparent manner, we established an analysis protocol with the following phases: 1) Research questions; 2) Search process; 3) Inclusion and exclusion criteria; 4) Data selection and extraction process; 5) Data synthesis.

• Phase 1: Research questions: Based on the aim of analyzing the characteristics of the social and educational studies that have been published in the last ten years, nine research questions were posed to locate the characteristics of the articles, the findings and digital technologies, the types of designs and lines of educational innovation, where they have contributed.

The origin of the nine research questions that drove the study was in the identification of gaps previously observed in studies conducted by the authors of this paper, as well as the challenges (and benefits) they have found in the implementation of the methodology, both in the implementation of other studies, as well as in training activities with their students and research groups. The possible answers came from the theoretical support on which the study was based. The motivation for the topics and research questions were based on the opportunity to contribute new ways of studying educational innovation.

• Phase 2: Search process. The protocol for the search of articles integrated electronic processes in the Scopus and Web of Science (WoS) databases, delimiting the keywords (mixed methods, education, social, digital), language (Spanish and English), time window (2010-2020), type of document (article), type of access (open) and language (English and Spanish). The search strings are presented in the integrated Excel (

• Phase 3: Inclusion and exclusion criteria. Articles were included if: they were articles that integrated in their title, abstract or keywords the topics of mixed methods, social or educational and digital or tech*, published between January 2010 and January 2020, written in English or Spanish and were open access.

Articles were excluded if: they were papers, book chapters, literature reviews related to the topics of the search (mixed methods, social or educational and digital or tech*), published in languages other than Spanish and English, that were not open access and that were published before January 2010 or after January 2020.

• Phase 4: Selection process and data collection. The search resulted in the identification of 190 articles in Scopus and 184 in WoS. Articles were reviewed in both databases with the aim of identifying and eliminating duplicates. The articles were reviewed to verify that they contained the integration of mixed methods in the studies and that they were related to the social and educational areas. Finally, 311 articles were selected and can be consulted in the integrated Excel.

The following data had been previously extracted from the studies identified: Authors, Title, DOI, Abstract, Country and Keywords. The researchers used a data extraction strategy specifically related to the nine questions under study in the SLR. Validation was done through peer review to check the identification of the answers; in cases of discrepancies, agreements were reached for the selection of answers and to approach 100% data verification.

• Phase 5: Data synthesis. In order to synthesize the answers, a previous classification of the possible graphic representations was made, an analysis was carried out to locate the intersection that could be interesting to relate terms, keywords, networks of co-terms, clusters and concatenation of categories and subcategories.

Results

In this section, which is organized by the themes and sub-themes identified in the research, we report the results related to the research questions. The graphing tools were Vosviewer and Tableau.

Characteristics of social and educational studies that have integrated mixed methods in recent years

• RQ1: What are the key words in the studies, how are they related and what groups of incidents are detected?

The keywords of the 311 articles were located (Figure 1a) and clusters of higher prevalence were identified in the words “Education” (Figure 1b), “Qualitative” (Figure 1c) and “Quantitative” (Figure 1d). Clusters establish networks of co-terms that are at the same level (keywords) and their most frequent relationships with other terms. The representation is shown in the following figures (Figure 1).

The keyword co-terms (Figure 1a) highlight four major groupings (colors) where the most related terms are highlighted: human, education, curriculum and teaching, followed by assessment, educational technology, students, higher education, and medical education. Based on the objective of this article, researchers conducted a deeper exploration of clusters of interest and their relationship with human, psychological, woman, man, learning and teaching located in Education (Figure 1b). The clusters of the words Qualitative (Figure 1c) and Quantitative (Figure 1d) relate with the terms under study: education, human, adult, learning and curriculum.

These results shed light on the key terms that have been recurrent in the articles and may be useful for theoretical frameworks of studies related to educational innovation, mixed methods applied to education, educational assessment and for support of training programs, with interest in educational innovation.

• RQ2: In which years have the articles been published and in which journal quartile levels are they located?

Quartile data from the journals were sought to classify them according to level (Q1, Q2, Q3, Q4). There were journals with ESCI and ERIH indexes and others that were classified as No data (from the WoS database) and No rank (from the Scopus database) because they are journals newly entered in these databases and do not yet reflect the level (Figure 2). The growth of publications in Q2 journals in the last few years, and the decrease of Q1 journals in this period are noteworthy.

Figure 2 is relevant to academic communities of educational institutions (with an interest in rankings), research groups (with an interest in consolidation), training programs (undergraduate and graduate), researchers (for their careers and publication visibility) and journal editors (thematic vs. impact factor), by enabling the recognition of the growth of publications that have used mixed methods in their research, as well as the range of impact factor, according to the type of journal.

• RQ3: Which journals have published the most on the subject and how many citations have their articles received?

The citations received for the articles were identified and related to the journals that have the most publications on the subject (Figure 3). The journal that stands out is BMC Medical Education with articles that have accrued 263 citations.

The contribution that this Figure gives is the location of two themes of great interest for communities attracted to educational innovation, both in the social sectors, as well as in government, academia and business, where training and innovation are important, by locating the journals that have published the most research articles with mixed methods and the number of citations they have had. The visibility of articles is an important consideration for the social appropriation of knowledge.

• RQ4: What is the geographical distribution of the authors? In order to locate where the authors who have worked most with mixed methods in recent years are, the country of the first author was identified, and the number is represented in Figure 4.

Networking in educational innovation is a strategic engine for growth at all levels: institutional, national, regional and international. Locating authors who have worked on a topic of interest, in this case, locating authors who have used mixed methods in their studies, can support the strategic relationships of students, teachers, researchers and trainers. Some possibilities include carrying out collaborative academic activities, research, publication or academic internships. Also, Figure 4 helps us to identify the regions with authors working on the topic of mixed methods.

• RQ5: What are the emerging digital technologies that have been studied with mixed methods? / RQ6- What kind of innovation does the article bring? It was relevant to correlate the types of emerging technologies identified in the articles, following the classification of González-Pérez et al. (2019) and the possibilities of contributions to educational innovation. It was done so that two same-level dimensions were analyzed for each article: the digital technologies that were identified in the articles and the type of innovation located in the publications. Figure 5 shows this crossing, highlighting a homogeneous frequency among the different types of innovation.

The result located in this figure brings threefold light: a) In the dimensions of the upper axis it is possible to identify which the emerging digital technologies that are being worked on are, such as digital education, and which are less , such as disruptive technologies; b) In the dimensions of the lower axis are located the new contributions given by the studies in terms of knowledge, processes, products or services and; c) At the intersection of both axes and the frequency identified in the articles, one can see the picture of how emerging technologies have supported innovation in research practices, leaving challenges, such as driving new products (not identified in four technology categories) and fostering disruptive innovations (which had less frequency in the analysis of the articles). These results provide development opportunities for creative and innovative program teams, where innovators, decision makers and the educational community can locate advances from research and visualize challenges to bring new knowledge.

• RQ7: What kind of design did they use? / RQ8- What sample have you worked on in the study? The studies were analyzed to locate the mixed-method research design and the sample being conducted in the studies (see Figure 6 at https://doi.org/10.6084/m9.figshare.12269414.v1). We highlighted the exploratory sequential designs with a large sample (over 200 participants).

The crossing of the design and sample dimensions is a data that we consider interesting to identify how the different types of research design have behaved (following the classification of Creswell, 2003), with respect to the number of participants that have been present in the studies. This may shed light for students, researchers and research groups interested in contributing using mixed methods.

• RQ8: What kind of design did they use? / RQ9: In what lines of research and topics have the studies been carried out with mixed methods? The research lines of educational innovation of the articles were analyzed, with emphasis on the research question of the study, given that ‘digital’ was a key word in the publications. The research designs used in those lines were identified, as well as the most studied topics in the lines (Figure 7 at https://doi.org/10.6084/m9.figshare.12269477.v1).

Once the articles were identified by their type of design, it was considered interesting to cross-refer them with the educational innovation lines of research (Ramírez-Montoya & Valenzuela-González, 2019; Rodríguez et al., 2015). In this way, it is possible to see the areas that are being worked on in the articles and the type of design. In addition, the topics that are being worked on are located within these lines. For example, in the psycho-pedagogical line, articles that deal with evaluation stand out, or in the disciplinary line, medical education studies are of great frequency. This result can be of interest to the areas of planning, instructional design, evaluation and educational research, among others, where decision makers, stakeholders, academic groups and training agents can have a reference to develop new forms of creation and research.

Discussion and conclusions

The publications related to research that integrates digital technologies, gives an opportunity to carry out analysis from a perspective that contributes to educational innovation. The objective of this article focused on providing original contributions to research the characteristics of these studies and the trends of new contributions to education. Our SLR highlighted important research foci on educational innovation. Particularly, around findings that were implied by the analysis and that led to the deduction of networks of co-terms, identification of educational innovations and types of designs that are being worked by scholars. It is worth highlighting some, including:

• The articles in the areas of social sciences and education, with components of digital integration, which are studied with mixed methods. They stand out for their emerging growth in recent years and for focusing on key aspects of educational sciences (the human factor and the educational process) and on processes that combine quantitative and qualitative strategies. Figure 1 shows the network of key words in the studies, the relations established between the most outstanding terms (human, education, curriculum and teaching) and their relationship with others of the same level; likewise, the clusters of qualitative and quantitative processes to study educational phenomena are highlighted. Figures 2, 3 and 4 report the growth in the number of articles, citations of these studies and geographical location where they are conducted. These findings are in line with what some methodologists indicate regarding conducting mixed methods, where collection and analysis must be balanced in the phases of the study process (Creswell, 2007; Tashakkori & Teddlie, 2010). The networks of co-terms and the characteristics of these studies analyzed in recent years can help to identify conceptual frameworks to support academic communities and stakeholders in project approaches, training and evaluation.

• Another theme is that of digital mediation in social science and education studies, which brings with it the integration of emerging digital technologies and contributions to new processes, products, services and knowledge. This finding is reflected in Figure 5 where the crossover of technologies and contributions is identified, highlighting that there is a greater incidence of technologies classified as digital education and the need to scale up new products and disruptive technologies. These integrations in training processes are linked to the basic ideas of Rogers (2003), who defines innovation as an idea, practice or project that is perceived as new, either individually or in adoption processes. Identifying digital technologies and new contributions helps in addition to establish a link with educational innovation, areas that can be of value to stakeholders, decision makers, scholars and creative teams interested in generating new options for education.

• We could also observe that the mixed methods are implemented with differentiated samples and designs that contribute to the research lines of educational innovation. The sample sizes in the different designs are reflected in Figure 6 and also in Figure 7. It shows a cross-section of the designs and the lines of research. The data is classified within the taxonomy provided by Ramirez-Montoya & Valenzuela-Gonzalez (2019), and Rodríguez et al. (2015) where the psycho-pedagogical, use and development of technology in education, educational management, socio-cultural and disciplinary lines are located. Educational innovation has -together with the different types of designs- an opportunity to generate new knowledge for processes such as design, evaluation, training and research, where academia, business, government and society in general have the opportunity to undertake and innovate.

One thing that became clear from the analysis of the data is that the growth of educational research must include the recognition that working with mixed methods implies designs that converge. This, in order to enrich the approach that scholars undertake when assessing facts on the ground. Therefore, it is not only the sum of collecting quantitative and qualitative data but the intersection in the different levels of the research process (research question, selection of techniques and strategies, data collection, analysis, interpretation, legitimation and report presentation) that marks, in fact, the difference between being innovative or not.

One of the most important aspects that we can highlight is that by embracing a mixed-methods approach in education research it has been possible for scholars to not only close important gaps in our understanding of the field, but also to enhance interdisciplinarity. Indeed, the mixed-methods approach not only provided a more holistic grounding to knowledge, but also changed the mindset about how researchers approach the issues in question. This review of existing literature also provides additional guidance towards an inquiry that, due to its nature, is always organic and on the move. That is, the changing nature of education itself requires a combination of research strategies that brings about a better and more comprehensive understanding of the subject in question.

Furthermore, our SLR highlights how the technological imprint and digitalization of processes has brought about changes in educational processes and in the ways and possibilities of doing research. Indeed, as Klingner and Boardman (2011) point out, the mixed methods research can lead to insights about possible challenges to implementation as well as the circumstances under which a practice is most likely to be successful; therefore, adding depth and breadth not available through quantitative designs alone. Particularly, because this approach is better suited to address the enormous complexity of the education phenomena, which tends to be overall heterogeneous and particular-specific to each case and discipline.

Based on the work found that has used mixed methods, one could ask how mixed methodologies help to carry out better research in the field analyzed here, at least compared to studies that use a single method. In other words, it is important to ask: what is the added value of the mixed methodology to advance and improve research in this field? The contribution lies in the possibility of approaching the knowledge of complex entities, such as studies in social sciences, education, communication, with views of depth and scope. In this sense, the different designs around mixed methods help also to link quantitative and qualitative data, providing meaning and sense to complex realities.

The differential value is found in the way of combining data, research designs, collection processes, terminologies, procedures, which in their mixture lead to differentiated results. The amalgamation (not just the sum) of quantitative and qualitative data in the same design or research method, is something new, with great potential for more complete studies. Hence, there is little doubt that the mixed-methods approach brings with it a combination of strategies and processes to address the complexity of the education phenomenon. One that offers a holistic and interdisciplinary understanding and that has the potential to change the way research is done in our field. This study is an invitation to continue exploring and researching this topic and specifically expand our knowledge around how to bring continuous educational innovation into our work. This, we believe it is an opportunity for change and improvement that should not be missed.1