Introduction, approach, and methodology

The technological evolution of recent years has had a positive and/or negative impact on societies worldwide, affecting people’s modus vivendi at work, in health, economy, and, obviously, education and training (Alonso-de-Castro & García-Peñalvo, 2022).

Sustainable Development Goal 4 (SDG4) of the 2030 Agenda (United Nations, 2019), promoted by UNESCO, is also a common goal of states and governments. SDG4 seeks to “ensure inclusive and equitable quality education and promote lifelong learning opportunities for all” (UNESCO, 2019). Furthermore, technologies play a crucial role in achieving this goal. Thus, over the last two decades, emerging technologies have disrupted teaching-learning processes at different levels of education. In this context, artificial intelligence (AI) is a potential technological tool (Vitanza et al., 2019) that, without being new, is expanding in all professional and knowledge fields, affecting, impacting, and causing a real revolution in the field of education. This branch of computer science is responsible for studying and assessing the possibilities for a machine to perform human tasks (Túñez-López & Tejedor-Calvo, 2019). According to the UNESCO (2021b) document “Recommendation on Artificial Intelligence”, AI systems are information processing technologies that integrate models and algorithms that produce a capacity to learn and perform cognitive tasks, leading to results such as prediction and decision-making in material and virtual environments.

Thus, SDG4 emphasizes that AI technologies are used to ensure equitable and inclusive access to education (UNESCO, 2019). This implies having to change the role of teachers in the transmission of the knowledge they provide to the younger generations. Other authors (Osetskyi et al., 2019) argue that the future of higher education is inextricably linked to the development and corresponding increase in the capacity of new intelligent machines capable of operating large amounts of information, self-learning, and improvement, whereby AI has become a new focus of international competition of countries in the educational market. Therefore, we are witnessing how AI’s presence in the educational field is constantly growing through robotics or algorithms. In fact, its immersion in this field has occurred “silently” hand in hand with other technologies already consolidated in the educational field, such as virtual campuses and academic social networks (Flores-Vivar & Zaharía, 2019). Along these lines, a series of research studies and analyses have been carried out by specialists who, through publications, seminars, and congresses, paint a good picture, but also one of the ethical solid, and deontological implications in the use of AI in the field of education. An example is the “International Forum on AI and the Futures of Education Developing Competencies for the AI Era” (UNESCO, 2021a), held in December 2020. Building on the Beijing Consensus, the forum shared policies and practices regarding the role of AI in education, with a specific focus on defining the competencies required in the AI era and strategies to prepare all people to live and work with AI effectively. The European Commission (2020) also proposes an “EU strategic framework based on EU values that will give citizens the confidence to accept AI-based solutions”. For its part, the European Parliament (2021) has been legislating on the implementation and use of AI technologies in the education sector, warning that they are high-risk technologies and should therefore be subject to stricter security, transparency, fairness, and accountability requirements. Being a particular area of study in engineering and computer science, we see that AI is increasingly being applied in the field of education, supporting the management, learning, and evaluation of students. One example is the latest developments in Google Classroom (Kiecza, 2022), which uses AI to help students and teachers. Other examples include using AI to automate routine administrative tasks, diagnose student competencies, and provide learning content and feedback tailored to individual progress. Nevertheless, the implementation and development of AI technologies applied to education must be seen as a complex and highly controversial issue (Selwyn, 2019), especially regarding the ethical and deontological values that must be impregnated from the design and subsequent development phase. Hence, the importance of AI in the field of teaching–learning is growing more and more. AI is increasingly pervasive and has transcended national borders, so “all countries must work together to ensure that AI remains under human control and is designed and applied for the common good” (UNESCO, 2021a: 5). In particular, it is essential that humans are protected from becoming victims of AI tools, so we must understand that AI must be used to augment and amplify human capabilities but not replace them. Moreover, this understanding begins in education.

According to the report “Artificial Intelligence in Education: Challenges and Opportunities for Sustainable Development” (UNESCO, 2019), “Artificial intelligence can be a great opportunity to accelerate the achievement of the SDGs. However, technological revolution leads to new imbalances that we must anticipate”. These imbalances are caused by the impact generated. Hence, rapid technological advances in AI and other advanced technologies, such as robotics, cloud computing, and the Internet of Things, are transforming disciplines, economies, and industries and challenging ideas about what it means to be human.

AI has enormous potential for social good and for promoting the achievement of SDGs if it is developed in a way that benefits humanity, respects global norms and standards, and is anchored in peace and development. Considering this, this paper aims to explore the contexts, uses, and conditions that favor –or not– the application of AI, along with the ethical and regulatory disjunctions that may be critical in the field of education. The background is that the ethical dilemma related to the understanding and interpretability of AI agents’ behavior is one of the fundamental challenges of the next decade in this area (Flogie & Aberšek, 2021).

On the one hand, this study seeks to analyze and explain an existing problem, which has been addressed for some years according to the scientific literature reviewed, the rise of AI in education, the unknown about its uses and ethical aspects in its development, and factors that affect the users of these technologies (teachers and students). On the other hand, it proposes new conjectures, challenges, and paradigms that, together with the implementation and consolidation of AI in education, should be considered for future – and present – generations of these groups.

Therefore, the methodology used in this work is based on a documentary review of primary and secondary sources on the development of AI and its capacity for action in the educational field, consistent with SDG4 for quality and inclusive education. This review has been done considering that the literature on the subject is still scarce, "focusing more on the computer part and less on the impacts of AI on society" (Canavilhas, 2022). In the criteria for the selection of sources from international institutions and organizations, such as UNESCO, United Nations, OECD, European Commission, or European Parliament, it was considered that they are organizations that seek to strengthen the development of quality and inclusive education worldwide and that, for years, have been addressing the use of AI in this area. As for the scientific papers analyzed, the criteria were based on the specific subject matter, the level of the publications, and the expert positions represented by many authors, considering that it was not possible to include all of them for space reasons. From these analyses, we seek to extrapolate and address the ethical and philosophical issues that have been raised by experts and organizations around the world, seeking reflection, debate, and foresight on the profound ethical and deontological implications that AI, in general, will have in the global educational field. According to UNESCO (2021a), regulation is necessary. With this research, we propose paradigms and challenges regarding the motivations and the ethical issues of AI applied to education.

Connecting AI with the educational ecosystem: opportunities, challenges, or risks?

Since UNESCO adopted the 2030 Agenda for Sustainable Development, whose key objectives include ensuring equitable access to quality education worldwide (SDG4), technological development and evolution have changed teachers’ roles in transmitting knowledge. Technological evolution configures a technological ecosystem, a “metaphor to express a necessary evolution of traditional information systems, with solutions based on the composition of different software components and services that share a set of semantically defined data flows” (García-Holgado & García-Peñalvo, 2019). Part of this evolution is the impact that algorithms and robotization are causing in the field of education, so many “fear that robots driven by artificial intelligence will soon completely replace human teachers” (Spirina, 2018). There is a risk in the making, but also an opportunity. It is more than likely that the e-learning industry (Crisol-Moya et al., 2020) will focus on “swarms of robots as a novel educational tool to focus on precisely those cross-cutting skills that are difficult to explain otherwise” (Global Market Insights, 2021). What is beyond doubt is that the role of the teacher is going through a golden age with AI (Benjamins & Salazar, 2020; Craig, 2018; OECD, 2021; Oliver, 2020; Ramírez-Montoya et al., 2022; Selwyn, 2019; UNESCO, 2021a). In other fields, it has reduced the demand for workers performing mundane tasks, improved output standards, and enabled humanity to solve complicated health, logistics, and safety tasks that require informed decision-making. In this sense, the trend is that AI will create more jobs than it will eliminate (Fundación Telefónica, 2015). However, many professions, such as teaching, training, and education, undoubtedly require the human mind, not AI. This does not mean that the cognitive capacity of teachers, the continuous flow in the transmission of knowledge among their students, and the tasks of management and content creation can be supported by algorithmic assistants and AI tools.

A teacher’s workload often exceeds what is reasonable, since a teacher is expected to monitor students’ academic performance, grade homework, prepare lessons and a long list of academic activities, and carry out research activity that requires more dedicated time. Thus, the advance of emerging technologies may be on the way to transforming teaching and learning, leading to a disruption in education, as we know it today. With this in mind, experts agree that AI in education has the mission to help in the planning, personalization, visualization, and facilitation of the learning process.

In this context, recent research suggests that AI will be a great novelty in education and the teaching–learning process, as well as the engine of what is already being called Education 4.0 (Fidalgo-Blanco et al., 2022; Ramírez-Montoya et al., 2022). Some argue that the role of AI will range from initial education (infants) to higher education, including business training. It will enhance the student experience by providing the opportunity to create adaptive learning functions and models with personalized tools (Flogie & Aberšek, 2021). It should be noted that both teachers and students are increasingly immersed in the use of technologies and platforms that optimize, on the one hand, the transmission of knowledge and, on the other hand, the acquisition of new learning. In summary, AI will have different educational roles (Affde, 2021; UNESCO, 2021a). Table 1 describes some of the roles and challenges in which AI is changing the field of education globally.

Ethical aspects and principles of AI for the common good

Digitalization opens up new possibilities for education (OECD, 2021). In this line, the evolution of technology and the development of AI present not only opportunities but also risks and, above all, many unknowns and concerns (Selwyn et al., 2022). One of the risks has to do with access to information that is being heavily manipulated by “behavior modification empires” (Lanier, 2018, p. 22) in companies such as Facebook, Google, and Twitter, or how an army of mathematicians is being hired to process information “with data extracted from social networks and e-commerce sites in order to analyze people’s behavior” (O’Neil, 2017, p. 10). Furthermore, what proliferates most in education are data and information. With this scenario full of unknowns, challenges to the ethics of AI are configured as the epicenter of the debate. Therefore, at a more general level, there is the “need to differentiate between doing ethical things and doing things ethically, to understand and to make pedagogical choices that are ethical, and to account for the ever-present possibility of unintended consequences” (Holmes et al., 2022), nuances that are necessary to underline in the implementation of AI in the educational field. In this context, the underlying questions are the following: What is the role of AI in education? What is the truth in the hypothetical replacement of teachers by AI? Can AI cultivate critical thinking in the minds of students? How do (or can) rapid advances in AI, learning analytics, robotics, etc. change the way teachers teach and students learn? (OECD, 2021). Moreover (Selwyn, 2019), should we put the judgments of humans above those of machines? Should systems and applications that prove effective in terms of learning –or saving money– be mandatory? What does the continued rise of AI mean for education? What are the implications for education in the next decade? What problems arise that require rethinking AI and how to implement it effectively in educational contexts? Have the ethical issues involved in ceding the field of education to algorithmic developments been addressed?

The latest studies published, not only in scientific media but also in the general public, seek to make people understand the scope of AI, especially because, as Stephen Hawking stated, “AI can be the best or the worst thing that has happened in the History of Humanity” (Infobae, 2017). Thus, the educational environment is affected by all these AI-generated changes, ranging from preschool stages to higher or graduate levels (Moreno & Pedreño, 2020). This implies, as experts argue, “that the complexity and continuous change of the forces of the new environment requires the rethinking of education from a vision of lifelong learning” (Latorre et al., 2021: 13). However, this rethinking with the digital automation of teaching does not simply consist of the technical aspect of designing, programming, and implementing systems with greater efficiency (Selwyn, 2019).

In this sense, several specialists, proper authorities in this technological field, wonder whether strong AI (SAI) would have actual intelligence, genuine understanding, and creativity, and, consequently, whether it would have an identity, moral integrity, and free will. For Boden (2022), “these are not scientific questions, but philosophical ones [...] some would say it does not matter: what matters is what SAIs will actually do. However, the answers could affect how we relate to them”.

For her part, Gabriela Ramos, Assistant Director-General for Social and Human Sciences at UNESCO, with the recent “Recommendation on the Ethics of Artificial Intelligence” (Urbán, 2021), maintains that “what happens is that we always try to persecute [the AI advances], but the whole cycle of AI must be contemplated from research, development, implementation, impact assessment, feedback, development of algorithms, the conceptual framework of algorithms... All these steps require ethical reflection and require controls to be established”.

In a similar spirit, Benjamins and Salazar (2020) argue that “since it is not possible to teach ethics to a machine, it will be up to the companies and their developers to demand that the development and use of AI be carried out ethically, under international human rights"’, applicable to all fields of knowledge, including education. For its part, the European Commission proposes several guidelines to be considered when developing reliable and safe ethics in the European Union.

These are issues that increasingly affect and concern the academic community: teachers and students, but they also affect organizations, public and private entities, governments, and, in short, society in general. However, as Oliver (2020) points out, the “first step to facing what we fear is to learn; to learn, in this case, with the technology we live with, with the aspiration of understanding the world better, and, therefore, to be able to make informed decisions about it”.

In this horizon of learning about technology and the potential of AI (Craig, 2018; Mondada et al., 2017; Nye, 2015), other key technologies that apply to the field of AI should be considered. For example, learning analytics, although still a young field, is a powerful resource for informed decision-making and better learning outcomes. One of the studies in the “Working Papers on Education Policy” (UNESCO, 2019) argues that learning analytics applies different areas of knowledge, such as sociology, psychology, ethics, and pedagogy, and can now access the digital revolution to collect a large amount of data that can be analyzed to extract insights or even develop smart tools useful for educational or administrative tasks. Another example is big data. According to Bonami et al. (2020), “AI has become a natural development of an intelligent system that needs to deal with Big Data, which is why the terms are structurally connected”. Also, one more example is robotics with the design and creation of robot teacher models. Indeed, recent generations of robots in classrooms (Selwyn, 2019) consist of computer systems designed to learn independently, develop logical processes, and formulate mental models that allow them to make complex decisions. Hence, “it is increasingly common for algorithms and programs to perform our functions, to the point that people have become, in a way, dispensable in some jobs” (Cukier et al., 2021), so the educational field does not escape these impacts. In fact, Selwyn (2019) argues that “the long-held professional status of teachers in schools and universities is undoubtedly under threat”. To this, we must add the argument made by Cukier et al. (2021), who state that AI algorithms are unable to impose constraints and limitations on their own as people do: “The large computational capacity we have today, machines can evaluate a larger decision space much more efficiently and quickly than a person in the same amount of time”.

Therefore, implementing AI technologies in education must be seen as a complex and highly controversial issue (Selwyn, 2019), where morals and ethical principles must be solidly supported through regulation with a global approach.

If the application of AI, in general, raises questions, several researchers (Holmes et al., 2022; Osetskyi et al., 2019; Selwyn et al., 2022) point out that AI, along with its ethical uses in the field of education, generates uncertainty and expectation among the various actors in the academic world, perhaps in the same way as happens with any disruptive technology applied to another field of knowledge. Hence, the conclusions of the UNESCO Forum (UNESCO, 2021a) emphasize that once AI’s critical human and technical competencies have been identified, school systems must ensure that all students are best prepared for a world in which AI is ubiquitous.

Experts point out that the many complex challenges associated with AI need to be appropriately considered. For example, it is known that AI is sometimes biased; however, the cause is not always apparent. AI decisions may be biased because the human data on which they are based are biased and/or because the algorithms applied and developed by humans are biased. In other words, AI mirrors and exemplifies human decision-making, which reinforces the need for humans to step up and take responsibility and control. In this regard, an increasing number of researchers (Cukier et al., 2021; Selwyn, 2019) agree with the idea of intelligent systems making decisions and then acting on these decisions, so it obviously has ethical implications. So, the following question arises: what moral and ethical principles should govern what we do with AI in education and what tasks should they be programmed to do on their “own initiative”? The answer (for the time being) is self-evident: the ethical code must be assumed by the creator/programmer of the AI system.

Potential and literacy of AI in quality education

The previous step to assuming the principles of ethics in AI requires not only the awareness of values but also more knowledge about the potential of this technology. Therefore, in coherence with the UNESCO Forum, held in December 2020, and SDG4 of the 2030 Agenda, knowledge of AI implies designing and developing an algorithmic literacy plan, which should be included in the educational curricula of any field of knowledge. These interdisciplinary and subject-specific curricula that include learning about AI (from its technological explanation to the ethical and philosophical issues of its impact) should reference what has been done by pioneering countries. Furthermore, they should be flexible, open, inclusive, and continuously evolving. Hence, multiple stakeholders could be involved in the design of learning materials, such as AI-based interactive textbooks. According to the “International Forum on AI and the Futures of Education Developing Competencies for the AI Era” (UNESCO, 2021a), “technology-oriented competencies focus on AI techniques, technologies, and applications, and include the advanced AI knowledge and skills needed to create, manipulate, implement and interpret AI”. In this regard, the studies analyzed agree that AI literacy should take both a specific and an interdisciplinary approach: “Specific curricula and courses, covering both human and technological aspects of AI, need to be established, building on existing ICT curricula and courses. In addition, the potential and impact of AI should be considered in all school subjects, be they sciences, humanities, or arts” (UNESCO, 2021a). Nevertheless, the most important thing lies in training the trainer, i.e., teacher training plans, so that teachers and educators can receive adequate training. In this line, both researchers and experts, as well as institutions and organizations (UNESCO, 2022; IRCAI, 2022), propose an ethics of AI, which should address various issues and fundamental principles based on the responsibility, privacy, fairness, and explainability (Villas & Camacho, 2022) of the same that should be included in a digital literacy plan (García-Orosa, 2021; Salazar & Benjamins, 2021). All agree that it is necessary to provide teachers with skills and digital competencies since today’s society, and even the very existence of the human being inserted in it, is increasingly dependent on access to communication and information technologies (Aguaded & Romero-Rodríguez, 2015). The field of education is no stranger to this dependence. AI competencies could also be developed in extracurricular activities, such as seminars, workshops, coding debate clubs, or the realization of hackathons, as well as in lifelong learning programs related to academic, scientific, and professional work (Spirina, 2018). Such integration would equip students to increasingly understand how to interact with AI systems, make informed decisions, and prepare for the societal impact of the widespread use of AI in employment, health, democracy, and, in short, in their daily lives. In this regard, Long and Magerko’s (UNESCO, 2022) proposal on “AI Literacy: Competencies and Design Considerations” may be emphasized. They propose a set of competencies and design considerations for AI literacy based on a scoping study of existing research, which sought to determine emerging themes in 1) what AI experts believe a non-technical audience should know and 2) common perceptions and misconceptions among learners.

In this context, current AI-driven educational tools, while effective in some contexts, remain limited. Most are limited by the variety of subjects they cover, generally mathematics and some sciences; by the pedagogical approach they take, often at odds with approaches inspired by learning science orthodoxy; and by their prioritization of machine-based (human–computer) interactions over human-to-human (human-to-human) interactions. Both the potential and limitations have yet to be fully identified or addressed. However, it is clear and widely accepted that AI should not (but perhaps can) replace schools or teachers (Selwyn, 2019). Instead, social interactions, between students and teachers and among students themselves, should remain the epicenter of learning. In addition, some existing AI tools can reduce students’ capacity, so it is essential to support this group to be active participants in their learning processes and take more responsibility for their own learning.

Results and conclusions

According to our initial approach, from the conclusions drawn, it can be deduced that the scope of SDG4 for Quality and Inclusive Education of the 2030 Agenda, among other goals, entails the access to and continuous use of emerging technologies that generate disruption in the teaching and learning process. AI, as part of that disruption, is increasingly immersed in the field of education, so it can easily become the best ally of students and teachers, providing personalized pedagogical content and personalized tutoring and assistance when needed. Therefore, in addition to their cognitive capabilities in algorithmic design, AI developers must consider ethical principles in developing tools that they design with goals or claims to teach better than teachers. Thus, AI regulations and ethics must be achieved without compromising human values, undermining diversity, and creating new inequalities.

However, it must be made clear that AI cannot be a source of inspiration for learning and that, with a human teacher, where empathy in the teaching-learning process is higher, such inspiration plays a key role. Moreover, this is the main disadvantage of AI-driven education. At the same time, this is the main reason it will not wholly replace human teachers. However, AI is already being used to support remote learning in different modes with remotely delivered, guided, and shared classes, each of which brings benefits and challenges. On the other hand, the various AI models created by Big Tech (GAFAM) (Benjamins & Salazar, 2020) are becoming increasingly sophisticated. However, expert assessments differ regarding whether SAI (for the moment) has real intelligence, understanding, or creativity, but even less identity, moral integrity, or free will. However, advances in AI and machine learning coupled with the development of natural language processing are changing and evolving rapidly, so it is likely that in little more than five years, the “technological singularity” (Benjamins & Salazar, 2020) of AI, i.e., the moment when AI catches up with human intelligence, will occur. All these trends will continue to crescendo, so it is necessary to focus attention on the ethical implications, especially in a field such as education. Therefore, without claiming to be exhaustive and based on the various reports and references cited, it is suggested to reflect on proposals for some paradigms and ethical challenges of AI in education. These challenges should be posed from both teaching and research perspectives and from the point of view of students and institutions. The resolution of these approaches would help clear the unknowns (raised in Section 3) and blur the impact that AI systems currently present, not only in education but also in society.