What does eye-tracking technology offer the teacher?

Eye-tracking technology is used as a tool supporting the study of human behavior and can be applied in various fields, including education (Tsai et al., 2022). More specifically, it can be used in the education field to study attention levels and their relationship with learners’ cognitive processes (Taub & Azevedo, 2019) while doing a task (Ollesch et al., 2021). As noted above, the technology provides static and dynamic metrics. The static metrics cover a range of different parameters (duration, frequency, speed, and number of fixations and saccades, along with pupil diameter, etc.) (Merchie et al., 2022). The ultimate aim of analyzing these metrics is to detect patterns of behavior with the stimulus, and, based on the learning results, determine which is more effective (Chango et al., 2022). The specific benefits of these static metrics in education are related to the information they give to teachers about students’ levels of attention and cognitive effort when doing various types of tasks (Prokop et al., 2020; Shojaeizadeh et al., 2019; Yang & Wang, 2023). This data allows teachers to tailor the content or structure of learning tasks to the needs of each student, driving the personalization of learning (Chemerys & Ponomarenko, 2022; Ollesch et al., 2021).

Eye-tracking technology also records dynamic metrics about the path and time of a student’s gaze while they complete a task (Diwanji, 2022; van-Marlen et al., 2018). Within each stimulus (elements making up a learning task), teachers can differentiate relevant Areas of Interest (AOI) (the information the teacher considers to be most important) and non-relevant areas (those they think are less important). Once that differentiation of AOIs is made, teachers can find patterns of student behavior within the applied learning stimulus (Coskun & Cagiltay, 2022; Merchie et al., 2022; Tsai et al., 2022). The relationship between those patterns and learning outcomes can then be analyzed (McLeod et al., 2022; Ollesch et al., 2021) to identify the most effective (Feng & Law, 2021; McLeod et al., 2022; Merchie et al., 2022; Tsai et al., 2022; Yang & Wang, 2023). In summary, the data offered by eye-tracking allows teachers to provide specific pedagogical guidance aimed particularly at students with less satisfactory results (Molina et al., 2017; Sáiz-Manzanares et al., 2021b).

What are the problems and limitations of using eye-tracking technology in a natural learning context?

Eye-tracking technology has become much more user-friendly over the last ten years and modern eye-tracking equipment can be applied more easily in natural settings (Schweizer et al., 2022). The technology can also be incorporated into augmented reality devices (Thees et al., 2022), virtual reality devices (Chango et al., 2022; Mills et al., 2016; Rother & Spiliopoulou, 2022; Yang & Wang, 2023), or mobile devices (Kuhnel et al., 2018). All this functionality will make it easier to apply the technology in day-to-day teaching. However, using eye tracking presents significant challenges for teachers.

Firstly, although the devices have become smaller and can be used outside the laboratory, they still need skilled users to deal with them. The process in an educational setting would involve 1) designing the learning task; 2) integrating the task and the eye-tracking device; 3) calibrating the students’ gaze for the screen; 4) once the learning activity is completed, the recorded data must be extracted, refined, treated, and interpreted. Currently, eye-tracking technology offers a simple automatic interpretation of data. However, that does not include all of the data treatment and analysis possibilities. The challenge in this regard is the large volume of data that these devices collect (Sáiz-Manzanares et al., 2020). This is a very important functionality but requires the application of Educational Data Mining (EDM) techniques in order to properly analyze and interpret the results (Chango et al., 2022; Feng & Law, 2021). These techniques involve using Machine Learning algorithms to analyze educational data (Bogarín et al., 2018). In short, eye-tracking technology in educational contexts is promising, but to use it well requires teachers to be trained in the aspects indicated above.

What are the possibilities of using other neurological evaluation resources with eye-tracking technology?

Other recording devices can be included with eye-tracking technology, such as devices to measure galvanic skin response (GSR) (He et al., 2022), electroencephalography (EEG), (Luo & Zhou, 2020; Scharinger et al., 2020), and recording the learner’s face for subsequent analysis of emotions during a task. The technology, depending on the sophistication of the equipment, can also facilitate incorporating these metrics in multiple records. That would give greater confidence in the interpretations of students’ learning processes (Alemdag & Cagiltay, 2018), although they would need to be applied in laboratory settings. It is also important to bear in mind that they would provide information about students’ information processing during different types of tasks (Giannakos et al., 2019).

As already noted, task design is fundamental in this context. This is because, depending on each learner’s learning style, one presentation or another of the same task could be more—or less—effective for the learning process. Along these lines, studies using multi-channel analysis (Azevedo & Gašević, 2019; Taub et al., 2017) have concluded that tasks that are offered through dual channels (audio and video) increase the cognitive load in information processing (Souchet et al., 2022).

In addition, tasks that are presented through video and which require the application of procedural knowledge have a higher cognitive load for the student (Pi & Hong, 2016). In contrast, it seems that including graphics or diagrams in tasks reduces cognitive load and eases understanding (Fiorella, 2022). Nonetheless, interpreting these results requires the application of EDM techniques, the most notable of which are described below.

What does using EDM techniques offer for the interpretation of eye-tracking results from the teaching-learning process?

EDM techniques can be defined as the application of Data Mining (DM) techniques in an educational setting (Chango et al., 2022). More specifically, DM techniques use some Automated Learning techniques. These may be supervised, which include prediction (Seinen et al., 2022) and classification techniques (Díez-Pastor et al., 2014); or unsupervised, which include clustering (Sáiz-Manzanares et al., 2021a). Table 2 gives an overview of the most important EDM techniques in the education field.

Does the use of eye-tracking technology present new ethical challenges in the teaching-learning process?

Clearly, making records that include student activity during tasks brings with it ethical challenges. These center on the necessary authorization from the student to participate in this type of teaching. Students might have misgivings about granting this authorization as it means recording a task and might include images of them. This means that a procedure of this kind might introduce biases due to authorization from some students and not others because the students who do participate will get feedback to improve their learning processes while others will not. That will clearly lead to a situation with an equality gap in the pedagogical intervention which may cause increased differences in academic performance between students who do and do not participate. In summary, eye-tracking technology is promising in the field of education as it allows teachers to produce data about how their students process information. By analyzing that data, teachers can also identify the patterns of behavior during learning. All this information will presumably help teachers to personalize the design of learning tasks, which will facilitate education tailored to the needs of each student. This will enhance attention to diversity and foreseeably promote better academic inclusion. Based on all of the above, the research questions (RQ) for the present study were:

• RQ1: Will there be significant differences between students with and without prior knowledge in the parameters of cognitive load (FC, FD, SC, PD, VC) and in the learning outcomes (LO)?

• RQ2: Will there be clusters of patterns in learning behaviors that are different between students?

• RQ3: Will the study find patterns of learning behaviors that are more effective and less effective?

Participants

The study used a non-probabilistic convenience sample (based on agreeing to participate in the study) of 20 university students from three universities: 7 from the University of Burgos (UBU) (3 women M_age=22 years, SD_age=2.00; 4 men M_age=21 years, SD_age=1.41) 11 from the University of Valladolid (UVA) (8 women M_age=22.38 years, SD_age=2.77; 3 men M_age=24 years, SD_age=1.00), and 2 from the University of Miño in Portugal (UMINHO) (2 women M_age=21.50 years, SD_age=0.70). The students were identified as with or without prior knowledge based on the knowledge branch of the course they were studying. Those studying Art History (n=7) were considered to have prior knowledge, whereas those studying Psychology (n=5) and Education (n=8) were considered not to have prior knowledge.

Instruments

a) Tobii Pro Lab version 1.194.41215 and a 15.6-inch monitor with a resolution of 1920 X 1080, at 64 Hz. Static metrics in this study were analyzed using SPSS software v. 28 (IBM, 2022). Dynamic metrics were recorded via the gaze point offered by Tobbi and Orange 3.33.0 was used to produce the heat maps.

b) The stimulus for the eye-tracking technology was a self-regulated video about the development of monasteries in Europe. A recorded voice read out information, putting stress on the most important data. The video also included diagrams and pictures summarizing the most important content. The information for the video was taken from the “SmartArt Toolkit for Knowledge Transfer” which can be found at https://bit.ly/3CPkEwq. The SmartArt project was co-financed by the European Union (information about the project can be found at http://bit.ly/3kLeF5J). The video used in the study is available at http://bit.ly/3VzdMd9. The video contains 1 non-relevant AOI—AO1_NR1— (an image of an avatar) and 3 relevant AOIs: Relevant AOI 1 (AOI R1; an outline of important concepts); Relevant AOI 2 (AOI R2; summary of the most important spoken content); and Relevant AOI 3 (AOI R3; key concepts from the spoken information). An example showing the distribution of the AOIs is given in Figure 1.

c) Knowledge test about the concepts seen in the video. This test consists of 9 questions about the most important concepts in the video. The questions were taken from the “SmartArt Toolkit for Knowledge Transfer” and were the first 9 questions from “Self-assessment questionnaire 1. Basic level” (p. 63-64), which can be found at https://bit.ly/3CPkEwq.

Procedure

Before carrying out the study, approval was obtained from the UBU Bioethics Committee (IR 27/2019). Following that, students at UBU, UVA, and UMINHO were offered the chance to participate in the study. Students who agreed signed their informed consent to participate voluntarily and without remuneration. The experimental phase was carried out with each student individually in two classrooms (one in UBU, the other in UVA) with controlled lighting conditions and without distracting stimuli. The experiment was performed in all cases by the same researcher who was an expert in the use of this technology.

Research design

This study was exploratory. To test RQ1, given the characteristics of the sample, we applied a quasi-experimental design without a control group where the independent variable was “prior knowledge of art history” and the dependent variables were the parameters of cognitive load and the learning outcome. To test RQ2 and RQ3 we applied a descriptive design (Campbell & Stanley, 2005).

Data analysis

Because non-probabilistic convenience sampling was used, and because the sample was smaller than 30 participants, we used non-parametric statistics to test the RQs. For RQ1, we used the Mann-Whitney U test and calculated the non-parametric effect size r, with the cutoff points: (a) up to 0.30, small; (b) from 0.30 to 0.50, moderate; (c) over 0.50, high. This analysis was done using SPSS v. 28 (IBM Corp, 2022). For RQ2 we used k-means clustering using SPSS v. 28 (IBM Corp, 2022) along with visualization using heat maps and clustering using Machine Learning and data visualization from the Orange toolbox 3.33.0 (Demšar et al., 2013). Finally, for RQ3, we used the visualization of gaze point patterns offered by Tobii Pro Lab version 1.194.41215.