Analysis of short videos on TikTok for learning Portuguese as a foreign language
Abstract
In recent years, short videos have proliferated on various social media platforms, serving as a visual resource with a long history of use for various educational purposes, including foreign language learning (FLL). Despite the growing interest in the use of such platforms for short video learning, little is currently known about the use of these platforms for FLL, highlighting the need to validate these learning resources within a theoretical framework. In this context, this article analyses 34 short videos for Portuguese language learning on TikTok using an adaptation of the principles of multimedia learning proposed by Mayer. It then analyses the relation between the number of principles and each cognitive processing dimension, and the number of views and likes. Results show the adapted principles are suitable for the analysis of short videos for FLL and reveal the most prevalent one is the principle of signalling. Furthermore, the article demonstrates that the more principles a video addresses, the more likes it receives, and that the fostering generative processing dimension plays an important role in this mediation. Overall, the article illustrates the practical application of online short videos as a targeted resource for facilitating FLL and streamlines the development of instructional design strategies for short videos in the context of online FLL.
Keywords
Short videos, instructional videos, multimedia learning, foreign language, Portuguese, TikTok
Palabras clave
Vídeos cortos, vídeos instructivos, aprendizaje multimedia, lengua extranjera, portugués, TikTok
Resumen
En los últimos años, los vídeos cortos han proliferado en diversas plataformas de medios sociales, sirviendo como un recurso visual con una larga historia de uso para diversos fines educativos, incluyendo el aprendizaje de lenguas extranjeras (FLL). A pesar del creciente interés en el uso de estas plataformas para el aprendizaje de vídeos cortos, poco se sabe actualmente sobre el uso de estas plataformas para FLL, destacando la necesidad de validar estos recursos de aprendizaje dentro de un marco teórico. En este contexto, este artículo analiza 34 vídeos cortos para el aprendizaje de la lengua portuguesa en TikTok utilizando una adaptación de los principios del aprendizaje multimedia propuestos por Mayer. Analiza la relación que existe entre el número de principios y cada dimensión de procesamiento cognitivo, así como el número de visualizaciones y «likes». Los resultados muestran que los principios adaptados son adecuados para el análisis de vídeos cortos para FLL y revelan que el más prevalente es el principio de señalización. El artículo demuestra que cuantos más principios aborda un vídeo, más «likes» recibe, y que la dimensión de fomento del procesamiento generativo desempeña un papel importante en esta mediación. En general, el artículo ilustra la aplicación práctica de vídeos cortos como un recurso específico para facilitar el FLL y agiliza el desarrollo de estrategias de diseño instruccional para vídeos cortos en el contexto del FLL en línea.
Keywords
Short videos, instructional videos, multimedia learning, foreign language, Portuguese, TikTok
Palabras clave
Vídeos cortos, vídeos instructivos, aprendizaje multimedia, lengua extranjera, portugués, TikTok
Introduction
Instructional videos, mainly consisting of text and visual content, are a well-known educational resource for teaching and learning foreign languages (Pisarenko, 2017). In instructional videos, the instructor presents words in the form of narration or printed text, while visual content is presented through on-screen slides or animations (Mayer, 2021).
In today's Internet video era, people have access to various forms of online and instructional videos in both formal and informal learning environments (Ou et al., 2019). However, while such access occurred several times during the twentieth century through different technological outlets, it quickly faded for educators, because such resources were not used effectively to support learning (Mayer, 2021). Therefore, the current era may face the same problem as the twentieth century, namely that the easy access to video brought by the Internet may fade in the education field due to a lack of educational impact (Mayer, 2020).
To minimise this risk, instructional videos should follow specific guidelines known to foster and improve learning (Brame, 2016). One example of such guidelines are the principles of multimedia learning (Mayer, 2020), which are grounded in cognitive theory and presume three kinds of instructional design goals or dimensions: 1) reducing extraneous processing, which includes principles for helping learners to avoid distractions and focus on essential material (coherence, signalling, redundancy, spatial contiguity, and temporal contiguity); 2) managing essential processing, which includes principles for helping learners to process complex essential material (segmenting, pretraining, and modality); and 3) fostering generative processing, which includes principles for motivating learners to work towards understanding essential material (multimedia, personalisation, voice, image, embodiment, and generative activity).
Recent research suggests that some of these principles are likely to be more applicable to students with lower levels of knowledge and to shorter courses (Mayer, 2020). For example, the multimedia principle, which assists learners in making connections between graphics and language, may be more appropriate for lower-level learners. The signalling principle is especially suitable for learners with limited skills and knowledge. The segmenting principle aims to break information into smaller pieces and adapt it to the user's learning pace. The modality principle works better with short, fast-paced videos and the personalization principle is most likely to be effective if the learner is a beginner and the lesson is short (Mayer, 2020).
Short videos are not a new concept in the field of FLL, having appeared in previous generations of visual technologies (Zhang et al., 2022a). Regarding social platforms, TikTok pioneered the short video format in recent years, but this format has caused other platforms to transform themselves, making short videos the most prominent content format nowadays in social media. TikTok has faced the challenge of rapidly growing and diversifying the platform's features and content to adapt to an increasingly international audience. This explosive change has led TikTok to be considered as a separate category from its long form ancestor, along with the short form video format (Kaye et al., 2022). However, despite the growing interest in the use of such platforms for learning (Khlaif & Salha, 2021), their use for FLL is still incipient. Therefore, as these platforms have begun to be used for FLL, it is important to analyse this form of learning against a validated theoretical structure. As such, in this study, we propose that Mayer’s multimedia learning principles may constitute a quality framework and index for the analysis of short videos on TikTok aimed at fostering FLL. In parallel, given the growing attention to the quality of user experience in online instructional videos (Habibi & Salim, 2021), this article also discusses the ongoing debate about the relationship between the number of views and likes and the quality of videos, using user engagement data provided by TikTok.
In this context, the aim of this study is twofold: 1) to analyse short videos for the teaching and learning of Portuguese as a foreign language using an adaptation of Mayer’s multimedia learning principles and 2) to understand the potential relationships among the principles and the number of views and likes of the videos. This will enable the identification of the principles that are more applicable to the design of short videos and those that are more likely to contribute to FLL. By doing so, this study offers a set of guidelines that may inform and be helpful for the design of quality and effective short videos for FLL on social platforms.
The principles of multimedia learning and FLL
First, the characteristics of FLL should be considered. For native speakers, on-screen text competes with the processing power of the visual channel, which may cause learners to miss some visual information in the video, especially in fast paced situations (Lee & Mayer, 2018). However, for foreign language learners, presenting text and narration together is more effective than presenting narration alone. This is because written material on the screen can provide a means of capturing attention, which is particularly useful when the learner is unable to access auditory information (Gass et al., 2019).
Therefore, the redundancy and modality principles are interpreted differently by native speakers and foreign language learners. For native speakers, the redundancy principle is that people learn better from pictures and narration than from pictures, narration, and printed text. And the modality principle states that people learn more deeply from pictures and spoken words than from pictures and printed words (Mayer, 2020). However, if foreign language learners are unfamiliar with the words, presenting them in a narrated video may be an extra burden, whereas a printed text may provide a more sustained presentation. Therefore, the redundancy and modality principles do not apply to foreign language multimedia learning, as narrated videos with printed text are more effective than narrated videos (Lee & Mayer, 2018).
Then, the characteristics of short online videos should also be considered; they are very short by nature and their pace is usually entirely controlled by the learner. Therefore, based on the suggestion of boundary conditions for the temporal continuity principle, i.e., that this principle may be less applicable when the subsequent lesson alternates between short segments or when the learner controls the lesson (Mayer, 2020), the conditions for applying the temporal continuity principle are considered incompatible with the characteristics of the short videos in this article. So, the temporal continuity principle does not apply to the context of this study.
In parallel, we considered the generative activity principle based on these characteristics of short videos. The principle of generative activity states that people learn better when they are guided in carrying out generative learning activities, such as summarising, mapping, drawing, imagining, self-testing, self-explaining, teaching, or enacting (Fiorella & Mayer, 2016). More specifically for video lectures, the generative activity principle aims to ask learners to type in a brief explanation of the foregoing segment at pauses in the video (Mayer, 2021), or to imitate the teacher's actions during a video presentation (Mayer, 2020).
The effectiveness of generative learning strategies, on the other hand, depends on the extent to which students construct coherent mental representations of the material during the learning process. This means that students need to understand which strategies to use, when to use them, and how to use them effectively (Fiorella & Mayer, 2016). However, in tens of seconds of the short video duration, it is not possible to effectively get learners to actively pause the video and use appropriate strategies to explain what they are learning. As such, the generative activity principle is not applicable to the short videos referred to in this article.
The last principle to be adapted is the segmenting principle. This principle states that people learn better when multimedia information is presented in segments at the user’s pace, rather than as a continuous unit. For example, learners are allowed to press a button to see the next part of a slide and hear the corresponding narration (Mayer, 2021). The TikTok player fully supports these user behaviours. According to some studies, breaking audiovisual material into multiple sequences can lead to more comprehensible input (Campoy-Cubillo, 2019), and learners are more likely to comprehend if the videos are short and focused on the learning objectives (Brame, 2016).
Similarly, the main purpose of applying the segmenting principle is to break down complex materials into more manageable parts (Mayer, 2020). Given TikTok’s ability to deliver small learning units in a very short time and based on the fact that TikTok initially limited videos to no more than 60 seconds (Fiallos et al., 2021; Khlaif & Salha, 2021), we adapted the description of the segmenting principle by keeping the video duration under one minute.
A detailed summary of the multimedia learning principles adapted for the design of short videos for FLL can be found in Table 1, along with a description and examples (or evaluation criteria) for each principle. A detailed summary of the principles excluded is also included, as well as the reasons for the exclusion.
User engagement and video quality
Multimedia learning principles as an indicator of video design quality have been discussed above, but there is another indicator that should also be measured, namely user engagement. In this case, user engagement represents the quality of user experience. This topic has been receiving increasing attention in educational research (Habibi & Salim, 2021), but there is more than one way to measure it and note should be given to the fact that user engagement metrics are influenced by different factors and can assume different meanings.
For example, in TikTok, views are generated through recommendations by its algorithm (Klug et al., 2021). Videos are ranked by metrics such as views, likes, and shares, and can become popular videos with different levels of views if the combined ranking reaches different specified values (Zhao, 2021). And in addition to these video engagement metrics, views can also be associated with specific release times and popular video sections (Klug et al., 2021). However, in comparison, likes are a more direct form of user voting to express the popularity of video content (Khan, 2017), and can also be interpreted as the acceptability of the content by the viewers (Throuvala et al., 2019). Therefore, these comparisons have led to the development of educational research on the relationship between user engagement and video quality.
Rittberg et al. (2016) rated instructional videos on YouTube according to reliability, comprehensiveness, and global quality scale, concluding that video quality does not correlate with video views. Bae and Baxter (2018) analysed instructional videos on YouTube with 14 key criteria, also concluding that there was no significant difference in the number of views between the most useful and other videos. However, Shoufan (2019) used learning analytics to look into the extent to which YouTube's educational videos supported the cognitive features articulated in cognitive theories of multimedia learning.
The findings of this study show a strong or moderate correlation between the number of likes or dislikes and the educational video's cognitive value, so that the quality of educational videos is expressed as a function of the number of likes, referred to as the video cognitive value. In parallel, De-Angelis et al. (2019) evaluated the overall video quality and usefulness of surgical videos on YouTube and found a significant relationship between the number of likes and moderate/good video quality.
Therefore, considering this debate, and taking into account that the object of analysis in this article is online short videos and that TikTok supports the collection of video views and likes, this article analyses short videos according to two quality indicators: the principles of multimedia learning, considered as a quality indicator of video design, and user engagement, considered as an indicator of user experience, in terms of the number of views and likes.
Methodology
Video selection
We conducted a search on TikTok on November 1, 2022, using the keyword “Aprender português” [Learning Portuguese]. The search returned a total of 445 short videos, but as TikTok shows all short videos related to “Learning” and/or “Portuguese”, the results were further refined by applying exclusion criteria: 1) short videos with generalist content (e.g., humor, advertising) (n=174); 2) short videos for learning other languages (n=29) and 3) short videos for learning Portuguese using other languages (n=208). Our main objective was to only include short videos using Portuguese that had the explicit goal of teaching Portuguese. After applying the exclusion criteria our sample totalled 34 short videos posted between September 2021 and October 2022 1 .
Video analysis
The analysis of the short videos was conducted using the adapted multimedia learning principles presented in Table 1. Coder A coded the total sample of videos (N=34) and Coder B coded around 30% of the total sample (N=10). As the coding process may be permeable to subjective interpretation (Lombard et al., 2002), an intercoder reliability check was performed, using the Intraclass Correlation Coefficient (ICC) in the statistical software R (Falissard, 2012), checking absolute agreement in a confidence interval of 95%. The ICC was of 0.845.
Data analysis
In the first step, we characterised the sample of short videos in terms of their quality using the multimedia learning principles. The total variable used was the sum of the principles identified in each video. The frequencies of each individual principle according to the three different processing dimensions (cf. Table 1) are also presented. To better understand the relationship between the three dimensions, and since the number of principles in each dimension differs, we performed a partial polychoric correlation (Revelle, 2019). To assess the relationship between the number of principles (total and per dimension) in the videos and the views and likes obtained, Spearman bivariate correlations were performed.
As the number of days between the publication date of the videos and the recording of the number of likes and views per video varies, Spearman correlations between the number of principles and dimensions, and the publication date of the videos were performed. For significant Spearman correlations, Bayesian correlations were performed to assess the robustness of the effects obtained. Bayes factors were interpreted according to the classification of Lee and Wagenmakers (2013).
To check whether it was the fostering generative processing dimension that mediated the significant relationship between the total number of principles of multimedia learning present in the videos and the number of likes, we performed a mediation analysis with 1000 bootstrapped samples.Analyses were conducted using R (Team, 2022) to generate the linear models and JASP (Team, 2023) for Bayesian analyses.
Results
Application of the multimedia learning principles in the short videos
The average number of the principles found in the 34 short videos can be seen in the violin graph on the left in Figure 1, where the black square is the mean, and the black line is the median. The distribution of short videos according to the number of principles can be seen in the graph on the right in Figure 1.
A visual analysis of the graphs in Figure 1 shows an average of 7.9 principles per short video (SD=1.675) and a registered variation of 5 to 10 principles. Interestingly, no short video adopted less than 5 of the 10 principles included in the analysis framework, indicating that the minimum quality according to the principles of multimedia learning was 50%. Looking at the distribution, it is possible to see that 64.7% (n=22) of the 34 short videos included 8 or more principles (Md=8) and that 50% of the videos complied with between 6 and 9 principles (see box plot).
Considering that the above results refer to a total analysis, the frequencies of the multimedia learning principles are now presented individually. Figure 2 shows the number of short videos that include each of the principles. The least observed principle in the analysed videos is coherence, which is present in only 29.4% (n=10) of the videos, and the most observed principle is signalling, which is present in all the analysed videos. It should be noted that coherence is the only principle that emerges in less than 50% of the videos and is most responsible for reducing the overall value of compliance with the principles of multimedia learning. Additionally, the personalisation principle is also noteworthy, appearing in only 58.8% (n=20) of the videos. Regarding the dimensions of the multimedia learning principles, the short videos compliance with these dimensions can be found in Table 2. Before analysing it, it is noteworthy to highlight the dimension does not include the same number of principles. There are 3, 2 and 5 principles respectively for reducing extraneous processing, managing essential processing, and fostering generative processing. The analysis of Table 2 reveals that the reducing extraneous processing dimension has the fewest videos containing all principles (only 29% of the videos) and the managing essential processing dimension has the most (82.4% of the videos). This result is not surprising, since the least considered principle is the coherence principle, which pertains to the reducing extraneous processing dimension.
Principles of multimedia learning and engagement metrics
To assess the relationship between the number of principles of multimedia learning (total and per dimension) in the short videos and the number of views and likes obtained, Spearman correlations were carried out. The correlation matrix resulting from this analysis is shown in Figure 3.
As we can see in Figure 3, significant correlations were only registered between the total number of principles found in the short videos and the presence of likes they received (r=.345, p=.046), and between the number of principles for the fostering generative processing dimension and the presence of likes they received (r=.360, p=.037). These two significant correlations, which are positive, indicate that the more principles of multimedia learning a video contains, the more likes it receives. This result is strengthened by the fact that there is no relationship between the total number of principles or the fostering generative processing with the publication date of the videos (all p>.220). It should also be noted that no significant correlations with the presence of views were found. Given the p values close to 0.05 in the two significant correlations, Bayesian correlations were performed to assess the robustness of the observed effect (Figure 3).
The Bayes factor obtained for the fostering generative processing dimension (FGP)*Likes correlation is 3.141, while the Bayes factor obtained for the total number of principles (NMP)*Likes correlation is 1.839. According to Lee and Wagenmakers (2013) classification, the Bayes factor for the correlation between the fostering generative processing dimension and the presence of likes has moderate evidence, while the Bayes factor for the correlation between the total number of principles presented in the short videos and the presence of likes has anecdotal evidence.
Given that fostering generative processing dimension is included in total principles, and the impact of its correlation with likes is larger than the correlation of total principles with likes, and none of the other dimensions show a significant correlation with likes, we would expect the significant correlation of total principles with likes to be driven by the impact of the fostering generative processing dimension.
To test whether it was the fostering generative processing dimension that mediated the significant relationship between the total number of principles presented in the short videos and the presence of likes, we performed a mediation analysis with 1000 bootstrapped samples. Table 3 presents the results of this mediation. The results show that although there is no significant mediation effect (a×b=1.315, p=.189), the direct relationship between the total number of principles presented in the short videos and the presence of likes is not significant (c=-0.248, p=.804), and only becomes significant when the fostering generative processing dimension is present (c'=2.150, p=.032). In addition, 82.97% of the total relationship is explained by the fostering generative processing dimension. The results of this analysis show that the principles for fostering generative processing explain the relationship between the total number of principles and the presence of likes.
Discussion and conclusion
The aim of this study was to 1) to analyse short videos for the teaching and learning of Portuguese as a foreign language using an adaptation of Mayer’s multimedia learning principles and 2) to understand the potential relationships among the principles and the number of views and likes of the videos.
Results show that the principles of multimedia learning are implemented well in the short videos. Based on the analysis of the three dimensions, we found that the highest application of the managing essential processing dimension is due to the high implementation of the segmenting and pre-training principle, as most of the videos are kept to less than one minute and with key content introductions. The lower application of the fostering generative processing dimension is influenced by the fact that the personalisation principle is not well implemented, as the instructors in some videos do not narrate in the first and second person.
The reducing extraneous processing dimension is the one that covers the least number of short videos, as the principle of coherence is applied minimally. This is because many videos have background music and coloured backgrounds, which is in line with what was previously found on short videos for English learning in TikTok (Zhang et al., 2022b). In these short videos, background music is used to evoke emotion and sound effects are used to add an element of humour. Only a small percentage of the short videos exhibited a plain background.
It should be noted that the criteria for evaluating the application of principles of multimedia learning to short videos were set in line with Mayer’s investigation, where background music is present in a way that affects learning outcomes. However, for FLL, there are positive findings that support further research into real-world-based musical materials to facilitate daily cognitive tasks (Kang & Williamson, 2014). This is because FLL outcomes are better when music is integrated, whether in the background, through songs, or during musical and rhythmic activities, than when no music or other artistic interventions are used (Degrave, 2019). In general, however, the role of background music in learning is not clearly established (De-la-Mora-Velasco & Hirumi, 2020). The same is true for FLL, as for some skills there is published data or conflicting evidence between studies (Degrave, 2019).
Thus, since the effects of using background music in FLL are not fully known, we can understand that this may be the reason why the coherence principle is not well applied in the short videos analysed. In parallel, TikTok is initially a music-based platform, therefore, the sound and background music are likely to influence whether the video is popular or not (Klug et al., 2021). This may also be another reason why the creators decided to add background music to their videos.
Results regarding the relationship between the number of principles of multimedia learning (total and per dimension) in the short videos and the number of views and likes obtained, showed the correlation of the number of principles is only apparent with likes. This result is not very surprising, as mentioned before, views can be associated with likes, shares, specific release times and popular video sections (Klug et al., 2021). However, likes can be interpreted in relationship with the perceived value of the video (Shoufan, 2019). This means that views were shown to be independent of the effectiveness of the video, and given that they can be influenced by various factors, whereas likes can directly represent the viewer's perception of the video content, this could explain why the number of principles applied to the video did not correlate with views but did correlate with likes. This also means that our results provide preliminary evidence that, even for videos of very short duration, the number of likes is more related to the quality of the video than the number of views.
Results also show that the principles for fostering generative processing dimension explain the relationship between the total number of principles and the presence of likes. Mayer’s research on instructional design principles initially focused on techniques to reduce extraneous processing and gradually expanded to include techniques to manage essential processing and foster generative processing (Mayer, 2020). A major challenge that has surrounded this design process is the limited cognitive capacity of learners (Mayer, 2020). Therefore, when focusing on instructional techniques to foster generative processing, researchers are challenged to consider more fully the role of motivation in multimedia learning (Fiorella & Mayer, 2016). This occurs because motivation is key to engaging learners in selecting, organising, and integrating knowledge, and it is the learners' motivation that leads to their generative cognitive processing (Mayer, 2020).
Thus, in this context of learner motivation as a driver of cognitive processing, the descriptions of the principles of multimedia learning have been gradually adapted. For example, in a second edition of the principles, Mayer emphasised that “we found no evidence that animation was more effective than static diagrams in multimedia lessons” (Mayer, 2009: 230). Additionally, regarding the image principle, he emphasises that “people do not necessarily learn better when the speaker’s image is added to the screen” (Mayer, 2009: 242). Later, in a third edition, the image principle was updated to state that “people do not learn better from multimedia presentations when a static image of the instructor is added to the screen” (Mayer, 2020: 331). Also, the embodiment principle was added to highlight the importance of high embodiment of the instructor on the screen, because high embodiment can serve as a positive social cue that motivates learners to work harder to understand the instructional information. In this context, based on the importance of fostering generative processing dimension, synthesising the details of the embodiment principle, such as dynamic mapping, gaze guidance, and perspective into five ways specifically improves the effectiveness of instructional videos by combining the generative activity principle and subtitle principle in FLL videos.
In summary, we can understand that the fostering generative processing dimension is considered more correlated with the effectiveness of video than the other dimensions, because it is generated by the learner's motivation. Similarly, the number of likes is also generated based on the viewer's perception of the video content, which represents the value, acceptance, and popularity of the video by the viewer and is fully associated with the learner's motivation to watch it. Thus, at the level of learning motivation, it can be explained that the fostering generative processing dimension is more correlated with the presence of likes than other dimensions. This also means that our results provide preliminary evidence that, in the multimedia learning principle, the fostering generative processing dimension reflects the quality of the user experience of the short video.
Finally, considering that the object of analysis in this article was generated based on our search on TikTok, we cannot deny that these 34 videos may not constitute all the videos related to learning Portuguese on TikTok made by native speakers, and that the number of videos may have an impact on the results of applying the principles of multimedia learning to short videos. The corpus could be expanded in several possible ways, such as by extending the analysis to other languages or by comparing videos from different platforms (e.g., Instagram and YouTube). It is also worth noting that the analysis in this study focused on video design quality and user experience quality but not on video content quality. This will be one of the directions for future research. Another aspect to be taken into consideration in future studies is the use of cross-sectional or longitudinal methods for established time periods analysis. However, what we can determine based on our data analysis is that short videos for learning Portuguese on TikTok conform to the construction of principles of multimedia learning and have full potential to become a specific learning resource for this era. The design quality of these videos, i.e., the application of multimedia learning principles, correlates with the number of likes. In addition, the fostering generative processing dimension reflects the quality of the user experience of the short video. It has a prominent role in facilitating the dissemination and effectiveness of short videos, which stimulates consideration about the design of online short videos for learning foreign languages, as much as having the potential to inform the design of educational materials and video-based language-learning programmes. (1)