Background and importance

The COVID-19 pandemic has put tremendous pressure and burden on the medical system and has negatively impacted economic development on a global scale (Nicola et al., 2020). Despite vaccinations, the number of newly confirmed cases continues to rise rapidly in many countries (Gardner, 2020; WHO, 2021). SARS-CoV-2 can survive on aerosols and various surfaces and use those as transmission routes (Santarpia et al., 2020). Moreover, an infected person can spread the disease with or without symptoms (Liu et al., 2020). However, as long as people follow the standard cleaning procedures, wear masks, and maintain social distance, the risk of contracting the disease via exposure to various surfaces, the air, or other people is low (Pitol & Julian, 2021); therefore, these three measures are the best ways to protect oneself against COVID-19 (Chamola et al., 2020; WHO, 2021). A study has found that the combined effect of adhering to all three measures is more powerful than using one measure alone (Álvarez-Pomar & Rojas-Galeano, 2021). The challenge for health educators, then, is to promote the implementation of these correct protection measures to prevent COVID-19. In 2020, distance education became ubiquitous as a result of the pandemic (Kim, 2020). The increasing popularity of e-learning and internet-based applications has greatly expanded the possibilities for learning, and the use of these innovative media has been trending upward (Chamola et al., 2020). Multimedia, including text, image, video, and audio (TIVA), has been widely used for health education on COVID-19 prevention websites (CDC, 2020; WHO, 2021) because TIVA-based materials are quick to produce, easy to use, and inexpensive. However, there are still many disadvantages in using TIVA for health education (Nayef, 2015). For example, it only delivers one-way information transmission, provides less diverse and uninteresting content, and it is unlikely to motivate the user to actively seek information (Garris et al., 2002). Additionally, it is difficult to provide scenario simulations in such a learning environment, which yields a lower learning retention rate and offers fewer opportunities for putting learning into practice (Brown et al., 1989). The difficulty in conducting assessment and diagnosis and the inability to correct the learning process are also significant barriers (Blackburn & Hakel, 2006). Thus, it is a challenge to design self-learning material to compensate for and overcome the limitations of TIVA and to improve the outcome of learning about self-protection against the pandemic. The use of a serious game, involving both entertainment and education purposes, is a useful model to address the aforementioned issues and has been applied to COVID-19 prevention through the quiz format to assess performance (Gaspar et al., 2020; Suppan et al., 2020).

Study aims

The current study developed an interactive scenario-based web serious game (hereafter, WSG-COVID-19.SP). The purpose of the game is for the users to learn about the self-protection measures against COVID-19. WSG-COVID-19.SP adopts four learning mechanisms: game scenarios for increasing learning motivation, simulated practice to build self-protection skills, diagnostic feedback to increase understanding, and portfolio analysis to identify misconceptions and problematic behaviors in COVID-19 prevention. WSG-COVID-19.SP is expected to motivate users to implement self-protection measures. It is also expected to be useful for preventing diseases with similar transmission models. Therefore, the content validity of the game was evaluated by involving healthcare workers. Usability testing was done to determine the degree to which the game enabled the users to learn about the self-protection measures and use them continuously. The research questions are as follows:

• Are the contents of WSG-COVID-19.SP valid and satisfactory?

• Do users' college major (information technology-related or non-related) and gender affect their perceived system usability?

• Can WSG-COVID-19.SP identify user misconceptions and problematic behaviors in COVID-19 prevention by analyzing the learning records?

Game-based learning

Motivation is a critical factor for effective learning. The element of uncertainty in games can increase and maintain the participation of players (Kim, 2020) and arouse their psychological need for quickly overcoming the challenges and reaching the goals (Freitas, 2018). As an instructional method, games are a valuable tool for enhancing learning outcomes. When learners play an enjoyable game, they often assimilate the game’s viewpoints. Therefore, the effectiveness of digital game-based learning (GBL) is a proven strategy to increase learning interest and motivation in a myriad of subject areas including mathematics, business, computer science, and language (Park et al., 2019). Using GBL as the foundation, serious games (SG) possess the entertainment characteristics of gameplay but are used for learning or training purposes (Wattanasoontorn et al., 2013). SG has also been applied to many domains for knowledge building, such as conducting efficient training in the field of health, e.g., using SGs with elderly people for disease prevention and rehabilitation (Wiemeyer & Kliem, 2012).

As a response to COVID-19, Suppan et al. (2020) developed "Escape COVID-19," a serious game that promotes safe practices for healthcare workers. The game uses single choice, multiple choices, and drag-and-drop test items, along with the corresponding feedback after answering. Similarly, Gaspar et al. (2020) developed a mobile serious game that uses a quiz format to assess user performance and cartoon-style cards to enhance motivation. They used Google Analytics to analyze the statistics of the game use. Their purpose was to educate and reach out to young players with scientific information about personal care relating to COVID-19 prevention. However, such an interactive scenario and practice experience may be insufficient if a game interacts with the players through choice questions only.

Situated learning theory and scenario-based learning

Multimedia materials (e.g., video, audio, and animation) have been widely adopted in teaching and training. They facilitate self-learning effectively because learners tend to find the content attractive (Chang et al., 2010). However, to offer learners an authentic learning experience, multimedia alone is insufficient (Kinshuk et al., 2016). Situated learning theory is a constructivist cognition theory that emphasizes learners’ need for an authentic learning context (Brown et al., 1989). The theory proposes that learners should be placed directly in an authentic scenario, a manually set up simulation setting, or a digitally produced virtual simulation scenario so that they can learn the targeted skills by continuously interacting with people in the different scenarios. Scenario-based learning (SBL) follows the principles of situated learning theory (Naidu et al., 2007) and it uses interactive scenarios to enable learners to demonstrate their decision-making process. With technology and a sound framework, SBL can motivate students to take the initiative in their learning and provide them with an instructional environment to safely construct and solve real-world problems (Vlachopoulos et al., 2017). For example, Richardson et al. (2017) applied the SBL approach to assist nursing and midwifery students in acquiring the appropriate attitudes and knowledge related to sustainability and climate change. Similarly, Torkshavand et al. (2020) adopted simulation-based learning to facilitate students’ knowledge and skills for elderly patient care.

Learning portfolio analysis

A learning portfolio is a collection of a learner’s work, organized in a way that supports the instructional process and evaluation (Alzouebi, 2020). It can enhance self-regulation and goal-oriented motivation (Šliogeriene, 2016) and effectively document and manage the learning processes and analyze them in real time. For instructors, learning portfolios can help identify the blind spots in their instruction, while for learners, comprehensive and detailed documentation of each step in the learning process provides an opportunity for reflection (Kim, 2020). Learning portfolios can be used, for example, to assess software operating skills (Su & Lin, 2015), investigate the differences in learner behavior and preferences by employing multi-platform learning analytics (Ruipérez-Valiente et al., 2020), or personalize learning with adaptive scaffolding (Su, 2020). In this study, the users’ learning and operational portfolios were analyzed to identify their learning problems to facilitate self-learning.

Theoretical basis

Rooted in situated learning theory and employing interactive scenario mechanisms, WSG-COVID-19.SP (Figure 1) is designed according to the following propositions. 1) An interactive scenario is used to increase the users’ motivation to revisit the TIVA-based information and be repeatedly tested using the simulated assessment to help them understand their weaknesses (Bardach et al., 2020). 2) Simulated practices (Torkshavand et al., 2020) can improve users’ knowledge about COVID-19 prevention and their ability to apply that knowledge in their daily lives. 3) Diagnostic feedback (Su, 2020) can enhance users’ understanding of epidemic prevention information and the necessary self-protection measures. WSG-COVID-19.SP has the functionality of storing all operational and assessment records, allowing the users to learn from the system-generated evaluations.

Concepts and assessment indicators of self-protection against COVID-19

The study designed the learning activities and assessment indicators (AI) (Table 1) in WSG-COVID-19.SP based on the information about the necessary self-protection measures against COVID-19 provided by WHO (2020) and CDC (2020). One AI example is 1.1.a, which deals with the concept of “face masks,” including “how to choose” the appropriate mask. The corresponding learning indicator is “surgical mask.” All AIs were evaluated and confirmed by medical professionals in fields such as infectious disease control, public health, family medicine, and nursing.

Definitions of scenario tasks in WSG-COVID-19.SP

There are four types of scenario tasks in the game, which are designed to simulate the knowledge and behaviors related to infection prevention that should be put into practice in daily life. The tasks are defined as follows. 1) Required task: the primary task related to epidemic prevention. 2) Sudden task: a sudden event that may occur in daily life. 3) Optional task: a task that is related to the required task but is optional. 4) Pass task: the task that determines the outcome of the game. Each task is associated with the prevention-related concepts defined in Table 1. The scenarios in WSG-COVID-19.SP were constructed as shown in Figure 2.I. The required task is the core of each game scenario and is supported by the sudden and optional tasks. The performance on these three tasks determines the result of the pass task and the user’s success in completing the game level.

Design and components of WSG-COVID-19.SP

The design of WSG-COVID-19.SP consists of six steps (Figure 2.II), each with a specific objective. Steps 1 and 2 create the core game scenario based on the scenario tasks (a required task, a sudden task, and an optional task as defined in Section 3.3). The pass task uses the “forehead temperature” as the primary condition for passing the level. The tasks in the scenarios are all connected to the essential epidemic prevention concepts to assess whether users can complete them by properly applying their epidemic prevention knowledge. The pass task evaluates users’ understanding of epidemic prevention, and the score is represented by the forehead temperature bar on the screen (see, for example, Figure 3.I.c). During the game, all responses to the scenario tasks are documented by the system to enable the analysis of conceptual understanding and behavior.

Subsequently, Step 3 takes the pass tasks from Steps 1 and 2 and determines the outcome of the game. Then, Step 4 conducts an automatic analysis of the system’s record of user responses through the diagnostic feedback mechanism. The system then produces a report on the accuracy of the responses in the epidemic prevention scenarios to help users understand their learning status. In Step 5, after the game scenarios are completed, the users are provided with a multiple-choice test to assess their epidemic prevention knowledge. In essence, their conceptual understanding is evaluated again using a different format.

Furthermore, a report is also provided for this second test to help the users reflect on their learning. They can examine their misunderstandings and reflect on their COVID-19 prevention knowledge based on the two personalized diagnostic results (the game report and the test report). They may choose to go through WSG-COVID-19.SP again to correct their previous conceptual and behavioral mistakes. In short, a scenario-based game with a variety of tasks can motivate users to self-learn and self-assess repeatedly to correct any misconceptions and problematic behaviors by re-playing the game. This cyclical self-learning model of “playing, assessment, diagnostic feedback, understanding the problem, practice, and correction” can help users implement COVID-19 prevention and self-protection measures more effectively.

Participants

Usability testing tool

The users completed a self-administered questionnaire on a mobile device to provide feedback on their experience with the system. The questionnaire was designed based on the Technology Acceptance Model (TAM) (Davis, 1989; Venkatesh & Bala, 2008). It investigates the users’ behavioral intention to use WSG-COVID-19.SP and the feasibility of promoting this game. The study also examined two external variables—users’ perceived comparison and diagnosis (Liaw, 2008)—to determine the effect of the scenarios. Additionally, we assessed the cognitive load that the system placed on the users (Sweller et al., 1998).

Table 2 defines each item in the questionnaire. The questionnaire contains eight measurement scales and uses a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). The internal consistency of the questionnaire was reliable (Cronbach’s alpha=0.903).

Learning process in WSG-COVID-19.SP

Scenarios and learning tasks in WSG-COVID-19.SP

Five scenarios (a hospital, public park, classroom, bookstore, and restaurant) and one additional sudden task were designed for Day 1 (Figure 3.II) and Day 2 (Figure 4). Supplies such as masks and alcohol-based sanitizers are shown on the screen. Users use their mobile devices to interact with the virtual people or events in the scenarios to complete the tasks.

In the process, they need to apply their TIVA-based knowledge through critical thinking and problem-solving skills to deal with the problems and situations they encounter. They can protect themselves by carrying out appropriate actions to prevent infection. To attract and maintain user attention, the game scenarios are designed such that both the difficulty and the fun increase as the challenge level goes up. Users must perform more actions in the task to achieve the learning objectives. The self-protection measures are repeatedly shown in various scenarios to ensure deeper learning. If the users do not demonstrate the correct self-protection behaviors in the game, they are at risk of infection.

DAY 1 scenario

For Day 1 of the game (Figure 3.II), users complete up to two actions for each task to achieve the learning objectives for self-protection.

• In the hospital scenario, the learning objectives include “taking the forehead temperature” and “wearing a mask” when visiting patients in the hospital.

• In the public park scenario, the learning objectives are “how” and “when” to use alcohol-based sanitizer to disinfect the hands before touching the eyes and nose.

• In the classroom scenario, the learning objectives are “taking the forehead temperature” and “how to wash your hands" before eating.

Personalized diagnostic report

After users complete the interactive simulation scenarios, the diagnostic feedback mechanism in WSG-COVID-19.SP automatically analyzes their learning portfolios to provide a personalized diagnostic report, which enables them to understand their misconceptions and incorrect practices. Based on the diagnostic results, the system then presents one of three possible outcomes: safe (low risk), moderate risk, and high risk (Figure 5.a).

The diagnostic report of performance (Figure 5.b) provides individualized results based on the items listed in Table 1 to help users understand their errors. Additionally, the game system uses choice questions to test their basic infection prevention knowledge (Figure 5.c), followed by a report of the test results (Figure 5.d). Consequently, users are motivated to further pursue the relevant knowledge and re-test themselves. As such, this model can promote users’ COVID-19 prevention knowledge and self-protection measures.

Results

Usability

This section presents the results of the website questionnaire on the usability of the system. We examined the item mean differences between the users in information technology (IT)-related majors and those in non-related ones, and between males and females (see Table 3).

The usability of WSG-COVID-19.SP was rated by all 71 users. The results show that the users were interested in the game and perceived enjoyment while playing (Mean±SD: 4.16±0.77). Furthermore, they rated the system as providing a better situated experience, and better conceptual learning and assessment (4.28±0.74) compared with TIVA-only materials. Because WSG-COVID-19.SP can automatically generate a diagnostic report at the end of a learning session, the users rated the diagnostic feedback mechanism (4.28±0.74) as highly useful.

WSG-COVID-19.SP was designed with the following functions in mind: game-based enjoyment, effective assessment and learning, and a meaningful diagnostic report. As expected, the users perceived the game system as useful (4.33±0.67), highly satisfactory (4.35±0.65), and easy to use (4.09±0.79). They also indicated a high intention to use it in the future (4.28±0.68). Moreover, except for cognitive load, all indicators in the usability test were rated at least 4.0 out of 5, and the scores were not affected by user background. While cognitive load was rated as moderate (2.92±1.10), it is worth noting that the users in an IT-related major experienced a significantly lower cognitive load (2.26±1.13) than those in a non-related major (3.18±1.03). Thus, “user major” is an independent variable that affects cognitive load. However, the same cannot be said of “gender” because while females appeared to have a lower cognitive load (2.7±0.89) than males (3.08±1.23), the difference was not statistically significant.

Limitations

This research is still in the developing stage for the serious game WSG-COVID-19.SP for learning self-protection skills against COVID-19. The game’s content validity and usability are continuously being tested. Note that the results presented in this paper do not confirm the effectiveness of this game in improving actual real-world self-protection or successful application to other populations.

The sample used for the usability test was a small number of college students. If the game were to be promoted in hospitals and used as an assistive device for healthcare professionals in health education, potential hospital users including the general public and clinic outpatients should be included in future research samples.