Measuring Artificial Intelligence Literacy in Higher Education: Validation of the ECAE-AI Scale
DOI:
https://doi.org/10.5281/zenodo.23192927Keywords:
AI Literacy, Attitudes toward AI, Confirmatory Factor Analysis, Digital Ethics, Higher Education, Psychometric Validation.Abstract
The accelerated integration of artificial intelligence (AI) into higher education has increased the need for valid instruments capable of assessing AI literacy beyond technical skills, including attitudinal and ethical dimensions. Despite growing international interest in AI literacy, validated instruments adapted to Latin American undergraduate contexts remain scarce, limiting evidence-based educational interventions and institutional AI policies. This study aimed to validate the factorial structure of the ECAE-AI scale, a 14-item instrument designed to assess knowledge, attitudes, and ethical awareness regarding AI use among university students. A non-experimental, cross-sectional, instrumental design was applied to 408 undergraduate students from a northeastern Mexican university. Confirmatory factor analysis (CFA) using the DWLS estimator confirmed an excellent fit for the three-factor correlated model (CFI = .997; TLI = .996; RMSEA = .057; SRMR = .048). Standardised factor loadings ranged from .607 to .894. Reliability was excellent at both the global (? = .915; ? = .917) and dimensional levels. Convergent and discriminant validity were also supported. The findings support the ECAE-AI as a psychometrically robust instrument for assessing AI literacy in higher education, integrating cognitive, attitudinal, and ethical dimensions within a single framework. The scale provides practical value for diagnosing AI literacy profiles, designing evidence-based educational interventions, and supporting responsible AI integration policies in higher education institutions.
References
Abdallah, N., Katmah, R., Khalaf, K., & Jelinek, H. F. (2025). Systematic review of ChatGPT in higher education: Navigating impact on learning, wellbeing, and collaboration. Social Sciences & Humanities Open, 12, 101866. https://doi.org/10.1016/j.ssaho.2025.101866
Acosta-Enriquez, B. G., Arbulú Ballesteros, M. A., Arbulu Perez Vargas, C. G., Orellana Ulloa, M. N., Gutiérrez Ulloa, C. R., Pizarro Romero, J. M., Gutiérrez Jaramillo, N. D., et al. (2024). Knowledge, attitudes, and perceived Ethics regarding the use of ChatGPT among generation Z university students. International Journal for Educational Integrity, 20(1), 10. https://doi.org/10.1007/s40979-024-00157-4
Almaraz-López, C., Almaraz-Menéndez, F., & López-Esteban, C. (2023). Comparative Study of the Attitudes and Perceptions of University Students in Business Administration and Management and in Education toward Artificial Intelligence. Education Sciences, 13(6), 609. https://doi.org/10.3390/educsci13060609
Amiri, H., Peiravi, S., rezazadeh shojaee, S. s., Rouhparvarzamin, M., Nateghi, M. N., Etemadi, M. H., ShojaeiBaghini, M., et al. (2024). Medical, dental, and nursing students’ attitudes and knowledge towards artificial intelligence: a systematic review and meta-analysis. BMC Medical Education, 24(1), 412. https://doi.org/10.1186/s12909-024-05406-1
Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103(3), 411–423. https://doi.org/10.1037/0033-2909.103.3.411
Brown, T. A. (2015). Confirmatory Factor Analysis for Applied Research (2nd ed.). The Guilford Press.
Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399–412. https://doi.org/10.1111/bjop.12046
Ferrés, J., & Piscitelli, A. (2012). Media competence. Articulated proposal of dimensions and indicators [La competencia mediática: propuesta articulada de dimensiones e indicadores]. Comunicar, 38, 75–82. https://doi.org/10.3916/C38-2012-02-08
Fornell, C., & Larcker, D. F. (1981). Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.1177/002224378101800104
Fuchs, K., & Aguilos, V. (2023). Integrating Artificial Intelligence in Higher Education: Empirical Insights from Students about Using ChatGPT. International Journal of Information and Education Technology, 13(9), 1365–1371. https://doi.org/10.18178/ijiet.2023.13.9.1939
García Peñalvo, F. J., Llorens-Largo, F., & Vidal, J. (2024). La nueva realidad de la educación ante los avances de la inteligencia artificial generativa. RIED-Revista Iberoamericana de Educación a Distancia, 27(1), 9–39. https://doi.org/10.5944/ried.27.1.37716
Gibson, D., Kovanovic, V., Ifenthaler, D., Dexter, S., & Feng, S. (2023). Learning theories for artificial intelligence promoting learning processes. British Journal of Educational Technology, 54(5), 1125–1146. https://doi.org/10.1111/bjet.13341
Girarte-Guillén, J. L., Hernández-Marín, M. R., & Hilt, J. A. (2025). Design and validation of an instrument to measure digital Competencies in artificial intelligence among Higher education students. Unaciencia, Revista de Estudios e Investigaciones, 18(35), 4–24. https://doi.org/10.35997/unaciencia.v18i35.1254
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate Data Analysis (8th ed.). Cengage.
Hu, L. t., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
?pek, Z. H., Gözüm, A. I. C., Papadakis, S., & Kallogiannakis, M. (2023). Educational Applications of the ChatGPT AI System: A Systematic Review Research. Educational Process: International Journal, 12(3), 26–55. https://doi.org/10.22521/edupij.2023.123.2
Kline, R. B. (2016). Principles and Practice of Structural Equation Modeling (4th ed.). The Guilford Press.
Kong, S. C., & Zhu, J. (2025). Developing and validating an artificial intelligence ethical awareness scale for secondary and university students: Cultivating ethical awareness through problem-solving with artificial intelligence tools. Computers and Education: Artificial Intelligence, 9, 100447. https://doi.org/10.1016/j.caeai.2025.100447
Laupichler, M. C., Aster, A., Meyerheim, M., Raupach, T., & Mergen, M. (2024). Medical students’ AI literacy and attitudes towards AI: a cross-sectional two-center study using pre-validated assessment instruments. BMC Medical Education, 24(1), 401. https://doi.org/10.1186/s12909-024-05400-7
Li, C.-H. (2016). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods, 48(3), 936–949. https://doi.org/10.3758/s13428-015-0619-7
Lo, C. K., Hew, K. F., & Jong, M. S.-y. (2024). The influence of ChatGPT on student engagement: A systematic review and future research agenda. Computers & Education, 219, 105100. https://doi.org/10.1016/j.compedu.2024.105100
León, OG, & Montero, I. (2020). Research methods in psychology and education: The quantitative and qualitative traditions (4th rev. ed.). McGraw-Hill.
Ma, D., Akram, H., & Chen, I. H. (2024). Artificial Intelligence in Higher Education: A Cross-Cultural Examination of Students’ Behavioral Intentions and Attitudes. The International Review of Research in Open and Distributed Learning, 25(3), 134–157. https://doi.org/10.19173/irrodl.v25i3.7703
Mora, Y. Y., & Rizzo, J. M. A. (2023). Influencia de la inteligencia artificial en los estudiantes universitarios. South Florida Journal of Development, 4(10), 3749–3762. https://doi.org/10.46932/sfjdv4n10-001
Ng, D. T. K., Wu, W., Leung, J. K. L., Chiu, T. K. F., & Chu, S. K. W. (2023). Design and validation of the AI literacy questionnaire: The affective, behavioural, cognitive and ethical approach. British Journal of Educational Technology, 55(3), 1082–1104. https://doi.org/10.1111/bjet.13411
UNESCO. (2021). Recommendation on the Ethics of Artificial Intelligence. UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000380455
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Comunicar
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
