AccScience Publishing / IJOSI / Online First / DOI: 10.6977/IJoSI.2026XX_XX(X).026400352
Cite this article
3
Download
13
Views
Related Info Links
More by Authors Links
Journal Browser
Volume | Year
Issue
Search
News and Announcements
View All
ARTICLE

Apparent predictability of learning-oriented participation from behavioral and contextual student data: Near-duplicate leakage and the roles of discussion participation and motivation

Shu Zhang1 ,  Anuar Ahmad1* ,  Jamsari Alias2,3
Show Less
1 Faculty of Education, National University of Malaysia, Bangi , Malaysia
2 Faculty of Liberal Science, National University of Malaysia, Bangi , Malaysia
3 Institute Islam Hadhari, National University of Malaysia, Bangi , Malaysia
Received: 28 September 2026 | Revised: 23 October 2026 | Accepted: 29 October 2026 | Published online: 17 November 2026
(This article belongs to the Special Issue Artificial Intelligence and Educational Innovation)
© 2026 by the Author(s). This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution -Noncommercial 4.0 International License (CC-by the license) ( https://creativecommons.org/licenses/by-nc/4.0/ )
Abstract

Learning analytics increasingly predicts student participation from behavioral and contextual data, but performance can be overstated when outcome components enter the predictors, related records appear across cross-validation splits, or uncertainty is ignored. This study examined whether learning-oriented participation can be predicted from a publicly available dataset and whether discussion participation and motivation add incremental information. Of 14,003 records (16 variables), 1,534 were exact duplicates that could not plausibly arise by chance (expected coincidental pairs 3.8 × 10−5). Of the 12,469 unique records, 10,701 (85.8%) belonged to multi-record near-duplicate families, within which only Resources, Motivation, Extracurricular, and Internet. A formative Learning-Oriented Participation Index (LOPI) averaged percentile-ranked Attendance, AssignmentCompletion, and OnlineCourses. Elastic Net, Extra Trees, and CatBoost were evaluated with nested five-fold cross-validation, keeping each family within one fold. Under row-wise validation, Extra Trees appeared to reduce mean absolute error (MAE) by 0.305 points, and Discussions appeared informative. Under family-grouped validation, no model outperformed the mean baseline (Extra Trees MAE = 13.663, R2 = −0.011; baseline: 13.605), the Discussions gain disappeared (ΔMAE = −0.017, 95% confidence interval = −0.035 to 0.002), Motivation added no stable information, the Discussions × Motivation interaction was unsupported, and lowest-quartile identification was at chance (area under the receiver operating characteristic curve: 0.46). Split-conformal intervals achieved near-nominal coverage (89.2% and 94.5% for nominal 90% and 95%) but spanned 55.1 and 64.3 LOPI points. The apparent predictability reflected near-duplicate leakage; public educational datasets should be audited for duplicate structure and evaluated with grouped validation before interpretation.

Keywords
Learning analytics
Educational data mining
Learning-oriented participation
Discussion participation
Explainable machine learning
Predictive uncertainty
Data leakage
Grouped cross-validation
Funding
None.
Conflict of interest
The authors declare they have no competing interests.
References

Abukader, A., Alzu'bi, A., & Adegboye, O. R. (2025). Intelligent system for student performance prediction: An educational data mining approach using metaheuristic-optimized LightGBM with SHAP-based learning analytics. Applied Sciences, 15, 10875. https://doi.org/10.3390/app152010875

 

Akpabio, I. I., & Savari, S. A. (2021). Uncertainty quantification of machine learning models: On conformal prediction. Journal of Micro/Nanopatterning, Materials, and Metrology, 20(4), 041206. https://doi.org/10.1117/1.JMM.20.4.041206

 

Arada, B. (2025). Level of social presence, social interaction, collaborative learning, and satisfaction in a BAMS course Discord server. International Journal on Open and Distance e-Learning, 11(1). https://doi.org/10.58887/ijodel.v11i1.294

 

Azuela, L., Azuela, H. P., Felipe, A. L., Aragoncillo, A., & Eleazar, M. (2025). From data to action: Narrative benchmarking and comparative evaluation of explainable AI-driven supervised machine learning models for academic risk prediction. In 2025 IEEE International Conference on Emerging Trends in Engineering and Computing (ETECOM). IEEE. https://doi.org/10.1109/etecom66111.2025.11319004

 

Choi, W. C., Lam, C.-T., & Mendes, A. J. (2024). Analyzing the interpretability of machine learning prediction on student performance using SHapley additive exPlanations. In Proceedings of TALE 2024. IEEE. https://doi.org/10.1109/tale62452.2024.10834292

 

Deutschmann, N., & Rigotti, M. (2023). Adaptive conformal regression with Jackknife+ rescaled scores. arXiv. https://doi.org/10.48550/arXiv.2305.19901

 

Diamant, N., Hajiramezanali, E., Biancalani, T., & Scalia, G. (2023). Conformalized deep splines for optimal and efficient prediction sets. arXiv. https://doi.org/10.48550/arxiv.2311.00774

 

Diamantopoulos, A., & Winklhofer, H. M. (2001). Index construction with formative indicators: An alternative to scale development. Journal of Marketing Research, 38(2), 269–277. https://doi.org/10.1509/jmkr.38.2.269.18845

 

Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53, 109–132. https://doi.org/10.1146/annurev.psych.53.100901.135153

 

Fisher, A., Rudin, C., & Dominici, F. (2019). All models are wrong, but many are useful: Learning a variable's importance by studying an entire class of prediction models simultaneously. Journal of Machine Learning Research, 20, 177.

 

Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59–109. https://doi.org/10.3102/00346543074001059

 

George, E. B., Senthilkumar, R., Al-Junaibi, F., & Al-Shuaibi, Z. (2025). Explainable AI methods for predicting student grades and improving academic success. Journal of Information Systems Engineering and Management, 10. https://doi.org/10.52783/jisem.v10i23s.3680

 

Getenet, S., Tualaulelei, E., Singh, N., Pillay, Y., & Hammermeister, K.-A. (2026). Interactive technologies and student engagement in online higher education: A multidimensional analysis. Technology, Pedagogy and Education, 35(1), 83–101. https://doi.org/10.1080/1475939X.2025.2498483

 

Gharbaoui, H., Mansouri, K., & Poirier, F. (2024). Improving student engagement and success in computer programming courses through social learning in online environments. International Journal of Engineering Pedagogy, 14(6), 54–68. https://doi.org/10.3991/ijep.v14i6.48705

 

Hajibabaee, P., Pourkamali-Anaraki, F., & Hariri-Ardebili, M. (2024). Adaptive conformal prediction intervals using data-dependent weights with application to seismic response prediction. IEEE Access, 12, 53579–53597. https://doi.org/10.1109/ACCESS.2024.3387858

 

Jarvis, C. B., MacKenzie, S. B., & Podsakoff, P. M. (2003). A critical review of construct indicators and measurement model misspecification in marketing and consumer research. Journal of Consumer Research, 30(2), 199–218. https://doi.org/10.1086/376806

 

Kahu, E. R. (2013). Framing student engagement in higher education. Studies in Higher Education, 38(5), 758–773. https://doi.org/10.1080/03075079.2011.598505

 

Kamaruddin, S. F., Saifullizam, N. Z., Mohd Zahid, A. Z., Khalid, N., & Mat Zin, M. (2025). Exploring the influence of learner-to-learner interaction in online learning. International Journal of Research and Innovation in Social Science. https://doi.org/10.47772/IJRISS.2025.90900059

 

Kapoor, S., & Narayanan, A. (2023). Leakage and the reproducibility crisis in machine-learning-based science. Patterns, 4(9), 100804. https://doi.org/10.1016/j.patter.2023.100804

 

Kumar, B., Kumar, T., Nadakuditi, S., Patel, H., & Gupta, K. (2024). Comparing conformal and quantile regression for uncertainty quantification: An empirical investigation. International Journal of Computing and Engineering, 5(5), 1–8. https://doi.org/10.47941/ijce.1925

 

Kuzilek, J., Hlosta, M., & Zdrahal, Z. (2017). Open University Learning Analytics dataset. Scientific Data, 4, 170171. https://doi.org/10.1038/sdata.2017.171

 

Lei, J., G'Sell, M., Rinaldo, A., Tibshirani, R. J., & Wasserman, L. (2018). Distribution-free predictive inference for regression. Journal of the American Statistical Association, 113(523), 1094–1111. https://doi.org/10.1080/01621459.2017.1307116

 

Lin, X., & Sun, Q. (2024). Discussion activities in asynchronous online learning: Motivating adult learners' interactions. The Journal of Continuing Higher Education, 72(1), 84–103. https://doi.org/10.1080/07377363.2022.2119803

 

Liu, Z., Zhou, X., & Liu, Y. (2025). Student dropout prediction using ensemble learning with SHAP-based explainable AI analysis. Journal of Social System and Policy Analysis, 2(3), 111–132. https://doi.org/10.62762/jsspa.2025.321501

 

Livieris, I. E., Karacapilidis, N., Domalis, G., & Tsakalidis, D. (2023). An advanced explainable and interpretable ML-based framework for educational data mining. In Artificial intelligence applications and innovations. Springer, Cham, Switzerland. https://doi.org/10.1007/978-3-031-42134-1_9

 

Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30, 4765–4774.

 

Luo, R., & Zhou, Z. (2025). Conformal thresholded intervals for efficient regression. Proceedings of the AAAI Conference on Artificial Intelligence, 39(18), 19216–19223. https://doi.org/10.1609/aaai.v39i18.34115

 

Marchi, E. J., & Liebl, M. (2026). U-TraCE: A conformal prediction approach to uncertainty quantification in black-box models. Machine Learning: Science and Technology, 7(1), 015017. https://doi.org/10.1088/2632-2153/ae35ce

 

Mary, S. P., Gambhir, V., Mahalakshmi, R., Begum, B., Kala, & Mahajan, S. (2025). Optimizing student engagement prediction with machine learning and big data analytics. In Proceedings of ICMCTC 2025. IEEE. https://doi.org/10.1109/ICMCTC62214.2025.11196115

 

Mudau, P. K., & van den Berg, G. (2023). Guidelines for supporting a community of inquiry through graded online discussion forums in higher education. Education Sciences, 13(9), 963. https://doi.org/10.3390/educsci13090963

 

Murattaly, Y., & Serek, A. (2025). Predictive analytics for student engagement in e-learning systems. Journal of Emerging Technologies and Computing, 3(3), 23–30. https://doi.org/10.47344/88c0mc52

 

Navaneetha, T., Saravanakumar, R., Vini Infanta, A., Mohammed, M. A., Sumra, V., & Ali, M. (2025). Behavioral analytics in e-learning: Deep learning for real-time interaction and performance prediction. In Proceedings of ICETETSIP 2025. IEEE. https://doi.org/10.1109/ICETETSIP64213.2025.11156864

 

Nnadi, L. C., Watanobe, Y., Rahman, M. M., & John-Otumu, A. M. (2024). Prediction of students' adaptability using explainable AI in educational machine learning models. Applied Sciences, 14(12), 5141. https://doi.org/10.3390/app14125141

 

Onwuzurike, M. A., & Kpogli, S. A. (2025). Predictive modeling of student engagement and behavioral outcomes using machine learning techniques in technology-enhanced classrooms. International Journal of Scientific Research in Humanities and Social Sciences, 2(6), 58–79. https://doi.org/10.32628/ijsrhss2525135

 

Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2008). Learning styles: Concepts and evidence. Psychological Science in the Public Interest, 9(3), 105–119. https://doi.org/10.1111/j.1539-6053.2009.01038.x

 

Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). National Center for Research to Improve Postsecondary Teaching and Learning, Ann Arbor, MI. https://files.eric.ed.gov/fulltext/ED338122.pdf

 

Rachha, A., & Seyam, M. (2023). Explainable AI in education: Current trends, challenges, and opportunities. In Proceedings of IEEE SoutheastCon 2023. IEEE. https://doi.org/10.1109/SoutheastCon51012.2023.10115140

 

Roberts, D. R., Bahn, V., Ciuti, S., Boyce, M. S., Elith, J., Guillera-Arroita, G., Hauenstein, S., Lahoz-Monfort, J. J., Schröder, B., Thuiller, W., Warton, D. I., Wintle, B. A., Hartig, F., & Dormann, C. F. (2017). Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure. Ecography, 40(8), 913–929. https://doi.org/10.1111/ecog.02881

 

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68–78. https://doi.org/10.1037/0003-066X.55.1.68

 

Sheeran, P. (2002). Intention–behavior relations: A conceptual and empirical review. European Review of Social Psychology, 12(1), 1–36. https://doi.org/10.1080/14792772143000003

 

Shin, A. (2023). Exploring the role of interaction in engagement and satisfaction within virtual learning environments. Journal of Student Research, 12(3). https://doi.org/10.47611/jsrhs.v12i3.5014

 

Singh, G., Moncrieff, G., Venter, Z., Cawse-Nicholson, K., Slingsby, J., & Robinson, T. B. (2024). Uncertainty quantification for probabilistic machine learning in earth observation using conformal prediction. Scientific Reports, 14, 16166. https://doi.org/10.1038/s41598-024-65954-w

 

Slade, S., & Prinsloo, P. (2013). Learning analytics: Ethical issues and dilemmas. American Behavioral Scientist, 57(10), 1510–1529. https://doi.org/10.1177/0002764213479366

 

Sun, S. (2022). Conformal methods for quantifying uncertainty in spatiotemporal data: A survey. arXiv. https://doi.org/10.48550/arXiv.2209.03580

 

Tirumanadham, N. S. K. M. K., Thaiyalnayaki, S., & Ganesan, V. (2025). Towards smarter e-learning: Real-time analytics and machine learning for personalized education. International Journal of Computational and Experimental Science and Engineering, 11(1). https://doi.org/10.22399/ijcesen.786

 

Triantafyllou, S. A., & Sapounidis, T. (2024). Algorithmic approaches to the Traveling Salesman Problem for the development of students' computational thinking. In Y. Farhaoui (Ed.), Artificial intelligence, big data, IOT and block chain in healthcare: From concepts to applications (pp. 509–523). Springer, Cham, Switzerland. https://doi.org/10.1007/978-3-031-65018-5_48

 

Varma, S., & Simon, R. (2006). Bias in error estimation when using cross-validation for model selection. BMC Bioinformatics, 7, 91. https://doi.org/10.1186/1471-2105-7-91

 

Wanjale, K., Suryawanshi, A., Mathurkar, P., Raut, A., & Pawar, Y. (2025). A data-driven approach to personalized learning and course enhancement in education technology. In Proceedings of WorldS4 2025. IEEE. https://doi.org/10.1109/worldsuas66815.2025.11198927

 

Xu, Y., Guo, W., & Wei, Z. (2025). Selective conformal risk control. arXiv. https://doi.org/10.48550/arxiv.2512.12844

 

Yu, J., Wang, Y., Zhong, Q., Luo, G., Mao, Y., Sun, K., Feng, W., Xu, W., Cao, S., Zeng, K., Yao, Z., Hou, L., Lin, Y., Li, P., Zhou, J., Xu, B., Li, J., Tang, J., & Sun, M. (2021). MOOCCubeX: A large knowledge-centered repository for adaptive learning in MOOCs. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management (pp. 4643–4652). Association for Computing Machinery, New York, NY. https://doi.org/10.1145/3459637.3482010

 

Zhang, Y., & Liu, Y.-K. (2025). Conformal prediction beyond the horizon: Distribution-free inference for policy evaluation. arXiv. https://doi.org/10.48550/arxiv.2510.26026

 

Zhou, X., Li, Q., Xu, D., Holton, A., & Sato, B. (2023). The promise of using study-together groups to promote engagement and performance in online courses: Experimental evidence on academic and non-cognitive outcomes. The Internet and Higher Education. https://doi.org/10.1016/j.iheduc.2023.100922

Share
Back to top
International Journal of Systematic Innovation, Electronic ISSN: 2077-8767 Print ISSN: 2077-7973, Published by AccScience Publishing