predictive-modeling
Filtering by topic predictive-modeling(2)Clear all filters
- PaperJournal of Learning Analytics14 Mar 2026
Modelability as a Strategy for Improving the Generalizability and Scalability of Predictive Models
Alice Xu, Yunyi Zhang, Adam Blake, James Stigler
Introduces the concept of 'modelability'—a learning ecosystem designed for developing generalizable predictive models. Outlines three design principles and demonstrates their application through CourseKata, a platform that enabled early prediction models of student grades that generalized across institutions.
Original abstract
Learning analytics has the potential to enhance education through data-informed decision-making, but persistent challenges around generalizability and scalability continue to limit its real-world impact. In this paper, we introduce the concept of a modelable world: a learning ecosystem purposefully designed to support the development of predictive models that generalize across diverse contexts. We outline three core design principles of modelability: (1) valid and interpretable measurements, (2) scalable and stable implementation, and (3) a collaborative research–practice–technology ecosystem. We then illustrate how these principles can be operationalized in the real world through a case study of CourseKata, a platform offering a fully instrumented online textbook adopted across a wide range of institutions and disciplines. Using CourseKata data, we developed early prediction models of students’ final course grades using behavioral measures and tested the model generalizability across institutions (something rarely done in the modeling literature). Results show that a system designed with modelability in mind can produce predictive models that generalize effectively across diverse educational contexts.
- PaperJournal of Learning Analytics30 Nov 2025
Learning Analytics for Early Identification of At-Risk Students and Feedback Intervention
Wei Dai, Jionghao Lin, Flora Ji-Yoon Jin, Yi-Shan Tsai et al.
Developed machine learning models using trace and academic data from a prior course offering to identify at-risk students early in a subsequent semester. Intervention emails designed using a relational feedback framework led to over 30% of identified students engaging with previously unvisited learning activities within two weeks. Survey responses indicated general satisfaction, with 60% of respondents preferring more frequent interventions.
Original abstract
Supporting academically at-risk students has attracted much attention in the field of learning analytics. However, much of the research in this area has focused on developing advanced machine learning models to predict students' academic performance, which alone is insufficient to improve student learning without the implementation of timely interventions. Among the studies that attempted to mitigate this limitation by deploying intervention feedback to enhance learning, few created their feedback based on established theories of effective feedback. This theoretical oversight may limit students' uptake of the provided intervention. In response to these gaps, we conducted a study that aimed at supporting at-risk students at the early stage of an undergraduate-level course. Specifically, we developed predictive machine learning models using trace and academic data from the previous offering of a course, and applied these models to identify at-risk students in the subsequent semester's offering of the same course. For the identified at-risk students, we sent intervention emails designed by feedback experts based on a relational feedback framework designed to enhance feedback effectiveness by strengthening student-instructor relationships. We evaluated the effectiveness of the proposed approach by assessing the performance of the predictive models in terms of generalisability, and measuring the impact of the feedback intervention on students' learning engagement. Results showed that i) our predictive models demonstrated a high prediction accuracy (with AUC scores above 0.8) when applied to a new cohort of students; ii) more than 30% of the identified at-risk students visited previously unengaged learning activities within two weeks following the intervention; and iii) survey responses from 9.27% of at-risk students indicated general satisfaction with the provided feedback intervention, and 60\% of the respondents expressed a preference for receiving the intervention more frequently than the twice-per-semester frequency implemented in the present study.