learning-analytics
Filtering by topic learning-analytics(21)Clear all filters
- PaperComputers and Education: Artificial Intelligence17 Jul 2026
Comparing human and LLM ordered coding of qualitative data: How coding differences cascade through temporal analysis
Kamila Misiejuk, Sonsoles López-Pernas, Eduardo A. Oliveira, Brendan Eagan et al.
This study compares human and large language model (LLM) ordered coding of qualitative learner data, revealing systematic differences that cascade through temporal analysis. Two evaluation approaches assess coding quality and demonstrate a context-aware prompting method. Results show significant disparities in structural, transitional, and code-level metrics, cautioning against relying on LLM outputs for automated feedback in learning analytics.
Original abstract
Automating the process of qualitatively coding text data from learners has been a long-standing ambition of learning analytics researchers since it represents an essential step toward delivering timely and scalable feedback. Automating this process is especially challenging in the case of ordered coding schemes —necessary for temporal analytical methods— where one text utterance can be assigned more than one qualitative code and the assignment order matters. This problem goes beyond multi-class and multi-label classification and, therefore, cannot be easily tackled using classic language models such as BERT. Recent advances in generative artificial intelligence, especially with the advent of large language models, have —allegedly— created a substantial step forward in making the goal of automatically coding complex temporal data attainable. However, little is yet known about how to implement this process in a way that most closely resembles human coding, i.e., taking into account the context in which the textual data appears for accurate interpretation. Moreover, due to the complexity of the data and its shape, the accuracy of the results cannot be computed using classic accuracy metrics. This study makes two main contributions: first, it presents two evaluation approaches for assessing the quality of ordered data coding and the usability of LLM in automatically coding ordered processes; and second, it demonstrates a method of LLM prompting that leverages a consistent context window. Our results reveal systematic and statistically significant differences between LLM and human coding across structural, transitional, and code-level metrics for binary and ordered tasks. As classification errors can propagate through automated feedback systems, relying on LLM outputs risks amplifying inaccuracies and producing misleading interpretations of learning processes.
- PaperJournal of Learning Analytics8 Jul 2026
Toward Reliable Estimation of Algorithmic Bias for Minority Groups
Jaeyoon Choi, Shamya Karumbaiah, Jeffrey Matayoshi
Predictive models in learning analytics often show performance disparities across demographic groups, but reliable estimation of group bias is hampered by small group sizes and sampling error, especially for marginalized students. Using simulations and real-world data, the study recommends bootstrapping for confidence intervals, using multiple metrics, and moving beyond p-values to improve bias estimation.
Original abstract
While predictive models are widely used in learning analytics, several studies have shown that the performance of these models can vary significantly across different demographic groups of students. The first step to audit for and mitigate these group biases is to accurately estimate them. However, the current practices for identifying and measuring group bias often suffer from reliability issues. In this paper, we use simulations and real-world data analysis to explore statistical factors that impact the reliable estimate of group bias and suggest approaches to improve their statistical robustness. Our analysis revealed that small group sizes lead to high variability in group bias estimation due to sampling error -- an issue that is more likely to impact students from historically marginalized communities. We then suggest statistical approaches, such as bootstrapping, to construct confidence intervals for a more reliable estimation of group bias. Based on our findings, we encourage future learning analytics researchers to ensure sufficiently large group sizes, construct confidence intervals rather than relying on p-values, use at least two metrics, and move beyond the dichotomy of the presence or absence of bias for a more comprehensive evaluation of group bias.
- PaperBritish Journal of Educational Technology3 Jul 2026
A balance between stability and flexibility: Adaptive patterns of self‐regulated learning processes shape game‐based learning
Elizabeth B. Cloude, Stefan E. Huber, Jingwei Wei, Bianca Esmanhoto et al.
This study analyzes self-regulated learning (SRL) as a complex system in game-based learning (GBL), using multimodal data to examine patterns and interactions among cognitive, affective, metacognitive, and motivational processes. Results show a quadratic relationship between the regularity of cognitive, affective, and metacognitive state transitions and learning outcomes, indicating an optimal balance between stability and flexibility. Physiological measures of motivation did not predict learning performance, suggesting motivation's role may be mediated through cognitive and metacognitive engagement.
Original abstract
To ensure learning efficiency in game‐based learning (GBL), learners must regulate cognitive, affective, metacognitive and motivational (CAMM) processes, collectively known as self‐regulated learning (SRL). SRL is dynamic and non‐linear, characterized by regulatory patterns and CAMM interactions that lead to macro‐level SRL behaviours. In this paper, we explore SRL as a complex system by analysing patterns and interactions among CAMM processes during GBL and examine their relation to learning outcomes. Thirty‐seven ( n = 37) healthy adults used Antidote COVID‐19, a GBL environment designed to increase emotional engagement and biology knowledge. To define momentary CAMM processes, subjective measures of cognitive, affective and metacognitive states via think‐ and emote‐alouds were synchronized with two physiological correlates of motivation: intensity of facial expressions (arousal) and skin conductance response. Sequence analysis and recurrence quantification defined the regularity of patterns within CAMM components, and transfer entropy estimated concurrent interactions between CAMM components. Our results indicated a quadratic relationship between the turbulence (a measure of regularity) of cognitive, affective and metacognitive (CAM) state transitions (via think‐ and emote‐aloud methods) provided the best fit, explaining 31% of variability in learning. This highlights an optimal balance between stability and flexibility in CAM state transitions that were beneficial for learning. However, physiological correlates of motivation were not predictive regarding learning performance. This lack of predictive capability of the considered measures may reflect their limitations in capturing nuanced motivation dynamics relevant to learning or suggest that motivation's role in learning is predominantly mediated through cognitive and metacognitive engagement rather than directly influencing outcomes. Practitioner notes What is already known about this topic Game‐based learning (GBL) is a promising intervention for improving science knowledge, but it places high self‐regulatory demands on learners. Learners must manage cognitive, affective, motivational and metacognitive (CAMM) processes to ensure GBL efficiency; otherwise known as self‐regulated learning (SRL). Prior research has shown that CAMM processes are each important for learning in games, but they are often studied in isolation rather than as interacting processes over time. What this paper adds This study conceptualizes SRL in GBL as a complex system utilizing mixed multimodal data, emphasizing patterns and interactions among CAMM processes rather than static averages. Results show that learning is maximized when learners exhibit an optimal balance between stability and flexibility in their cognitive, affective and metacognitive state transitions, neither overly rigid nor overly random regulation patterns. Cognitive, affective and metacognitive processes (assessed via think–/emote‐alouds) were strongly related to learning gain, whereas physiological indicators of motivation alone were not predictive. Implications for practice and/or policy Designers of GBL environments should support adaptive regulation, encouraging learners to flexibly shift SRL strategies and emotions while maintaining coherence in their learning process. Educators and researchers should be cautious about relying solely on physiological measures (eg, arousal) for assessing learning effectiveness, as these may not capture meaningful regulatory dynamics. Policies and evaluation frameworks for educational games should prioritize tools and analytics that capture process‐level SRL patterns over time, rather than focusing exclusively on outcomes or isolated behavioural indicators.
- PaperJournal of Learning Analytics21 Jun 2026
Practitioner-Informed Learning Analytics Metrics for Measuring Curricular Complexity for Transfer Students
David Reeping, Dustin Grote, Julie Christensen, Sulabh Khadka et al.
Introduced three new metrics—inflexibility factor, transfer delay factor, and credit loss—to better measure curricular complexity for transfer students. Validated these metrics through focus groups with 38 transfer professionals, who affirmed their alignment with real-world curricular barriers.
Original abstract
This study expands the concept of curricular complexity as defined in the Curricular Analytics framework (Heileman et al., 2018) by providing qualitative evidence of the suitability of three new metrics, concerning timing of course offerings, extended time-to-degree, and credit loss, that more adequately address curricular challenges encountered by transfer students. Curricular Analytics is a method for analyzing a curriculum that enables practitioners and researchers to quantify and systematically analyze the impacts of course sequencing in a plan of study on student outcomes. However, the original conceptualization falls short of capturing the substantive challenges faced by transfer students who enter an undergraduate program at various points in the curricular sequence. This study was guided by the following research question: “How do three new measures of curricular complexity (i.e., inflexibility factor, transfer delay factor, and credit loss) align with transfer professionals’ perceptions of curricular barriers for transfer students?” Using a grounded theory approach, we conducted seven focus groups with 38 transfer professionals across the United States. We presented these transfer experts with each new measure and prompted them to reflect on its validity based on their experiences supporting transfer students. We found transfer professionals resonated strongly with all three new metrics, suggesting strong initial construct and content validity.
- PaperComputers and Education: Artificial Intelligence19 Jun 2026
Federated and explainable learning analytics for privacy-preserving academic risk modeling across heterogeneous educational institutions
William Villegas-Ch, Alexandra Maldonado Navarro, Jaime Govea, Joselin García-Ortiz et al.
This study proposes a federated, explainable learning analytics framework for modeling academic risk across heterogeneous educational institutions, integrating temporal behavioral features with socio-academic data under a multitask learning scheme. Results show that federated models maintain discriminative performance and stable convergence across non-IID partitions, while calibration metrics are more sensitive to distributional shifts. Explainability analysis reveals that feature importance remains structurally stable across institutions, supporting the deployment of privacy-preserving models in diverse settings.
Original abstract
The increasing digitization of higher education has enabled the development of learning analytics models to identify students at risk of academic failure or dropout; however, most existing approaches rely on centralized training and assume homogeneous data distributions, limiting their applicability across institutions with heterogeneous student populations and interaction patterns, while privacy constraints restrict data sharing and hinder collaborative model development. To address these challenges, this study proposes a federated, explainable learning analytics framework for modeling academic risk trajectories under controlled institutional heterogeneity. The proposed architecture integrates temporal behavioral representations with socio-academic features within a multitask learning scheme, evaluated under both centralized and federated regimes, while modeling institutional heterogeneity through parameterized non-IID partitions that introduce controlled class imbalance, temporal drift, and feature-level variability. Experimental results show that federated models preserve strong discriminative performance and stable convergence as heterogeneity increases. At the same time, calibration metrics exhibit greater sensitivity to distributional shifts, revealing a decoupling between ranking performance and probabilistic reliability. In parallel, explainability analysis shows that the relative importance of features remains structurally stable across institutions, despite variations in contribution magnitudes. Cross-platform evaluation further shows that models retain discriminative capacity when transferred across educational environments, while exhibiting changes in calibration and explanatory intensity. These findings highlight the importance of multidimensional evaluation in federated learning systems, jointly considering performance, calibration, and interpretability, and provide a methodological framework for deploying robust and privacy-preserving learning analytics models in heterogeneous educational settings.
- PaperAssessment & Evaluation in Higher Education4 Jun 2026
University instructors’ contemporary assessment literacy: development and validation of a questionnaire
Goudarz Alibakhshi
Developed and validated a 35-item questionnaire measuring university instructors' contemporary assessment literacy, covering nine dimensions including AI-responsive assessment, digital assessment literacy, and learning analytics. The instrument showed satisfactory reliability and validity based on expert reviews and factor analyses with 462 instructors. Findings highlight the need for measures that capture emerging assessment competencies in higher education.
Original abstract
Assessment literacy has become a key professional competence in higher education, where instructors are expected to design learning-oriented, ethical, inclusive, digitally mediated and evidence-informed assessment practices. However, existing measures do not fully capture contemporary demands related to feedback, learner involvement, artificial intelligence, learning analytics, accessibility and assessment consequences. This study developed and validated a questionnaire measuring university instructors’ contemporary assessment literacy. Using a multiphase, mixed-methods instrument development design, the study was conducted in two sequential phases. In Phase 1, a preliminary 39-item pool was reviewed by 22 assessment experts from three universities in Tehran. Expert ratings supported item relevance, clarity, representativeness and essentiality, and 12 items were revised. In Phase 2, the revised questionnaire was administered to 670 university instructors from four Tehran universities; 462 usable responses were returned. Exploratory factor analysis supported a nine-factor solution explaining 66.66% of the variance. The retained dimensions were learning-oriented assessment literacy, feedback literacy, learner involvement, fairness and ethics, digital assessment literacy, AI-responsive assessment literacy, inclusive and accessible assessment literacy, consequential validity and washback literacy, and assessment data and learning analytics literacy. Confirmatory factor analysis supported the final 35-item model, with satisfactory reliability, convergent validity, discriminant validity and model fit.
- PaperJournal of Learning Analytics18 May 2026
Learning-Aware Reliability Estimation for Tutor Skill Assessment Using Large Language Models
Conrad Borchers, Danielle R. Thomas, Jionghao Lin, Kenneth R. Koedinger
Introduces a learning-aware reliability estimation method using a Rasch-based split-half approach to adjust for learning gains when assessing LLM scoring reliability. Results show GPT-4 scoring achieves satisfactory reliability, with open-ended items (0.733) outperforming multiple-choice (0.652), and both combined yielding the highest reliability (0.774). The findings support using LLMs for formative assessment of complex instructional skills in online learning contexts.
Original abstract
Assessment is foundational to learning analytics, especially in evaluating instructional interventions and guiding improvement in online learning environments. With the growing use of large language models (LLMs) to score open-ended responses, questions arise about the reliability of these model-generated scores, particularly in short pre-post formats where learners are expected to improve. This study introduces a novel method for estimating test reliability that adjusts for learning gains using a Rasch-based split-half approach. We validated this approach through simulation under realistic conditions of missing data and score change, showing tangible improvements in reliability estimation compared to baseline methods. Applying this method to a dataset of 985 tutors completing 12 online lessons, we find that GPT-4-based scoring achieves satisfactory reliability, with open-ended responses (0.733) outperforming multiple-choice items (0.652). Both item types jointly yielded the highest reliability (0.774). Hence, as few as 14 open-ended items (across an average of 3-4 completed lessons) were sufficient to surpass common reliability thresholds of 0.7 or higher. Principal component analysis revealed a skill structure with a strong primary dimension shared across almost all lessons and interpretable subdimensions—socio-emotional, cognitive, and fairness-related tutoring skills—supporting a bifactor-like model. These findings demonstrate that GPT-4 and similar LLMs can be effectively used for formative assessment of complex instructional skills in online and personalized learning contexts, provided their reliability is empirically verified. This study contributes an open-source, learning-aware framework for scalable and reliable AI-supported assessment in learning analytics contexts.
- PaperComputer Assisted Language Learning5 May 2026
Learning analytics on multimodal GAI-driven EFL oral learning: uncovering learning behavior clusters with motivation and performance dynamics
Yuting Chen, Morris Siu-Yung Jong, Michael Yi-Chao Jiang, Ming Li
This study uses learning analytics to analyze multimodal data from generative AI-driven EFL oral learning, identifying clusters of learning behaviors and examining their relationships with motivation and performance dynamics.
- PaperBritish Journal of Educational Technology4 May 2026
Student profiles of change in formative assessment behaviour: Replication and evaluation for grade prediction
Oleksandra Poquet, Jelena Jovanovic, Stephan Krusche
Using assessment logs from 1362 students in a programming course, this study replicates a complex dynamical systems approach to characterize changes in formative assessment submission patterns. Three student profiles of behavioural change were identified, with higher entropy of recurrence associated with better performance and timeliness. While these dynamics-based features do not outperform conventional metrics for grade prediction, they offer complementary insights for interpreting student data.
Original abstract
As students learn and practice new skills in university courses, their behaviour can change in response to competing demands and increasing content complexity. However, most metrics used to evaluate study behaviour focus on the number or sequence of activities rather than on the change of behaviour. To address this, we replicate and extend a complex dynamical systems approach to characterise recurrence in behavioural patterns and whether it changes.Using assessment logs from 1362 students in the first 5 weeks of a semester‐long programming course, we examine whether changes in the patterns of formative assessment submissions can differentiate student sub‐groups and predict their performance. We identify three student profiles of behavioural change. We find that higher entropy of recurrence in assessment submission patterns is associated with better performance, and that changes in this entropy signal upcoming changes in performance. We also show that higher entropy of recurrence is associated with greater timeliness of submissions. Finally, we evaluate the predictive value of early behavioural patterns and find that while student profiles of change do not outperform conventional predictive metrics, they offer complementary insights that can enable timely interpretations of student data and inform interventions. Overall, our findings extend the generalisability of behavioural metrics based on complex dynamical systems by demonstrating consistent patterns across courses, LMS types and data sources. Practitioner notes What is already known about the topic Recurrence quantification analysis can capture dynamics of student behaviour. Prior work proposed a methodology based on recurrence of behaviour in a complex system to quantify study behaviour with trace data. Students whose behavioural patterns showed consistently high entropy of recurrence performed better. What this paper adds This study replicates a CDS‐based methodology in a new context, with a typical data source: traces of student assessment submissions. Dynamics‐based features are associated with the timeliness of student submissions and course performance. Dynamics‐based features do not outperform conventional LA features in predicting student performance, but offer complementary insights. Implications for policy/practice More research is needed to interpret what dynamics‐based features mean for teaching practice before they can be acted on. Future research and teaching activities could integrate interviews and self‐reported instruments to examine potential interpretations of dynamics‐based features, such as students' propensity to adapt.
- PaperJournal of Learning Analytics30 Mar 2026
Advancing 21st-Century Professional Competencies with Learning Analytics in the Age of Generative AI
Abhinava Barthakur, Olga Viberg, René F. Kizilcec, Ryan S. Baker et al.
This special issue introduction examines how learning analytics and generative AI can be leveraged to develop and assess 21st-century professional competencies in higher education. It presents four research directions: benchmarking LLMs for curricular alignment, designing GenAI chatbots for self-regulated learning, quantifying competencies via psychometric modeling, and institutional governance of curriculum analytics. The authors advocate for dynamic, process-sensitive assessments over static metrics.
Original abstract
The rise of generative artificial intelligence (GenAI) and accelerated globalization have necessitated a fundamental recalibration of higher education to prioritize domain-agnostic, 21st-century professional competencies. While institutional commitment to these skills is high, their systematic integration into the curriculum and evaluation remains fragmented, highlighting a critical gap between traditional academic success metrics and demonstrated workforce readiness. This special issue presents five complementary studies that investigate how the intersection of learning analytics (LA) and GenAI can bridge the gap between institutional rhetoric and demonstrated professional readiness. The contributions collectively advance a research agenda across four dimensions: 1) benchmarking large language models (LLMs) for curricular-competency alignment using reasoning-based prompting, 2) the iterative design of Socratic-style GenAI chatbots to scaffold self-regulated learning, 3) the application of psychometric modelling and Latent Profile Analysis to quantify 21st-century professional competencies, and 4) institutional governance and adoption of curriculum analytics. Collectively, these studies advocate for an epistemological shift toward process-sensitive assessments that move beyond static, episodic indicators toward dynamic, longitudinal representations of learner capability. We conclude by outlining the sociotechnical infrastructure, including robust governance and interdisciplinary collaboration, required to responsibly transition these AI-driven innovations from research prototypes to sustainable enterprise infrastructure, ensuring that analytics serve the evolving needs of students, educators, and professional bodies.
- PaperJournal of Learning Analytics30 Mar 2026
Function Art
Guillermo Bautista, Roderick Cacuyong, Zsolt Lavicza, Barbara Sabitzer et al.
Students from Grades 8 to 12 in the Philippines created digital artworks using mathematical functions in GeoGebra, revealing three distinct learner profiles: Repetitivists, Simplists, and Multifunctionists. The study found that students' mathematical strategies and precision in transformations varied widely, often independent of the quantity or diversity of functions used. These findings offer insights for tailoring interdisciplinary instruction in STEAM education.
Original abstract
This study explores how students across Grades 8 to 12 engage with mathematical functions in creative, visual ways through function art—an innovative STEAM-based educational approach. Grounded in the Trends in International Mathematics and Science Study (TIMSS) framework and employing a Design-Based Research methodology, the project involved 400 students from the Philippines who created digital artworks using GeoGebra. To uncover learner profiles, a person-centred clustering method—hierarchical clustering on principal components—was applied to variables representing the number and types of functions used. The results revealed three distinct student profiles: Repetitivists (high function quantity, low diversity), Simplists (low quantity and diversity), and Multifunctionists (high diversity, low quantity). Further analysis showed meaningful associations between cluster membership, grade level, and function strategies. Qualitative evaluation using TIMSS cognitive domains—Knowing, Applying, and Reasoning—highlighted that students’ use of mathematical strategies and precision in transformations varied widely, often independently of the quantity or diversity of functions used. These findings suggest that function art, when analyzed through learning analytics, provides a rich lens for understanding students’ mathematical thinking and offers valuable insights for tailoring interdisciplinary instruction in STEAM education.
- PaperJournal of Learning Analytics22 Mar 2026
Human-Centred Development of Indicators for Self-Service Learning Analytics
Shoeb Joarder, Mohamed Amine Chatti
Designed and implemented an Indicator Editor for self-service learning analytics using a human-centred approach, evaluated with a qualitative user study (n=15). Found that user interaction and control in indicator implementation positively affect transparency, trust, satisfaction, and acceptance.
Original abstract
The aim of learning analytics (LA) is to turn educational data into insights, decisions, and actions to improve learning and teaching. The reasoning of the provided insights, decisions, and actions is often not transparent to the end-user, and this can lead to trust and acceptance issues when interventions, feedback, and recommendations fail. In this paper, we shed light on achieving transparent LA by following a transparency through exploration approach. To this end, we present the design, implementation, and evaluation details of the Indicator Editor, which aims to support self-service LA (SSLA) by empowering end-users to take control of the indicator implementation process. We systematically designed and implemented the Indicator Editor through an iterative human-centred design (HCD) approach. Further, we conducted a qualitative user study (n=15\) to investigate the impact of following an SSLA approach on users' perceptions of and interactions with the Indicator Editor. Our study showed qualitative evidence that supporting user interaction and providing user control in the indicator implementation process can have positive effects on different crucial aspects of LA, namely transparency, trust, satisfaction, and acceptance.
- PaperJournal of Learning Analytics20 Mar 2026
12 Heuristics for Learning Analytics in Simulation-Based Professional Learning
Susan Harrington, Charlott Sellberg
Developed a set of heuristics for evaluating learning analytics in simulation-based professional learning by combining theoretical frameworks and empirical findings, refined through expert collaboration. The resulting framework accounts for technological, pedagogical, and social dimensions.
Original abstract
This study aims to develop a set of heuristics tailored for evaluating learning analytics in simulation-based professional learning, focusing on the following research questions: (1) What heuristics are appropriate for evaluating learning analytics in simulation-based professional learning contexts? (2) How can theoretical frameworks and empirical findings be combined in the development of such heuristics? (3) How can expert evaluation inform their refinement and applicability? The study combines a top-down approach, drawing on a theoretical framework for learning experience design, with a bottom-up analysis of empirical findings from prior studies in the context of a design project. An initial set of heuristics was iteratively reviewed and refined in collaboration with experts in user and learning experience design. The outcome is a detailed heuristic framework that supports the evaluation of learning analytics in simulation-based settings and accounts for the technological, pedagogical, and social dimensions of professional learning.
- PaperJournal of Learning Analytics18 Mar 2026
Beyond Time on Task
Paul V. Sargent, Isabel Hilliger, Jorge A. Baier
Student workload analysis beyond average time-on-task metrics is explored using self-reported and LMS data from 14 engineering courses. Results show that workload dynamics, such as peaks, correlate with perceived learning and difficulty, suggesting that dynamic metrics can inform course design.
Original abstract
Student workload analysis has the potential to play a crucial role in providing both actionable insights to inform course design and curricular adjustments that promote student learning and well-being. While numerous studies have emphasized the need for analyzing workload beyond single-value metrics, such as credit hours, the interpretation and practical application of these metrics for educational interventions remains unclear. In this study, we explore the interplay between time-on-task measurements with student-perceived learning and difficulty.We move beyond average indicators of time-on-task by proposing and examining various metrics related to the dynamics of workload over time. Across 14 engineering courses taught at Pontificia Universidad Católica de Chile, we analyze three different sources of data: (1) self-reported time-on-task and perceived difficulty obtained through a weekly timesheetsurvey, (2) interactions with the learning management system (LMS), and (3) perceived learning attainment obtained from the course evaluation survey. Our results show that LMS-based and self-reported time-on-task were highly correlated. Also, workload dynamics metrics, such as the presence of workload peaks, were highly correlated with perceived learning and perceived difficulty. As such, this study provides evidence in support of considering workload dynamics, rather than average measures of time-on-task, to predict variables related to student learning. The metrics proposed by this framework could be used to implement practical tools for educators and administrators willing to optimize course design and improve learning attainment.
- PaperJournal of Learning Analytics15 Mar 2026
Assessing Patterns of Students’ Attainment of Professional Standards in Higher Education
Abhinava Barthakur, Jelena Jovanović, Ryan Baker, Vitomir Kovanović et al.
A novel curriculum analytics method maps assessment grades to professional standards in a Teacher Education program. Using psychometric modeling and learning analytics, distinct patterns in learners' acquisition of professional standards were identified. The approach offers scalable, automated inference of standard attainment without altering current assessment practices.
Original abstract
It is widely recognized that higher education (HE) graduates require a broad range of professional skills and abilities to succeed in their future careers. However, despite this acknowledgement, assessment practices in HE remain focused on content-based knowledge. This narrow emphasis limits the capacity to effectively and holistically evaluate a student’s professional competency and readiness for employment. This issue is particularly acute for HE degrees that require graduates to demonstrate attainment of externally regulated professional standards. While the curricula are mapped to professional standards for accreditation purposes, demonstrating a student’s attainment of these standards is not straightforward and has mostly been done through self-reported surveys. This study offers a novel curriculum analytics method for mapping assessment grades to the attainment of professional standards across a Teacher Education program. Specifically, we present an approach that uses psychometric modelling and learning analytics to identify distinct patterns in learners’ acquisition of professional standards. This method does not alter current assessment practices in HE. Instead, the approach offers a scalable, automated means to infer a learner’s attainment of documented professional standards, complementing current measures of academic success, such as GPA. The study underscores the advantages of complementing the current HE assessment practises with an outlined curriculum analytics approach, providing a holistic representation of a student’s learning progress.
- PaperJournal of Learning Analytics14 Mar 2026
Modelability as a Strategy for Improving the Generalizability and Scalability of Predictive Models
Alice Xu, Yunyi Zhang, Adam Blake, James Stigler
This study introduces the concept of a 'modelable world' with three design principles to improve the generalizability and scalability of predictive models in education. Using data from the CourseKata platform, early prediction models of final course grades were developed and shown to generalize across institutions.
Original abstract
Learning analytics has the potential to enhance education through data-informed decision-making, but persistent challenges around generalizability and scalability continue to limit its real-world impact. In this paper, we introduce the concept of a modelable world: a learning ecosystem purposefully designed to support the development of predictive models that generalize across diverse contexts. We outline three core design principles of modelability: (1) valid and interpretable measurements, (2) scalable and stable implementation, and (3) a collaborative research–practice–technology ecosystem. We then illustrate how these principles can be operationalized in the real world through a case study of CourseKata, a platform offering a fully instrumented online textbook adopted across a wide range of institutions and disciplines. Using CourseKata data, we developed early prediction models of students’ final course grades using behavioral measures and tested the model generalizability across institutions (something rarely done in the modeling literature). Results show that a system designed with modelability in mind can produce predictive models that generalize effectively across diverse educational contexts.
- PaperJournal of Learning Analytics14 Mar 2026
Profiling Pre-service Teachers’ Computational Thinking
Tanya Chichekian, Maria Cutumisu, Annie Savard, Yi-Mei Zhang
Using multimodal data from 128 pre-service teachers, latent profile analysis identified three computational thinking profiles (Novice, Developing, Proficient) based on digital literacy, problem-solving, and coding comfort. Metacognitive strategies and prior coding experience significantly predicted profile membership, and proficient learners showed greater task efficiency and fewer perceived challenges. Findings support non-linear skill acquisition and suggest design of learning analytics dashboards for adaptive teacher training.
Original abstract
Computational thinking (CT) is a vital skill set for pre-service teachers who will need to foster computational literacy in K–12 classrooms, yet the factors influencing their CT skills remain less understood than those for K–12 students or in-service teachers. This study leverages multimodal data to investigate how pre-service teachers (n=128) differ in CT skills, the predictive role of metacognitive strategies and prior coding experience, and variations in online behaviours. Using latent profile analysis, we identified three profiles based on digital literacy, problem-solving, and coding comfort (Novice, Developing, and Proficient), revealing heterogeneity in CT, and supporting non-linear skill acquisition. Linear discriminant analysis revealed that metacognitive strategies and prior coding experience significantly predict profile membership, validating the interplay of technical and cognitive factors in the development of CT skills. Behavioural data from an interactive problem-solving task showed that, compared to Novices and Developing learners, Proficient learners were more task efficient and perceived fewer challenges during task completion. Implications for designing a learning analytics dashboard to visualize profiles and behavioural metrics to support adaptive, equitable, and personalized teacher training are discussed, thereby enhancing pre-service teachers’ readiness to integrate CT into K–12 education.
- PaperJournal of Learning Analytics25 Feb 2026
Learning Analytics to Uncover Ethnic Bias in Educational Texts
Josmario Albuquerque, Bart Rienties, Martin Hlosta, Wayne Holmes
This study uses learning analytics and machine learning to detect ethnic bias in text-based educational materials. Support vector machines and random forest classifiers performed consistently (F1-scores ~0.70), while naïve Bayes achieved highest precision (0.75). The findings highlight correlations between perceived bias and social sciences content, offering a scalable foundation for automated bias detection.
Original abstract
Online learning platforms have expanded access to education but also raise concerns about biased content, particularly in text-based learning materials such as textbooks, lesson plans, and course excerpts. Such biases can perpetuate discrimination, can harm student outcomes, and can often be difficult to detect, as identification typically relies on time-consuming human review. Learning analytics (LA) can enhance this process by supporting human reviewers through automated detection, offering a scalable solution while retaining human judgment for nuanced evaluations. Accordingly, this LA study explores two research questions: RQ1: Which features might support the identification of ethnic bias in text-based online learning materials? and RQ2: Which classification approaches might be suitable for identifying ethnic bias in text-based online learning materials? First, we identified features signalling potential ethnic bias (presence or absence) in textual content using a dataset (N = 345) labelled by 193 students from diverse ethnic backgrounds. Then, we evaluated multiple machine learning (ML) models for their effectiveness in bias classification. The results suggest significant correlations between perceived bias and content from social sciences. Additionally, through bootstrap analysis, support vector machines and random forest classifiers showed consistent performance in bias identification (with F1-scores of 0.71 and 0.70 on the test set, respectively). In contrast, the naive Bayes (NB) model demonstrated the highest precision (0.75 on the test set). We discuss these findings and their implications for LA, emphasizing the importance of quality and inclusive educational tools. As an initial step toward automated bias classification in education, this study provides a foundation for spotting ethnic bias in learning content, supporting fairer technologies for more inclusive learning environments.
- PaperJournal of Learning Analytics15 Dec 2025
Fostering Human Agency in Age of AI
Olga Viberg, Oleksandra Poquet, Vitomir Kovanovic, Hassan Khosravi
This editorial examines key dilemmas in designing AI-mediated learning systems that preserve human agency, focusing on maintaining meaningful human control and fostering critical engagement. It outlines future research opportunities for the learning analytics community in the context of generative AI's rapid transformation of learning practices.
Original abstract
As learning analytics (LA) and artificial intelligence (AI) increasingly shape how learning processes are monitored and supported, human agency has emerged as a critical concern. With generative AI rapidly transforming learning practices and influencing pedagogical decision-making, safeguarding the agency of both learners and educators is becoming essential. This editorial discusses key dilemmas in designing and evaluating AI-mediated learning systems that maintain meaningful human control, foster critical engagement, and enable ethical and effective integration into learning settings. We conclude by outlining future research opportunities for the learning analytics community and reflecting on the journal’s development over the past year.
- PaperJournal of Learning Analytics15 Dec 2025
Augmented Reality Enhanced Analytics for Education
Manjeet Singh, Shaun Bangay, Atul Sajjanhar
A systematic literature review of augmented reality (AR) applications using enhanced analytics in primary, secondary, and higher education from 2000 to 2025 finds that AR enhanced analytics expand measures of engagement and effectiveness, offering richer real-time insights into student learning processes beyond traditional self-report measures. The review highlights challenges and opportunities for broader extended reality (XR) integration and methodological standardization.
Original abstract
This systematic literature review of augmented reality (AR) applications utilizing enhanced analytics in education settings surveys publications from 2000 to 2025, specifically targeting AR use in primary, secondary, and higher-education sectors. With the growing use of AR in such settings, this review informs educators, application designers, and scholars on insights and evidence gained in current and past studies on the use of enhanced analytics in AR applications. There are relative advantages and challenges in using AR enhanced analytics in educational settings, with the potential to contribute to knowledge in understanding students' engagement, motivation, behaviour, learning experiences, and acquisition of skills.The synergy of AR and learning analytics (LA) is significant as it enables richer, unobtrusive, real-time insights into student learning processes, advancing research beyond traditional self-report measures. The review demonstrates that AR enhanced analytics expand measures of engagement and effectiveness, while also pointing to opportunities for broader extended reality (XR) integration and methodological standardization.
- PaperJournal of Learning Analytics30 Nov 2025
Learning Analytics for Early Identification of At-Risk Students and Feedback Intervention
Wei Dai, Jionghao Lin, Flora Ji-Yoon Jin, Yi-Shan Tsai et al.
Developed predictive machine learning models using trace and academic data from a previous course offering to identify at-risk students in a subsequent semester, then sent theory-based intervention emails designed to strengthen student-instructor relationships. The models achieved high prediction accuracy (AUC > 0.8), and over 30% of at-risk students engaged with previously unvisited learning activities within two weeks of the intervention.
Original abstract
Supporting academically at-risk students has attracted much attention in the field of learning analytics. However, much of the research in this area has focused on developing advanced machine learning models to predict students' academic performance, which alone is insufficient to improve student learning without the implementation of timely interventions. Among the studies that attempted to mitigate this limitation by deploying intervention feedback to enhance learning, few created their feedback based on established theories of effective feedback. This theoretical oversight may limit students' uptake of the provided intervention. In response to these gaps, we conducted a study that aimed at supporting at-risk students at the early stage of an undergraduate-level course. Specifically, we developed predictive machine learning models using trace and academic data from the previous offering of a course, and applied these models to identify at-risk students in the subsequent semester's offering of the same course. For the identified at-risk students, we sent intervention emails designed by feedback experts based on a relational feedback framework designed to enhance feedback effectiveness by strengthening student-instructor relationships. We evaluated the effectiveness of the proposed approach by assessing the performance of the predictive models in terms of generalisability, and measuring the impact of the feedback intervention on students' learning engagement. Results showed that i) our predictive models demonstrated a high prediction accuracy (with AUC scores above 0.8) when applied to a new cohort of students; ii) more than 30% of the identified at-risk students visited previously unengaged learning activities within two weeks following the intervention; and iii) survey responses from 9.27% of at-risk students indicated general satisfaction with the provided feedback intervention, and 60\% of the respondents expressed a preference for receiving the intervention more frequently than the twice-per-semester frequency implemented in the present study.