feedback
Filtering by topic feedback(2)Clear all filters
- PaperarXiv — AI in Education (cs.CY)21 Jul 2026
Assessment in Team Problem-Solving Exercises in Computing Education
Valdemar Švábenský, Jan Vykopal, Sukrit Leelaluk, Pavel Čeleda et al.
Compared two post-exercise team assessment methods—clustering and large language models (LLMs)—for tabletop exercises in computing education. Clustering grouped teams by task approach, enabling faster, targeted feedback with low computational cost. LLMs showed mixed accuracy against instructor scores, with GPT-5.2 performing better than GPT-4o.
Original abstract
This full paper in the research-to-practice track presents methods for assessing student teams in tabletop exercises (TTXs). TTXs enable learner teams to prepare for workplace tasks and practice crisis responses, such as resolving cybersecurity incidents. While assessment is essential for determining how well teams achieve learning objectives, the complex, open-ended nature of TTXs often leads to delayed or incomplete feedback. TTX learning platforms can record teams' actions and communication; yet, leveraging these data to assess performance is underexplored. To address this gap, we compared two post-TTX team assessment methods -- clustering and large language models (LLMs) -- using an original dataset from 81 participants across two countries. We evaluated these methods against instructor-assigned scores based on standardized rubrics. Clustering grouped teams that approached TTX tasks similarly, enabling instructors to deliver faster, targeted feedback to teams within a cluster. This method was valid and reliable, with low computational requirements. LLMs used the standardized rubrics to assess teams' communication. While GPT-4o frequently disagreed with instructor scores, GPT-5.2 demonstrated considerably lower error. The researched methods have been integrated into INJECT, an open-source TTX learning platform, to support scalability and teaching practice. To encourage community adoption, we publicly share all datasets, software tools, and a full-fledged TTX scenario.
- PaperEdArXiv (OSF Preprints)20 Jul 2026
Student Learning Rates When Practicing with an Intelligent Tutoring System Versus On Paper
Conrad Borchers, Qianru Lyu, Ritesh Kanchi, Kenneth R. Koedinger et al.
This study compared learning rates and gains between middle-school students practicing math problems with an intelligent tutoring system (ITS) versus on paper. Results showed significantly higher learning rates and gains with ITS, especially for lower-proficiency students, while paper-based practice showed little measurable learning. The findings emphasize the importance of immediate feedback and adaptive support in learning.
Original abstract
Intelligent tutoring systems (ITSs) are widely used in K-12 education and are known to improve learning outcomes compared to traditional instruction. However, prior work has focused primarily on end-of-test performance, with little attention to learning processes, such as how much students learn at each practice opportunity. In particular, learning rates have been extensively studied in ITS environments but not for paper-based problem solving. We conducted a within-subjects classroom experiment with 97 middle-school students solving matched mathematics problems either with an ITS or on paper. We compared both overall learning gains and process-level learning rates, defined as improvement per problem-solving step. To enable this comparison, we introduce a novel method for converting paper-based work into step-level transaction data compatible with learning-curve modeling. Results show that ITS-supported practice led to significantly higher learning gains and substantially higher learning rates than paper-based practice. While students in the ITS condition exhibited consistent positive learning rates, learning rates in the paper condition were indistinguishable from zero, indicating little measurable learning during problem solving without feedback. These differences were especially pronounced for students with lower prior proficiency. This study provides the first direct comparison of learning rates between ITS and paper-based practice and introduces a generalizable methodology for analyzing learning processes in non-digital environments. The findings highlight the critical role of immediate feedback and adaptive support in enabling learning during practice and support the broader adoption of ITS in K-12 education.