computing-education
Filtering by topic computing-education(4)Clear all filters
- PaperarXiv — AI in Education (cs.CY)7 Jul 2026
Say What? Examining Text and Voice Input Modalities for Prompt-Based Programming in Computing Education
Kaitlin Riegel, Yan Cathy Hua, Paul Denny, Victor-Alexandru Pădurean et al.
This exploratory study with 919 introductory programming students compared text and voice input modalities for prompt-based programming tasks. Students who typed prompts were more successful on first attempts than those using unedited voice prompts, though editing voice transcripts eliminated the difference. The findings suggest modality choice affects problem-solving, with most students preferring text, and have implications for designing multimodal LLM-powered educational tools.
Original abstract
Large language models (LLMs) are increasingly integrated into computing education, yet nearly all prior research has focused on text-based interactions. As voice-enabled interfaces become more capable and more common, there is growing interest in understanding how voice input might shape students' use of LLM-powered tools. In this exploratory study, we investigated how introductory programming students interact with Prompt Problems, which are programming tasks that require crafting natural-language prompts to generate correct code. Students (N = 919) solved a series of Prompt Problems with the freedom to select or switch between text and voice input modalities. We collected their prompt submissions as well as post-activity survey responses, then analysed differences in prompt accuracy, persistence, and perspectives by modality. For two of the three problems, we found that students who typed their prompts using text were more likely to have those prompts succeed on the first attempt than students who submitted unedited voice prompts. There was no difference in success rate if students edited their transcribed voice prompts before submission. Across the problems, we found evidence that students who tried voice prompting varied in their usage of modality - perhaps indicating a complementary, or non-preferential approach. However, most students only tried and reported preferring text. Our qualitative analysis revealed how students' perceived the roles of voice and text input in shaping their problem-solving process, as well as the reported drawbacks and advantages of each modality. We discuss implications for future multimodal tools and instructional design in computing education.
- PaperarXiv — AI in Education (cs.CY)2 Jul 2026
Automated grading of Linux/bash examinations using large language models: a four-level cognitive taxonomy approach
Manuel Alonso-Carracedo, Ruben Fernandez-Boullon, Pedro Celard, Francisco J. Rodriguez-Martinez et al.
Four frontier large language models (GPT, Claude Opus, Gemini, GLM) were evaluated for grading Linux/bash command responses using a four-level cognitive taxonomy. Gemini 3.0 Pro with rubric-guided prompting achieved the highest agreement with expert instructors (ICC=0.888). Agreement decreased with higher taxonomy levels, indicating that question complexity predicts grading difficulty and that rubric quality matters more than model choice.
Original abstract
Scalable and reliable grading of command-line examinations remains a challenge in computing education, where rising enrolments make manual marking difficult and rule-based autograders cannot handle partial credit, equivalent solutions, or syntactic variation. This paper evaluates whether four frontier Large Language Models (GPT, Claude Opus, Gemini, and GLM) can approximate expert judgment when grading short Linux/bash command responses. The study adopts a four-level cognitive taxonomy that combines cognitive complexity and operational impact, ranging from information retrieval (L1) and basic file manipulation (L2) to structural operations (L3) and advanced system management (L4). The models were tested with two prompt variants, a minimal baseline and a rubric-enhanced version, on 1200 real responses from second-year Computer Engineering students independently graded by three expert instructors. Gemini~3.0 Pro with rubric-guided prompting achieved the highest human-AI agreement (ICC(3,1) = 0.888, MAE = 0.10, Bland-Altman bias = -0.014). Agreement declined consistently as taxonomy level increased, with the largest discrepancies at higher levels. Across all models, rubric quality had a larger effect than provider choice, with structured prompts consistently improving agreement. These results show that question complexity is a reliable predictor of the difficulty LLMs face in grading accurately, and they establish a principled, taxonomy-based framework for determining which questions are suitable for AI-assisted grading and which require human review, while also providing a transferable evaluation protocol and prompt templates.
- PaperarXiv — AI in Education (cs.CY)30 Jun 2026
CogTax: A Four-Level Cognitive Taxonomy for Command-Line Computing Education
Manuel Alonso-Carracedo, Ruben Fernandez-Boullon, Pedro Celard, Francisco J. Rodriguez-Martinez et al.
A four-level cognitive taxonomy called CogTax integrates cognitive complexity from Bloom's Revised Taxonomy with operational impact (observational, reversible, structural, administrative) for command-line computing education. An automated classifier combining abstract syntax trees and semantic embeddings achieved 89% accuracy on 585 expert-annotated Linux/bash commands, demonstrating cross-language extensibility.
Original abstract
As computing education expands beyond traditional programming into operational domains such as systems administration and command-line environments, existing pedagogical frameworks struggle to capture a dimension that is critical in these contexts: the real-world consequences of learner actions. Existing cognitive taxonomies classify learning objectives by mental operations but do not account for system impact, leaving a critical gap in command-line education where conceptually simple commands can have severe consequences. This work presents CogTax, a four-level cognitive taxonomy that integrates two dimensions: cognitive complexity, derived from Bloom's Revised Taxonomy, and operational impact, which distinguishes observational, reversible, structural, and administrative operations. The four progressive levels range from safe read-only inspection to advanced system management requiring integration of multiple abstract models. Then, the taxonomy level is defined as the maximum of these dimensions, ensuring that both conceptual understanding and operational awareness are addressed. CogTax gives instructors a principled framework for sequencing course material and calibrating assessment difficulty, and gives students an explicit reference for self-assessment and gap identification. To demonstrate that taxonomy levels are automatically assignable, making the framework scalable without manual expert annotation, a classifier that combines syntactic representations derived from abstract syntax trees with semantic embeddings is trained. Evaluated on 585 expert-annotated Linux/bash commands, this combined approach achieves 89% accuracy, outperforming either representation alone, and demonstrates cross-language extensibility through structural equivalences across command languages.
- PaperarXiv — AI in Education (cs.CY)29 Jun 2026
Less Deliberate in Teams: Student LLM Use Across Individual and Collaborative Work
Sehrish Basir Nizamani, Zannah Ziew, Saad Nizamani, Khyati Goyal
A semester-long study of 96 undergraduates found that students used large language models (LLMs) significantly less in team projects than in individual homework, with a 42.7 percentage point drop in usage at the first team milestone. Students also wrote fewer and simpler prompts and verified AI output less carefully when working collaboratively. The findings suggest that collaborative contexts reduce deliberate LLM engagement beyond task type alone.
Original abstract
As large language models (LLMs) become common in computing courses, we need to understand how the social setting shapes how students use them. This paper reports findings from a semester-long study of 96 undergraduate students who completed six assignments, alternating between individual homework and team project milestones. We tracked LLM usage, prompting habits, and how students verified AI-generated output across all six assignments. LLM usage dropped by 42.7 percentage points when students moved from individual work to their first team milestone, then partly recovered in later team tasks. Students also wrote fewer and simpler prompts, used fewer intentional prompting strategies, and checked LLM output less carefully. The share of students who ran tests on AI-generated code fell by 19.4 percentage points during team assignments and never fully rebounded. A within-student analysis found that 18.9% of students who consistently used LLMs on their own stopped using them entirely in teams, while only 3.2% went the other direction. These results suggest that collaborative context is associated with reduced deliberate LLM engagement beyond what task type alone can explain. The moment students form teams appears to be a critical and currently unsupported turning point in computing course design.