natural-language-programming
Filtering by topic natural-language-programming(2)Clear all filters
- PaperarXiv — AI in Education (cs.CY)6 Jul 2026
Understanding Student Perceptions, Mistakes, and Debugging Approaches when Solving Natural Language Programming Tasks
Victor-Alexandru Pădurean, Kaitlin Riegel, Gweneth Barbre, Musa Blake et al.
A study of over 900 CS1 students solving Prompt Problems found that while students generally found prompting easier and more enjoyable than traditional coding, they often omitted key details, indicating over-reliance on AI. When prompts failed, students focused more on clarifying intent and revisiting problem details than on debugging code.
Original abstract
Learning to communicate with code-generating AI models is an emerging skill for novice programmers. One recent pedagogical approach, Prompt Problems, has students solve computational tasks by writing natural-language prompts for code-generating AI models. However, little is known about the specific prompt-level mistakes novice programmers make, the kinds of computational details they fail to communicate, and what strategies they use to recover when generated code is incorrect. In a CS1 course, we studied attempts by more than 900 students to solve dialogue-based Prompt Problems. We analyzed student reflections, unsuccessful prompts, and reported debugging strategies. Compared to traditional coding tasks, students generally found prompting easier, more enjoyable, and better targeted at developing problem-solving skills. The most common mistakes are related to the omission of key details, suggesting both a failure to acknowledge their importance and over-reliance on AI to infer them. When prompts failed, students focused more on clarifying their intent and reflecting on the provided problem details than on tracing generated code or examining test cases.
- PaperarXiv — AI in Education (cs.CY)29 Jun 2026
Teaching Prompt-Based Programming with LLMs: A 45-Minute Lesson with Guided Practice for End-User Programmers
Keith Tran, Samiha Marwan, Thomas Price
A 45-minute prompt-based programming intervention with guided practice was evaluated against a control activity. Engineering students in the experimental group showed greater gains in prompting self-efficacy and slight improvement in ability to specify computational goals to LLMs. The results suggest that even a brief intervention can modestly improve prompting skills, but more time may be needed for significant gains.
Original abstract
Prompt-based programming, a new modality enabled by large language models (LLMs), allows users to express computational goals through natural language rather than traditional code. While this approach lowers barriers to entry, especially for non-CS learners, it does not eliminate the need for foundational CS skills. Learners often struggle to communicate their intent clearly to LLMs, resulting in vague or underspecified prompts. Prior work has documented the need for explicit prompting for both CS and non-CS learners. However, it remains less clear how such instruction can fit into busy classrooms or how much time is needed to produce meaningful gains. In this paper, we evaluated a 45-minute prompt-based programming intervention, consisting of a lesson with guided practice, against a business-as-usual CS lab activity (code tracing) of equal length, representing a class without prompt-focused instruction. We conducted a randomized controlled study with 55 engineering students. We found that students in the experimental condition improved more on average (though not significantly more) from pre- to post-test than the control group (+10.8 vs +1.1 percentage points) and showed significantly greater average gains in prompting self-efficacy (+35.4 vs +21.9 percentage points). Our results suggest it is likely that a brief intervention can improve learners' ability to specify computational goals to LLMs. However, the effect was modest, suggesting that prompting skills may require more time and practice to develop. We provide a lightweight lesson that requires no prior CS background and can be readily dropped into existing courses.