educational-ai
Filtering by topic educational-ai(5)Clear all filters
- PaperComputers and Education: Artificial Intelligence1 Jul 2026
VETTING: A dual-LLM framework for in-loop safety verification via policy isolation in educational AI
Hongming Li, Shan Zhang, Anthony F. Botelho
VETTING introduces a dual large language model framework for real-time safety verification in educational AI systems by isolating and enforcing safety policies. The approach aims to prevent harmful outputs while maintaining pedagogical utility.
- PaperarXiv — AI in Education (cs.CY)27 Jun 2026
Bad company corrupts good morals: Understanding and Measuring Narrative-Induced Moral Reasoning Degradation in LLMs
Wanying Yu, Boyang Ma, Zhibo Eric Sun, Minghui Xu et al.
Large language models exposed to negative narratives in long-term interactions show degraded moral reasoning, with accuracy drops of 12-31%, especially in ambiguous scenarios. The study introduces BreakingBad, a three-stage framework to measure narrative-induced alignment degradation, finding that first-person narratives have stronger effects and that these shifts propagate into real-world deployments like counseling and education. The results reveal a new class of alignment risk not captured by existing safety defenses.
Original abstract
Large language models are deployed in long-context, emotionally interactive environments like digital humans, AI companions, educational assistants, and counseling systems. Unlike jailbreak attacks with explicit adversarial prompts, these systems interact with emotionally charged narratives involving bullying, betrayal, loneliness, social hostility, and institutional unfairness. This raises an important question: can prolonged narrative exposure reshape the reasoning and alignment stability of LLMs? We present the first systematic study of narrative-induced alignment degradation in LLMs. We design BreakingBad, a three-stage framework that measures how negative narrative immersion affects moral reasoning, behaviors, and deployment risks. It combines ethical decision evaluation, behavioral probing, and digital-human interaction analysis. Our experiments reveal three findings. First, negative narrative exposure degrades moral accuracy across multiple LLMs, with average drops of 12%-31%, especially in ambiguous scenarios and those involving vulnerable individuals. Second, the degradation is structured: different narratives induce distinct shifts, and first-person narratives produce stronger effects than third-person. Third, these shifts propagate into real deployments. Across counseling, education, medical, and financial/legal scenarios, narrative-conditioned models increasingly normalize hopelessness, cynicism, emotional detachment, and ethically questionable reasoning while remaining superficially policy-compliant. More broadly, our findings suggest alignment robustness is not static but a dynamically conditioned state shaped by long-term semantic environments and interaction history. These results reveal a new class of alignment risk that existing safety defenses largely fail to capture.
- PaperComputers & Education24 Jun 2026
Color me confounded: A critical analysis of media comparisons on ChatGPT in education
Alyssa P. Lawson, Amédee Marchand Martella, Joshua Weidlich, Miriam Mulders et al.
This paper critically analyzes how media outlets compare ChatGPT's role in education, highlighting confounding factors and inconsistencies in reporting.
- PaperAssessing Writing16 Jun 2026
When AI reframes what ‘counts’ as ‘good’ writing
Kelly Hartwell, Laura Aull
This paper argues that AI tools are reshaping the criteria for what is considered 'good' writing, challenging traditional notions of authorship and quality in academic and educational contexts.
- PaperETS Research Report Series20 Apr 2026
On the Representation of Racial and Ethnic Subgroups in AI-generated Texts: A Case Study in Automated Essay Scoring
Akshay Badola, Mo Zhang, Chen Li
Using GPT-4 and GPT-4o to generate essays for specific racial/ethnic subgroups from example essays, this study finds that the generated racial distribution does not match the real distribution. Augmenting automated essay scoring training data with these LLM-generated essays, even when race is mispredicted, reduces bias without harming performance.
Original abstract
In this study, we assess the capability of LLMs in generating essays of a specific race/ethnicity after being given example essays and rubric, and investigate the efficacy of data augmented in this manner for Automated Essay Scoring with respect to model performance and bias. In a series of experiments, we use models GPT-4 and GPT-4o, and ask them to generate essays from a given subgroup after inferring the race/ethnicity of the writer. We find that while LLMs can be directed to generate essays for specific demographic groups, the inferred racial and ethnic distribution in the generated data does not closely mirror the actual distribution observed in the source dataset. We augment existing data for underrepresented subgroups with LLM generated data separated into two groups with correct LLM race prediction and with incorrect race prediction and assess the improvement in agreement with human scores with quadratic weighted Kappa and bias mitigation as change in standardized mean difference. Our analysis shows that while LLMs struggle to predict the race accurately from given samples, augmentation with such data can be helpful to mitigate bias regardless. Suggested citation: Badola, A., Zhang, M., & Li, Chen. (in press). On the representation of racial and ethnic subgroups in AI-generated texts: A case study in automated essay scoring. ETS Research Report Series. https://doi.org/10.64634/ac01td58