moral-reasoning
Filtering by topic moral-reasoning(2)Clear all filters
- PaperarXiv — AI in Education (cs.CY)27 Jun 2026
Bad company corrupts good morals: Understanding and Measuring Narrative-Induced Moral Reasoning Degradation in LLMs
Wanying Yu, Boyang Ma, Zhibo Eric Sun, Minghui Xu et al.
Large language models exposed to negative narratives in long-term interactions show degraded moral reasoning, with accuracy drops of 12-31%, especially in ambiguous scenarios. The study introduces BreakingBad, a three-stage framework to measure narrative-induced alignment degradation, finding that first-person narratives have stronger effects and that these shifts propagate into real-world deployments like counseling and education. The results reveal a new class of alignment risk not captured by existing safety defenses.
Original abstract
Large language models are deployed in long-context, emotionally interactive environments like digital humans, AI companions, educational assistants, and counseling systems. Unlike jailbreak attacks with explicit adversarial prompts, these systems interact with emotionally charged narratives involving bullying, betrayal, loneliness, social hostility, and institutional unfairness. This raises an important question: can prolonged narrative exposure reshape the reasoning and alignment stability of LLMs? We present the first systematic study of narrative-induced alignment degradation in LLMs. We design BreakingBad, a three-stage framework that measures how negative narrative immersion affects moral reasoning, behaviors, and deployment risks. It combines ethical decision evaluation, behavioral probing, and digital-human interaction analysis. Our experiments reveal three findings. First, negative narrative exposure degrades moral accuracy across multiple LLMs, with average drops of 12%-31%, especially in ambiguous scenarios and those involving vulnerable individuals. Second, the degradation is structured: different narratives induce distinct shifts, and first-person narratives produce stronger effects than third-person. Third, these shifts propagate into real deployments. Across counseling, education, medical, and financial/legal scenarios, narrative-conditioned models increasingly normalize hopelessness, cynicism, emotional detachment, and ethically questionable reasoning while remaining superficially policy-compliant. More broadly, our findings suggest alignment robustness is not static but a dynamically conditioned state shaped by long-term semantic environments and interaction history. These results reveal a new class of alignment risk that existing safety defenses largely fail to capture.
- PaperBritish Journal of Educational Technology5 Apr 2026
When machines join the moral circle: The persona effect of generative AI agents in collaborative reasoning
Yueqiao Jin, Roberto Martinez‐Maldonado, Wanruo Shi, Songjie Huang et al.
A between-subjects experiment with 217 university students examined how supportive or contrarian generative AI teammates influence moral reasoning during collaborative discussion of an autonomous-vehicle dilemma. Supportive AI increased the quality of reasoning by consolidating integrative moral frames, while contrarian AI broadened value pluralism; both conditions reduced thematic drift compared to human-only groups, though final moral decisions rarely changed.
Original abstract
Generative AI is increasingly positioned as a peer in collaborative learning, yet its effects on ethical deliberation remain unclear. We report a between‐subjects experiment with university students ( N = 217) who discussed an autonomous‐vehicle dilemma in triads under three conditions: human‐only control, supportive AI teammate or contrarian AI teammate. Using moral foundations lexicons, argumentative coding from the augmentative knowledge construction framework, semantic‐trajectory modelling with BERTopic and dynamic time warping, and epistemic network analysis, we traced how AI personas reshape moral discourse. Supportive AIs increased grounded/qualified claims relative to control, consolidating integrative reasoning around care/fairness, while contrarian AIs modestly broadened moral framing and sustained value pluralism. Both AI conditions reduced thematic drift compared with human‐only groups, indicating more stable topical focus. Post‐discussion justification complexity was only weakly predicted by moral framing and reasoning quality, and shifts in final moral decisions were driven primarily by participants' initial stance rather than condition. Overall, AI teammates altered the process, the distribution and connection of moral frames and argument quality, more than the outcome of moral choice, highlighting the potential of generative AI agents as teammates for eliciting reflective, pluralistic moral reasoning in collaborative learning. Practitioner notes What is currently known about this topic AI tools can support discussion in collaborative learning, but evidence on ethical reasoning processes is mixed. Moral foundations theory and argumentation frameworks offer useful lenses for analysing value‐laden dialogue. Conversation analytics (eg, ENA and topic models) can reveal changes in discourse structure beyond outcome scores. What this paper adds Supportive AI teammates increase grounded/qualified claims compared with human‐only groups, improving the quality of moral reasoning. Contrarian AI teammates sustain value pluralism by connecting grounded claims to a wider moral repertoire, with only modest shifts in specific frames. Both AI personas reduce thematic drift, stabilising discussion focus; however, final moral decisions rarely change and justification complexity gains are small. Implications for practitioners Treat AI as persona‐configured teammates: use supportive styles to scaffold integrative reasoning and contrarian styles to elicit critical contrast. Design for process gains: instrument chats, monitor framing/argument quality and avoid over‐weighting post hoc decision change as the sole outcome. Govern participation: cap consecutive AI turns, keep timing natural and align persona goals with learning goals to prevent dominance while sustaining reflective dialogue.