21st-century-competencies
Filtering by topic 21st-century-competencies(1)Clear all filters
- PaperJournal of Learning Analytics25 Feb 2026
Evaluating 21st-Century Competencies in Postsecondary Curricula with Large Language Models
Zhen Xu, Xin Guan, Chenxi Shi, Qinhao Chen et al.
This study evaluates how large language models (LLMs) can map postsecondary curricula to 21st-century competencies using 7,600 manually annotated alignment scores across 38 competencies and 200 courses. Detailed instructional activity descriptions were the most informative curriculum source; open-weight LLMs matched proprietary models on coarse-grained tasks but not fine-grained pedagogical reasoning. A curricular chain-of-thought prompting strategy modestly improved reasoning by reducing keyword bias and enhancing detection of nuanced evidence.
Original abstract
The growing emphasis on 21st-century competencies in postsecondary education, intensified by the transformative impact of generative artificial intelligence (GenAI) on the economy and society, underscores the urgent need to evaluate how they are embedded in curricula and how effectively academic programs align with evolving workforce and societal demands. Curricular analytics, particularly recent advancements powered by GenAI, offer a promising data-driven approach to this challenge. However, the analysis of 21st-century competencies requires pedagogical reasoning beyond surface-level information retrieval, and the capabilities of large language models (LLMs) in this context remain underexplored. In this study, we extend prior research on curricular analytics of 21st-century competencies across a broader range of curriculum documents, competency frameworks, and models. Using 7,600 manually annotated curriculum-competency alignment scores (38 competencies and 200 courses across five curriculum document types), we evaluate the informativeness of different curriculum document sources, benchmark the performance of general-purpose LLMs on mapping curricula to competencies, and analyze error patterns. We further introduce a reasoning-based prompting strategy, curricular chain-of-thought (CoT), to strengthen LLMs’ pedagogical reasoning. Our results show that detailed instructional activity descriptions are the most informative type of curriculum document for competency analytics. Open-weight LLMs achieve accuracy comparable to proprietary models on coarse-grained tasks, demonstrating their scalability and cost-effectiveness for institutional use. However, no model reaches human-level precision in fine-grained pedagogical reasoning. Our proposed curricular CoT yields modest improvements by reducing bias in instructional keyword inference and improving the detection of nuanced pedagogical evidence in long text. Together, these findings highlight the untapped potential of institutional curriculum documents and provide an empirical foundation for advancing AI-driven curricular analytics.