textbook-evaluation
Filtering by topic textbook-evaluation(1)Clear all filters
- PaperEdArXiv (OSF Preprints)17 Jul 2026
Distributed Auditing of Cognitive Balance in History Textbooks: Application of the CAP Protocol within a Pre-Service Teacher Network
Hossein Talebzadeh
Analyzed the cognitive balance of Iranian upper secondary history textbooks using the Comparative Adjudication Protocol (CAP) with 43 pre-service teachers. Found extremely low student engagement indices (text involvement 0.019, image involvement 0), with 96.7% of lessons classified as passive 'banking education'. The CAP protocol achieved 86.4% inter-rater agreement, demonstrating reliability for content analysis.
Original abstract
Background: Despite theorists like Sam Wineburg emphasizing the development of historical thinking, history textbooks in centralized educational systems continue to be dominated by rote-learning approaches, leaving the systematic evaluation of their cognitive balance—particularly at the upper secondary level—largely neglected. Objective: This study aims to analyze the cognitive balance of upper secondary history textbooks (History 1, 2, 3, and Contemporary History of Iran) and evaluate the efficacy of the Comparative Adjudication Protocol (CAP) within a network of human auditors. Methodology: Engaging 43 pre-service teachers divided into 15 independent research teams, this study analyzed the content of 4 textbooks comprising 61 independent lessons using the William Romey technique and the CAP protocol. The process was executed across four distinct phases: capacity building, independent AI-assisted auditing, blind peer auditing, and discrepancy adjudication. Findings: The overall text involvement index was exceptionally low (0.019), far below the active learning threshold of 0.4, with 96.7% of the lessons falling into the passive "banking education" category. The image involvement index was absolute zero (0), and while the question involvement index was 1.29 (with an unbalanced distribution ranging from ∞ to 0.06), it revealed a severe structural gap between passive textual content and active analytical expectations. The CAP protocol demonstrated high reliability, achieving an 86.4% final inter-rater agreement and yielding a 12.5% increase in initial consensus. The dominant discrepancy pattern (A vs. B at 48.3%) indicated that distinguishing "objective historical facts" from "author interpretation" remains the primary analytical challenge for human coders in historical texts. Conclusion: Despite their narrative essence, high school history textbooks exhibit a highly passive cognitive structure lacking a coherent strategy to foster active student engagement. Implementing the CAP protocol within the R2A-TACI-PACT conceptual framework underscores the vital necessity of a "Human-in-the-Loop" (HITL) approach in historical content analysis. Curriculum revision and empowering pre-service teachers with algorithmic auditing literacy are highly recommended.