textbook-evaluation
Filtering by topic textbook-evaluation(2)Clear all filters
- PaperEdArXiv (OSF Preprints)17 Jul 2026
Distributed Auditing of Cognitive Balance in History Textbooks: Application of the CAP Protocol within a Pre-Service Teacher Network
Hossein Talebzadeh
Analyzed the cognitive balance of Iranian upper secondary history textbooks using the Comparative Adjudication Protocol (CAP) with 43 pre-service teachers. Found extremely low student engagement indices (text involvement 0.019, image involvement 0), with 96.7% of lessons classified as passive 'banking education'. The CAP protocol achieved 86.4% inter-rater agreement, demonstrating reliability for content analysis.
Original abstract
Background: Despite theorists like Sam Wineburg emphasizing the development of historical thinking, history textbooks in centralized educational systems continue to be dominated by rote-learning approaches, leaving the systematic evaluation of their cognitive balance—particularly at the upper secondary level—largely neglected. Objective: This study aims to analyze the cognitive balance of upper secondary history textbooks (History 1, 2, 3, and Contemporary History of Iran) and evaluate the efficacy of the Comparative Adjudication Protocol (CAP) within a network of human auditors. Methodology: Engaging 43 pre-service teachers divided into 15 independent research teams, this study analyzed the content of 4 textbooks comprising 61 independent lessons using the William Romey technique and the CAP protocol. The process was executed across four distinct phases: capacity building, independent AI-assisted auditing, blind peer auditing, and discrepancy adjudication. Findings: The overall text involvement index was exceptionally low (0.019), far below the active learning threshold of 0.4, with 96.7% of the lessons falling into the passive "banking education" category. The image involvement index was absolute zero (0), and while the question involvement index was 1.29 (with an unbalanced distribution ranging from ∞ to 0.06), it revealed a severe structural gap between passive textual content and active analytical expectations. The CAP protocol demonstrated high reliability, achieving an 86.4% final inter-rater agreement and yielding a 12.5% increase in initial consensus. The dominant discrepancy pattern (A vs. B at 48.3%) indicated that distinguishing "objective historical facts" from "author interpretation" remains the primary analytical challenge for human coders in historical texts. Conclusion: Despite their narrative essence, high school history textbooks exhibit a highly passive cognitive structure lacking a coherent strategy to foster active student engagement. Implementing the CAP protocol within the R2A-TACI-PACT conceptual framework underscores the vital necessity of a "Human-in-the-Loop" (HITL) approach in historical content analysis. Curriculum revision and empowering pre-service teachers with algorithmic auditing literacy are highly recommended.
- PaperDOAJ — ELT & TESOL1 Jul 2020
An ESP textbook evaluation: the case of English for Islamic Insight
Faizatul Istiqomah, Binti Maunah
This study evaluates an ESP textbook for Islamic Education students using Cunningsworth's and Miekley's checklists. Results show the textbook is adequate overall but focuses heavily on vocabulary enrichment, with recommendations for improvement addressing technological advancements.
Original abstract
Abstract. The crucial presence of instructional materials at English classes, especially English for Specific Purposes, has commonly been approved. ESP materials provide exposure to the genres needed by the students, including those at Islamic Education Department. It is, therefore, inevitable that the textbook meets some criteria for the success of the program. In this paper, the writers evaluated an ESP textbook institutionally prepared for students of Tarbiyah and Teacher Training Faculty at IAIN Tulungagung. The analysis in this qualitative descriptive research was based on Cunningsworth’s and Miekley’s checklists for textbook evaluation. The book was evaluated on the category of aims and approaches, design and organization, content, skills, topic, methodology, vocabulary and grammar, exercises and activities, attractiveness of the text and physical make up, context, and practical considerations. The results showed that almost all categories are marked as adequate. It is indicated that the textbook mainly focused on vocabulary enrichment. Some recommendations for textbook improvement and revision are given at the final part of this study, particularly those related to the challenge on the advanced of technology. Keywords. ESP; Textbook Evaluation; English for Islamic Insight