textbook-analysis
Filtering by topic textbook-analysis(6)Clear all filters
- PaperEdArXiv (OSF Preprints)20 Jul 2026
Beyond the Average: Analyzing Heterogeneity and Latent Profiles of Cognitive Involvement in Iranian Senior High School Sociology Textbooks
Hossein Talebzadeh
This study analyzed cognitive involvement in Iranian high school sociology textbooks, revealing three latent profiles: fully passive (35.3% of lessons), hyper-active, and a persistent mismatch between passive text and demanding end-of-chapter tasks. The findings highlight the need for structural alignment between content and assessment in textbook design.
Original abstract
Previous evaluations of school textbooks have predominantly focused on aggregate overall averages, thereby overlooking critical intra-textbook heterogeneity. This study aimed to identify latent profiles of cognitive involvement and assess the instability of instructional design within Iranian senior high school sociology textbooks. Utilizing a secondary data analysis of datasets derived from the Comparative Adjudication Protocol (CAP)—encompassing 51 lessons across 4 core textbooks (Sociology 1, Sociology 2, Sociology 3, and Civic Identity)—cognitive involvement indices for text, images, and questions were calculated using the William Romey (1968) method. To quantify internal heterogeneity, the Coefficient of Variation (CV) and Levene's test were utilized, while K-means clustering with Euclidean distance (Hair et al., 2010) was employed to uncover latent profiles. The findings revealed that the Sociology 2 textbook exhibited the highest textual irregularity (CV = 122.45%), whereas the Civic Identity textbook possessed the most uniform text structure (CV = 36.52%). Levene's test confirmed no statistically significant difference in variances across the textbooks for any component (p > 0.05), indicating a more structurally homogeneous design across sociology textbooks compared to history textbooks. However, cluster analysis uncovered three distinct structural profiles, revealing that 35.3% of the analyzed lessons fell into a "fully passive" cluster, while 21.6% were classified as "hyper-active." These findings demonstrate that while sociology textbooks exhibit lower overall heterogeneity than history textbooks, a persistent question-text disjuncture remains—where end-of-chapter tasks demand high analytical engagement while the textual content remains predominantly passive. Consequently, future curricular interventions must prioritize structural alignment between narrative content and assessment tasks, alongside variance reduction across grade levels.
- PaperEdArXiv (OSF Preprints)20 Jul 2026
Beyond the Average: Analyzing Heterogeneity and Latent Profiles of Cognitive Involvement in Iranian Senior High School History Textbooks
Hossein Talebzadeh
This study analyzes latent profiles of cognitive involvement in Iranian senior high school history textbooks, revealing significant internal heterogeneity. The History 3 textbook showed the highest instructional irregularity, while only 11.5% of lessons were structurally balanced. The findings suggest that reducing content variance is more critical than raising average cognitive involvement.
Original abstract
Previous evaluations of school textbooks have predominantly focused on aggregate overall averages, thereby overlooking critical intra-textbook heterogeneity. This study aimed to identify latent profiles of cognitive involvement and assess the instability of instructional design within Iranian senior high school history textbooks. Utilizing a secondary data analysis of datasets derived from the Comparative Adjudication Protocol (CAP)—encompassing 61 lessons across 4 core textbooks—cognitive involvement indices for text, images, and questions were calculated using the William Romey (1968) method. To quantify internal heterogeneity, the Coefficient of Variation (CV) and Levene’s test were utilized, while K-means clustering with Euclidean distance (Hair et al., 2010) was employed to uncover latent profiles. The findings revealed that the History 3 textbook exhibited the highest instructional irregularity ("CV"=144.50% in the question component), whereas the Contemporary History of Iran textbook possessed the most uniform structure ("CV"=81.58%). Levene’s test confirmed a statistically significant difference in variances across the textbooks specifically within the question component (p<0.01). Furthermore, cluster analysis uncovered four distinct structural profiles, revealing that a meager 11.5% of the analyzed lessons fell into a structurally "balanced" cluster. These findings demonstrate that internal content dispersion, particularly in higher grade levels, poses a more severe pedagogical challenge than low average involvement. Consequently, future curricular interventions must prioritize variance reduction over a simple inflation of baseline averages.
- PaperEdArXiv (OSF Preprints)20 Jul 2026
Beyond the Average: Analyzing Heterogeneity and Latent Profiles of Cognitive Involvement in Iranian Junior High School Social Studies
Hossein Talebzadeh
This study analyzed cognitive involvement heterogeneity in Iranian junior high school social studies textbooks using the William Romey method and cluster analysis. Three latent profiles were identified: Active-Question Dominant, Fully Passive, and Hyper-Active, with Grade 7 showing extreme textual irregularity. The findings highlight a mismatch between passive textual content and demanding assessment tasks, suggesting a need for structural alignment.
Original abstract
Previous evaluations of school textbooks have predominantly focused on aggregate overall averages, thereby overlooking critical intra-textbook heterogeneity. This study aimed to identify latent profiles of cognitive involvement and assess the instability of instructional design within Iranian junior high school social studies textbooks. Utilizing a secondary data analysis of datasets derived from the Comparative Adjudication Protocol (CAP)—encompassing 44 lessons across 2 core textbooks (Social Studies Grade 7 and Social Studies Grade 8)—cognitive involvement indices for text, images, and questions were calculated using the William Romey (1968) method. To quantify internal heterogeneity, the Coefficient of Variation (CV) and Levene's test were utilized, while K-means clustering with Euclidean distance (Hair et al., 2010) was employed to uncover latent profiles. The findings revealed that the Social Studies Grade 7 textbook exhibited the highest textual irregularity (CV = 186.84%), whereas the Social Studies Grade 8 textbook demonstrated more moderate text heterogeneity (CV = 134.78%). Levene's test confirmed a statistically significant difference in text variances across the two textbooks (p = 0.028), indicating that Grade 7 social studies is structurally more fragmented than Grade 8. However, cluster analysis uncovered three distinct structural profiles, revealing that 50.0% of the analyzed lessons fell into an "Active-Question Dominant" cluster, while 31.8% were classified as "Fully Passive" and 18.2% as "Hyper-Active." These findings demonstrate that middle school social studies textbooks—particularly at Grade 7—exhibit extreme textual heterogeneity and a persistent question-text disjuncture, where end-of-chapter tasks demand high analytical engagement while the textual content remains predominantly passive. Consequently, future curricular interventions must prioritize structural alignment between narrative content and assessment tasks, alongside systematic reduction of textual variance across grade levels.
- PaperEdArXiv (OSF Preprints)17 Jul 2026
Distributed Auditing of Cognitive Balance in History Textbooks: Application of the CAP Protocol within a Pre-Service Teacher Network
Hossein Talebzadeh
An analysis of upper secondary Iranian history textbooks using the Comparative Adjudication Protocol (CAP) with 43 pre-service teachers found extremely low cognitive engagement, with a text involvement index of 0.019 (far below the active learning threshold of 0.4) and zero image involvement. The CAP protocol achieved 86.4% final inter-rater agreement, demonstrating reliability for distributed auditing, though distinguishing objective facts from author interpretation remained the primary challenge.
Original abstract
Background: Despite theorists like Sam Wineburg emphasizing the development of historical thinking, history textbooks in centralized educational systems continue to be dominated by rote-learning approaches, leaving the systematic evaluation of their cognitive balance—particularly at the upper secondary level—largely neglected. Objective: This study aims to analyze the cognitive balance of upper secondary history textbooks (History 1, 2, 3, and Contemporary History of Iran) and evaluate the efficacy of the Comparative Adjudication Protocol (CAP) within a network of human auditors. Methodology: Engaging 43 pre-service teachers divided into 15 independent research teams, this study analyzed the content of 4 textbooks comprising 61 independent lessons using the William Romey technique and the CAP protocol. The process was executed across four distinct phases: capacity building, independent AI-assisted auditing, blind peer auditing, and discrepancy adjudication. Findings: The overall text involvement index was exceptionally low (0.019), far below the active learning threshold of 0.4, with 96.7% of the lessons falling into the passive "banking education" category. The image involvement index was absolute zero (0), and while the question involvement index was 1.29 (with an unbalanced distribution ranging from ∞ to 0.06), it revealed a severe structural gap between passive textual content and active analytical expectations. The CAP protocol demonstrated high reliability, achieving an 86.4% final inter-rater agreement and yielding a 12.5% increase in initial consensus. The dominant discrepancy pattern (A vs. B at 48.3%) indicated that distinguishing "objective historical facts" from "author interpretation" remains the primary analytical challenge for human coders in historical texts. Conclusion: Despite their narrative essence, high school history textbooks exhibit a highly passive cognitive structure lacking a coherent strategy to foster active student engagement. Implementing the CAP protocol within the R2A-TACI-PACT conceptual framework underscores the vital necessity of a "Human-in-the-Loop" (HITL) approach in historical content analysis. Curriculum revision and empowering pre-service teachers with algorithmic auditing literacy are highly recommended.
- PaperTESOL Journal29 Jun 2026
The Case for Quantitative Frameworks in Assessing Racial and Ethnic Representation in ELT Materials
Derek Hopper
This position paper addresses the methodological gap in research on racial and ethnic representation in English Language Teaching (ELT) materials, which has largely relied on qualitative assessments. It proposes three quantitative frameworks—publisher-country demographics, global demographics, and target market demographics—to provide measurable benchmarks and improve rigor in representation studies. The paper argues that adopting transparent, quantitative criteria would benefit researchers, publishers, and educators by moving beyond circular critique toward measurable improvement.
Original abstract
This position paper addresses a critical methodological gap in research on racial and ethnic representation in English Language Teaching (ELT) materials. Despite decades of criticism regarding diversity in textbooks, researchers have largely relied on qualitative assessments without establishing quantitative frameworks for measuring representation. This absence of measurable benchmarks has led to inconsistent expectations, ambiguous claims, and limited practical guidance for publishers and educators. Through analysis of existing literature, this paper demonstrates how the lack of quantitative definitions undermines both the analytical rigor of representation research and the practical application of its findings. The paper proposes three quantitative frameworks for assessing representation: publisher‐country demographics (reflecting the demographic composition of the publishing country), global demographics (mirroring world population statistics), and target market demographics (tailored to the demographics of primary user markets). Each approach offers distinct advantages and limitations, with dramatically different implications for what constitutes “appropriate representation.” These frameworks would yield substantially different outcomes, highlighting why explicit quantitative standards are essential. The paper argues that adopting transparent, measurable criteria would benefit multiple stakeholders by providing researchers with clearer parameters for empirical studies, offering publishers concrete guidelines for inclusive material development, and equipping educators with informed evaluation tools. Rather than prescribing a single correct approach, this work advocates for methodological transparency and precision to move the field beyond circular critique toward measurable improvement in representation practices.
- PaperELT Journal26 Jun 2026
Where is learner agency in locally produced ELT textbooks?
Amber N Warren, Natalia A Ward, Shuling Yang, Gokce Unlu et al.
A multimodal analysis of elementary ELT textbooks from China, Egypt, Russia, and Türkiye reveals that while textbooks position learners as active and culturally situated, agency is pedagogically managed and bounded, limiting open-ended meaning-making. The study proposes a teacher toolkit to help educators notice curricular modification opportunities and invite more expansive learner agency.
Original abstract
This study examines how locally produced elementary-level ELT textbooks from China, Egypt, Russia, and Türkiye frame opportunities for learner participation and construct learner agency. Focusing on curricular architecture across three dimensions—multimodal and multilingual resources, localisation of contexts and content, and global citizenship and intercultural participation—we demonstrate how these textbooks structure what learners are invited to do, say, and become in classroom contexts. The findings indicate that while the materials frequently position learners as active and culturally situated participants, the forms of agency they construct are largely pedagogically managed and bounded. As a result, learner participation is often guided toward predetermined outcomes, limiting opportunities for open-ended meaning-making, critical engagement, and more expansive expressions of agency. We propose a teacher toolkit designed to support teachers in noticing opportunities for curricular modification and invite learner agency.