differential-item-functioning
Filtering by topic differential-item-functioning(2)Clear all filters
- PaperLanguage Testing10 Jun 2026
Beyond Traditional Differential Item Functioning Detection: A Rasch Tree Approach to Evaluating Item Fairness in a Large-Scale German Reading Comprehension Test
Farshad Effatpanah, Olga Kunina-Habenicht, Katharina Antonia Michiko Tremmel, Philipp Sonnleitner
This study applies a recursive partitioning Rasch tree model to detect differential item functioning (DIF) in a large-scale German reading comprehension test for fifth graders, examining effects of gender, socioeconomic status, immigration status, personality, and need for cognition. Unlike traditional methods, the Rasch tree does not require pre-specified groups and can handle continuous covariates. Analysis of 4,252 students revealed nine non-predefined nodes, with eleven items showing moderate to large DIF, indicating that combinations of covariates impact test performance.
Original abstract
A key psychometric phenomenon in educational testing is differential item functioning (DIF), which evaluates whether specific test items function differently across subgroups of examinees who have the same level of the underlying (latent) ability. DIF happens when examinees with the same latent trait have different probabilities of correctly responding to a test item, influenced by their subgroup membership. This study aims to illustrate the application of the recursive partitioning Rasch tree model, shortly called the Rasch tree, to examine DIF in a large-scale German reading comprehension test for fifth-grade elementary students across gender, socioeconomic status, immigration status, personality disposition, and the need for cognition. Unlike the conventional DIF detection methods, the Rasch tree does not require a pre-specification of groups for exploring DIF, and continuous covariates can be easily included in the analysis. To investigate DIF of the test, item responses of 4,252 students were analyzed. The Rasch tree analysis generated nine non-predefined nodes, with slightly different patterns of item difficulties. Eleven items were flagged as exhibiting moderate and large DIF in the four splitting nodes. No splits were produced based on the need for cognition. The results indicated that the combination of the covariates impacted students’ test performance.
- PaperERIC — Assessment & second language1 Jan 2025
A Systematic Review of Differential Item Functioning in Second Language Assessment
Xueliang Chen, Vahid Aryadoust, Wenxin Zhang
This systematic review of 83 articles found that differential item functioning (DIF) analysis in second language (L2) assessment primarily relies on classical methods like Rasch, Mantel-Haenszel, and IRT, with emerging approaches such as cognitive diagnostic models also appearing. Most studies focused on manifest grouping variables (e.g., gender, language background) and receptive skills, and often lacked empirical justification for DIF causes, highlighting the need for improved practices and broader consideration of test-taker diversity.
Original abstract
The growing diversity among test takers in second or foreign language (L2) assessments makes the importance of fairness front and center. This systematic review aimed to examine how fairness in L2 assessments was evaluated through differential item functioning (DIF) analysis. A total of 83 articles from 27 journals were included in a systematic review. The findings suggested that classical DIF techniques were dominant in use, particularly Rasch-based methods, the Mantel-Haenszel procedure, item response theory (IRT) approaches, logistic regression, and SIBTEST, but emerging methods such as DIF analysis based on cognitive diagnostic models were also identified. Most DIF studies examined manifest grouping variables such as gender and language background and were based on assessments of receptive language skills such as reading and listening comprehension. DIF analyses were mostly conducted in an exploratory fashion and causes of DIF were often justified on speculative rather than empirical grounds. In addition, the quality of DIF analyses was undermined by suboptimal reporting practices. Our results suggest the need to improve current DIF practices, to consider alternative DIF detection methods aligning with emerging views of measurement bias, and to adequately account for the heterogeneity of L2 test takers. The findings have implications for test design and use, fairness, and validity in L2 assessments.