language-testing
Filtering by topic language-testing(16)Clear all filters
- PaperIELTS Partnership Research Reports
A concordance study of CELPIP General and IELTS General Training
A concordance analysis of the CELPIP General and IELTS General Training tests was conducted to evaluate score alignment between the two assessments.
- PaperIELTS Partnership Research Reports
Safeguarding equity, access and inclusion in IELTS
This paper examines strategies to ensure fairness and inclusivity in the IELTS test, addressing potential biases and barriers for diverse test-takers.
- PaperIELTS Partnership Research Reports
Aligning scores of language proficiency tests: A score concordance study between IELTS Academic and TOEFL iBT
This study establishes score concordance tables between IELTS Academic and TOEFL iBT, enabling direct comparison of proficiency levels across the two tests.
- PaperLanguage Testing19 Jul 2026
Outcomes of One-Skill Retakes in a Four-Skills Proficiency Test: Evidence From Large-Scale Test Data
Hye-won Lee, Emma Bruce, Jan Langeslag, Reza Tasviri
The study analyzes over 20,000 test takers using IELTS One Skill Retake (OSR) and finds that average retake scores were higher than original scores, with overall band changes similar to short-interval full-test repeaters. Survey data from 578 test takers indicates factors like insufficient preparation, stress, and fatigue contributed to initial underperformance. The findings provide empirical evidence for interpreting one-skill retake scores and inform discussions on validity and equity in large-scale language testing.
Original abstract
A test taker may underperform for reasons not fully attributable to language proficiency, including psychological or contextual influences such as anxiety or illness. IELTS One Skill Retake (OSR) was launched in 2022, allowing test takers to retake, within 60 days, a single component in which their initial performance may have been affected by extenuating circumstances. This study examines outcomes associated with OSR by analysing test-taking patterns and score changes among over 20,000 OSR test takers from its launch through Spring 2024. It also reports survey findings from 578 OSR test takers on their experiences, including whether they achieved target scores and their perceived reasons for not achieving the desired score on the original full test. Across skills, average OSR component scores were higher than the corresponding scores on the original full test, and overall-band changes among OSR test takers were comparable to those observed among short-interval full-test repeaters (⩽60 days). Survey responses commonly cited factors such as insufficient preparation, stress and anxiety, and fatigue and lack of focus as perceived contributors to underperformance on the initial test. The findings contribute empirical evidence relevant to the interpretation and use of one-skill retake scores and to ongoing discussions of the validity and equity implications of retake policies in large-scale language testing.
- PaperLanguage Testing19 Jul 2026
Can an AI Agent Replace Human Examiners in High-Stakes Interactive Speaking Tests? A Debate
Jing Xu, Lynda Taylor, Xiaoming Xi, Yasin Karatay et al.
This viewpoint article presents arguments for and against replacing human examiners with AI agents in high-stakes interactive speaking tests, drawing on a debate at the 2025 LTRC. It examines construct theory, practicality, washback, fairness, and ethical uses of AI in language assessment.
Original abstract
Generative artificial intelligence (GenAI) is advancing at a remarkable speed, and its promise in transforming current practice in language assessment has been articulated by applied linguistics researchers. An emerging application of GenAI is to integrate the technology into Spoken Dialogue Systems (SDSs) to simulate human interlocutors for the purpose of speaking practice or assessment. Despite rapid technological advances, the issue of whether an AI agent can replace a human examiner in one-on-one, high-stakes interactive speaking tests remains contentious. Building on a lively academic debate on this topic at the 2025 Language Testing Research Colloquium (LTRC) in Bangkok, this Viewpoint presents arguments both for and against this proposition in terms of construct theory, practicality, washback, fairness, ethical uses of AI, and so forth.
- PaperReCALL13 Jul 2026
The effectiveness of computerized dynamic assessment in improving L2 performance: A three-level meta-analysis
Qi Lu, Mengqi Chen, Lianrui Yang, Shaofeng Li et al.
A three-level meta-analysis of 35 studies (2000-2025) found that computerized dynamic assessment (C-DA) significantly improves L2 performance, with both cake-format (mediation embedded in tests) and sandwich-format (mediation between pretest and posttest) designs yielding large positive effects. The cake format showed larger effect sizes, and effectiveness was moderated by number of items, test content, and learners' first language.
Original abstract
The growing body of research on the effects of computerized dynamic assessment (C-DA) on second language (L2) learning underscores the need for a comprehensive research synthesis to identify future research directions and inform the application of C-DA in L2 educational contexts. This meta-analysis employed a three-level modeling approach to examine the effectiveness of C-DA in improving L2 learners’ performance. It synthesized 27 effect sizes from cake format designs, in which mediation is embedded within the test sequence, and 24 effect sizes from sandwich format designs, where mediation is delivered between a pretest and a posttest, across 35 studies published between 2000 and May 27, 2025. This study also investigated the key variables that moderate C-DA effectiveness. Findings reveal large, significant positive effects of both the cake and sandwich formats on L2 performance improvement (cake format: g = 2.120, p < .001; sandwich format: g = 1.676, p < .001), with the cake format tending to yield larger effect sizes. This may be because the cake format captures gains during mediation, whereas the sandwich format reflects post-mediation outcomes. Moderator analyses show that the number of items, test content, and learners’ first language affect C-DA effectiveness in promoting L2 performance. Drawing on the synthesized findings, this study contributes to theoretical, methodological, and technological understandings of C-DA and offers suggestions for future research in this domain.
- PaperLanguage Testing2 Jul 2026
Test Review: The Test of Proficiency in Korean
So-Young Lim, John Dylan Burton
This test review provides an overview of the TOPIK's history, purposes, design, and administration, and appraises its strengths and challenges. The lack of publicly available information on test construct, psychometric properties, and standard-setting procedures makes it difficult to fully evaluate score validity and reliability. Additional validation research is needed to examine score generalizability beyond academic domains.
Original abstract
As a nationally accredited test, the Test of Proficiency in Korean (TOPIK) assesses general Korean proficiency as a foreign language for various high-stakes purposes, such as university admissions, employment, and visa issuance. For this reason, the social impact of TOPIK on test takers is significant and cannot be underestimated. Despite the growing number of international test takers and stakeholders, there is limited validation research evaluating TOPIK’s various uses, as well as critical evaluation of the test itself. Thus, this test review provides an overview of the history, test purposes and use, design, and administration of TOPIK and offers an appraisal of its strengths and challenges. While the test scores are broadly utilized for their intended purposes and provide some evidence of language development across four skills in Korean, the lack of publicly available information on the test construct, psychometric properties, and standard-setting procedures makes it difficult to fully evaluate the validity and reliability of the scores. Furthermore, additional evidence and validation research are needed to examine the generalizability of TOPIK scores beyond academic domains, given the test’s diverse applications. Addressing these gaps is critical to meet the needs of various stakeholders and to strengthen the overall validity of the test.
- PaperDOAJ — Language assessment1 Jul 2026
The Design of Language Testing and Evaluation Materials for the English Department
Irra Wahidiyati, Windhariyati Dyah K, Ghaida Thifal
A needs analysis of English department students and lecturers informed the design of a 10-chapter textbook on language testing and evaluation. The material covers assessment concepts, principles, and specific techniques for listening, speaking, reading, writing, grammar, and vocabulary. The resulting resource aims to improve Language Assessment Literacy by integrating theory, test construction, and rubric development.
Original abstract
Background - The students of Tadris Bahasa Inggris need explicit material about assessments and tests. They need the material to develop listening and speaking skills. Next, the TBI students must master the creation of writing and reading test items. In addition to creating writing and reading test items. Urgency of Research – The needs analysis shows that students’ needs regarding the Language Testing and Evaluation subjects in the English Department of UIN Prof. K. H. Saifuddin Zuhri Purwokerto. They need information on how to create assessment rubrics for them. The learning materials used are compiled to meet the students' needs. Research Objectives – This study conducts a needs analysis of students' needs regarding the Language Testing and Evaluation subject and designs new material for it. Research Method - A mixed method was used to answer the research questions. The researcher analyzed the institution's problems and assessed whether students' needs aligned with the syllabus or lesson plan. The information sources were 100 students and 2 lecturers. Research Findings – The draft of the material consists of 7 chapters. Chapter 1 discusses assessment concepts and issues. Chapter 2 discusses the principles of language assessment. Chapter 3 discusses the design of classroom language tests and standardized testing. Chapter 5 discussed the assessment of listening, including intensive, responsive, selective, and extensive listening. Chapter 6 discusses assessing speaking, including imitative, intensive, responsive, interactive, and extensive speaking. Chapter 7 discusses the assessment of reading, including perceptive, selective, interactive, and extensive reading. Chapter 8 discusses assessing writing, including imitative, intensive, responsive, and extensive. Chapter 9 discusses the assessment of grammar and vocabulary. The last chapter discusses the grading and evaluation process. Research Conclusion & Novelty - The materials integrate assessment theory, test construction, and rubric development for all language skills into a single contextualized resource designed to promote Language Assessment Literacy.
- PaperLanguage Teaching1 Jul 2026
LTA volume 59 issue 3 Cover and Back matter
This item is the cover and back matter of Language Testing journal volume 59 issue 3, containing non-article administrative content.
- PaperDOAJ — Language assessment1 Jul 2026
A LINEAR LOGISTIC TEST MODEL (LLTM) APPLICATION IN FOREIGN LANGUAGE TESTING
Jose Fabián Elizondo-González, Peyman Jahanbin
This study applies the Linear Logistic Test Model (LLTM) to a reading comprehension subtest, finding that a Q-matrix of cognitive predictors (e.g., inferences, subtask complexity) explains 77% of item difficulty variance. The model enhances interpretability of item difficulty and identifies features for test refinement.
Original abstract
This study applies the Linear Logistic Test Model (LLTM) to the reading comprehension subtest of an English certification exam developed by the Foreign Language Assessment Program (PELEx, for its acronym in Spanish). A Q-matrix operationalized cognitive and linguistic predictors, such as paraphrasing and inferential reasoning, to explain item difficulty. Delta-squared results showed that the Q-matrix accounted for 77% of the variance in item difficulty, with “Inferences” and “Subtask Complexity” being key contributors. While overlaps in item difficulty coefficients reflected the nested nature of the Common European Framework of Reference for Languages (CEFR) levels, this progression aligns with the framework’s principles. The findings show that LLTM can make item difficulty more interpretable in reading assessment, while also helping to identify which item features merit further refinement in future test development.
- PaperLanguage Testing25 Jun 2026
Reflections on the Practical Implementation of Knoch and Fan’s (2024) Good Practice Principles for Score Concordance Studies
Spiros Papageorgiou, Tony Clark
Reviews the practical implementation of Knoch and Fan's (2024) good practice principles for score concordance studies, drawing on experience from a large-scale concordance study comparing IELTS Academic and TOEFL iBT scores. Emphasizes methodological rigor, transparency, construct comparability, and limitations of concordance tables for admissions decisions. Offers recommendations for fair score requirements regardless of test choice.
Original abstract
When different tests are used for the same purpose, score requirements should be comparable so that examinees cannot obtain an unfair advantage simply because of the test they chose. Drawing on our experience conducting a large-scale concordance study to allow for an empirical comparison of IELTS Academic and TOEFL iBT test scores, we review Knoch and Fan’s evaluative framework, explore methodological best practices and challenges, and offer future directions for score concordance research. We emphasize the importance of methodological rigor in collecting test-taker score data, transparency in analyzing such data to build score concordance tables, and a reasonable degree of construct comparability as a prerequisite for conducting a score concordance study, while also highlighting the limitations of concordance tables as standalone tools for admissions decisions. We note that some aspects of Knoch and Fan’s good practice principles are more straightforward to implement in practice than others. The good practice principles could be updated or adjusted after real-world application, which we describe with a view to furthering best practice in concordance research. We conclude this viewpoint with recommendations for decision-making that are based on fair score requirements, irrespective of which test the examinees chose.
- PaperLanguage Testing25 Jun 2026
Concordance or Discordance? The Broader Context of English Tests for Australian Immigration
John Read
The paper examines the alignment or misalignment of English language tests used for Australian immigration within their broader socio-political context.
- PaperLanguage Testing10 Jun 2026
Beyond Traditional Differential Item Functioning Detection: A Rasch Tree Approach to Evaluating Item Fairness in a Large-Scale German Reading Comprehension Test
Farshad Effatpanah, Olga Kunina-Habenicht, Katharina Antonia Michiko Tremmel, Philipp Sonnleitner
This study applies a recursive partitioning Rasch tree model to detect differential item functioning (DIF) in a large-scale German reading comprehension test for fifth graders, examining effects of gender, socioeconomic status, immigration status, personality, and need for cognition. Unlike traditional methods, the Rasch tree does not require pre-specified groups and can handle continuous covariates. Analysis of 4,252 students revealed nine non-predefined nodes, with eleven items showing moderate to large DIF, indicating that combinations of covariates impact test performance.
Original abstract
A key psychometric phenomenon in educational testing is differential item functioning (DIF), which evaluates whether specific test items function differently across subgroups of examinees who have the same level of the underlying (latent) ability. DIF happens when examinees with the same latent trait have different probabilities of correctly responding to a test item, influenced by their subgroup membership. This study aims to illustrate the application of the recursive partitioning Rasch tree model, shortly called the Rasch tree, to examine DIF in a large-scale German reading comprehension test for fifth-grade elementary students across gender, socioeconomic status, immigration status, personality disposition, and the need for cognition. Unlike the conventional DIF detection methods, the Rasch tree does not require a pre-specification of groups for exploring DIF, and continuous covariates can be easily included in the analysis. To investigate DIF of the test, item responses of 4,252 students were analyzed. The Rasch tree analysis generated nine non-predefined nodes, with slightly different patterns of item difficulties. Eleven items were flagged as exhibiting moderate and large DIF in the four splitting nodes. No splits were produced based on the need for cognition. The results indicated that the combination of the covariates impacted students’ test performance.
- PaperAssessing Writing2 Jun 2026
Examining the relevance of three TOEFL Essentials writing tasks to the accounting profession: The role of domain experts
Ute Knoch, Jason Fan, Michael Davey, Sally O’Hagan et al.
Domain experts (accountants) evaluated three TOEFL Essentials writing tasks for relevance to accounting. The Build a Sentence task was seen as least relevant, while Write an Email was most relevant, but experts focused on a narrow range of features and sometimes misinterpreted task demands.
Original abstract
Large-scale English language proficiency tests are increasingly used to make decisions about professional registration despite not originally being developed to make predictions about language use in the workplace. Domain experts can play a valuable role as informants in establishing the relevance of test tasks to a specific TLU domain. However, this practice has seldom been critically examined. In particular, no studies to date have examined the nature of the interview questions used when engaging with domain experts in this type of research. The current study was designed with two aims: to (1) explore the relevance of three writing tasks (i.e., Build a Sentence, Write for an Academic Discussion, Write an Email) from the TOEFL Essentials test to the accounting profession and (2) evaluate the judgements of domain expert participants. Twenty accountants from non-English speaking backgrounds as well as three accounting educators were interviewed for the study, drawing on a methodology with broad, general questions. The data was analysed qualitatively to identify (a) to what extent the participants considered the three tasks relevant and (b) what task features they attended to when commenting on the relevance of the tasks. The findings showed that the participants generally found the Build a Sentence the least relevant of the three writing tasks, and the Write an Email task the most relevant. When reviewing the task features the participants judged as relevant to writing demands in their workplace, it was shown they focused on a small/narrow range of features. They also engaged in ‘misinterpretations’, comparing aspects of test and workplace tasks that did not align (e.g., comparing a writing task to events in a spoken meeting). The findings are discussed in terms of domain expert involvement in validation research.
- PaperLanguage Testing21 May 2026
Book Review: Language Testing and Assessment: From Theory to Practice PhakitiA., Language Testing and Assessment: From Theory to Practice. Bloomsbury Academic, 2025. 312 pp. ISBN 978 1 47429 012 8 (paperback), £22.49 (paperback), £67.50 (hardback), £22.49 (eBook)
Niles Yan Zhao
This book review examines Phakiti's 2025 work on language testing and assessment, which bridges theoretical foundations with practical applications for TESOL professionals.
- PaperRELC Journal7 May 2026
Assessing the impact of the Common European Framework of Reference for Languages on policy and testing practices in Thai higher education
Anchana Rukthong, Punjaporn Pojanapunya, Somruedee Khongput
This study analyzed the implementation of CEFR-based English Exit Exams at 12 Thai universities, finding inconsistent skill coverage and unclear alignment of scores with CEFR levels, which led to doubts about exam quality among practitioners. While 84% of surveyed stakeholders supported the policy for raising learner awareness and motivation, opponents questioned whether test scores accurately reflected actual communicative ability. The authors conclude that English Exit Exams alone may not effectively drive CEFR-informed classroom practice.
Original abstract
The application and impact of the Common European Framework of Reference for Languages have been observed worldwide, especially in testing practices, since its introduction in 2001. This is partly because Common European Framework of Reference-based policies have often used the framework's proficiency scales as a key indicator of success. Although the Common European Framework of Reference as policy is expected to improve current practices, there appear potential challenges that could undermine its effectiveness in particular contexts. This study reports on the case of the Common European Framework of Reference-based policy in Thai higher education, where an English Exit Exam has been imposed to verify the proficiency of graduates. To obtain a clear picture of how the policy has been implemented, the study analysed 12 sets of English Exit Exam materials from 12 institutions, 44 questionnaire responses from university lecturers and administrators, and 17 online interviews with practitioners who were involved in the development of English Exit Exams. The results show that the English Exit Exams of different institutions focused on assessing different skills, sub-skills and linguistic knowledge, with exams including listening, reading, vocabulary and grammar sections; however, how the exam results were aligned with the Common European Framework of Reference scales remained unclear despite the framework's centrality in policy, and this made about half of the interview participants doubt the quality of the English Exit Exam. Eighty-four percent of the questionnaire respondents supported the policy, considering it a tool for raising learners’ awareness and motivation in learning English. The opponents, however, question the effectiveness of the English Exit Exam practices, being concerned that the test scores may not accurately indicate actual ability in English communication. The study concludes that although a test can theoretically serve as a mechanism to impose language policy, the endorsement of English Exit Exams alone may not effectively function as an instrument to drive the Common European Framework of Reference-informed practice at a classroom level.