l2-research
Filtering by topic l2-research(1)Clear all filters
- PaperTESOL Quarterly7 Jul 2026
Rethinking Quality in L2 Research Synthesis: Collective Initiative Toward Developing Validated Tools to Assess Methodological Rigor of Primary Studies
Yushi Kashimura
This paper identifies critical gaps in current appraisal tools for methodological rigor in second language (L2) research syntheses, including a lack of field specificity, limited coverage of designs, and absence of open science principles. It proposes a collective initiative using the Delphi method to develop and validate field-specific tools for assessing primary study quality.
Original abstract
This Research Issues paper calls for the development and validation of field‐specific appraisal tools for the methodological rigor of primary studies included in research syntheses. The methodological quality of primary studies serves as a foundation for conducting robust syntheses, interpreting their findings, and deriving empirically grounded implications for theory, methodology, and practice. However, although the evaluation of methodological rigor using appraisal tools is strongly encouraged, such tools currently available to second language (L2) researchers are limited in at least five aspects: (a) they are not specific to L2 studies, (b) they offer limited coverage of the range of research designs, analytic approaches, and subfields represented in primary studies, (c) their developmental and validation processes lack transparency, (d) practical guidance for their application is insufficiently defined, and (e) explicit consideration of open science principles is absent. In light of these limitations, the L2 research community currently lacks validated field‐specific tools to support researchers engaged in secondary research. To address this gap, the present paper calls for a collective initiative to develop and validate such tools. Specifically, it argues that the Delphi method represents a promising approach for systematically establishing expert consensus on the methodological criteria required for rigorous appraisal.