diagnostic-assessment
Filtering by topic diagnostic-assessment(2)Clear all filters
- PaperLanguage Testing26 Jun 2026
Developing and Validating a Computerized Dynamic Diagnostic Assessment of Pragmatic Competence for Chinese Learners of English
Qi Lu, Ying Chen, Lianrui Yang
A computerized dynamic diagnostic assessment (CDDA-P) was developed and validated for assessing pragmatic competence in Chinese learners of English, using refusals as an example. The assessment empirically derived response options from a learner corpus and provided fine-grained diagnosis of strengths/weaknesses, identified zones of proximal development, and promoted significant improvement in learner performance.
Original abstract
Recent initiatives have sought to integrate dynamic assessment and diagnostic assessment to facilitate language learning. Extending this line of innovation, the present study uses the example of refusals to illustrate the development and validation of a computerized dynamic diagnostic assessment of pragmatic competence (CDDA-P). To enhance the authenticity of assessed pragmatic performance, a bottom-up approach was adopted in item design, in which response options were empirically derived from a corpus of productions by 335 language users. The effectiveness of the CDDA-P was examined from three key perspectives: its ability to diagnose learners’ strengths and weaknesses, to identify their zones of proximal development (ZPDs) and learning potential, and to promote the development of learners’ pragmatic performance. A pretest–immediate posttest-delayed posttest design was employed to track changes in 66 Chinese learners’ performance before and after the implementation of the CDDA-P. Findings reveal that the CDDA-P can provide a fine-grained diagnosis of learners’ strengths and weaknesses in performing L2 refusals and identify their diverse ZPDs and learning potential. Furthermore, learners demonstrated significant improvement after mediation. This study presents both theoretical and methodological implications for the development of integrated dynamic and diagnostic language assessment, while also offering insights into promoting L2 pragmatic competence through assessment.
- PaperComputers and Education: Artificial Intelligence6 Jun 2026
Personalized neural cognitive architecture search: AutoML-driven diagnostic model generation for heterogeneous learner profiles
Liyuan Jia, Ke Dong
This study proposes a personalized neural cognitive architecture search framework driven by automated machine learning to generate diagnostic models for heterogeneous learner profiles. The framework integrates multi-modal educational data (cognitive states, behavioral sequences, contextual factors) using heterogeneous information network technology and automatically searches for optimal diagnostic model architectures. On 28.45 million educational records, it achieves 89.1% AUC, outperforming traditional psychometric and deep learning approaches, though challenges remain in computational efficiency and fairness.
Original abstract
With the rapid advancement of digital transformation in education, personalized learning has become a central objective for improving educational quality. However, traditional educational models struggle to effectively process multi-source heterogeneous learning behavior data, resulting in limited learner profiling and inefficient diagnostic model development. In addition, existing teaching quality evaluation systems mainly rely on static examination outcomes and simple statistical indicators, lacking the capability to dynamically analyze learning processes. To address these challenges, this study proposes a personalized neural cognitive architecture search framework driven by automated machine learning. The framework integrates multi-modal educational data using heterogeneous information network technology to construct dynamic learner profiles incorporating cognitive states, behavioral sequences, and contextual factors. A hierarchical neural architecture search space is then designed, combining differentiable search strategies with a Bayesian performance predictor to automatically identify optimal diagnostic model structures. Experimental results based on 28.45 million educational records show that the proposed method achieves an AUC of 89.1%, accuracy of 85.3%, and F1-score of 84.7%. The model significantly outperforms traditional psychometric approaches and surpasses mainstream deep learning and AutoML baselines. Ablation experiments further confirm the importance of multi-source data fusion and automated architecture search. Despite strong performance, challenges remain regarding computational efficiency and fairness validation across diverse learner populations.