llm
Filtering by topic llm(3)Clear all filters
- PaperarXiv — AI in Education (cs.CY)13 Jul 2026
LLM-Generated Design Problems for Assessing Higher-Order Thinking in Project-Based Learning
Ahmad D. Suleiman, Daqing Hou, Maliha Noushin Raida
This study introduces design problems (DPs)—concise, scenario-based prompts—to assess higher-order thinking in project-based learning. Surveys of 31 instructors and evaluations of 80 LLM-generated DPs showed that instructors value DPs but find creation effort a barrier, while LLMs can generate high-quality prompts. Student performance data indicated that DPs capture distinct aspects of higher-order thinking, with negligible correlation to traditional project grades.
Original abstract
Project-based learning (PjBL) is common in computing education, but traditional assessments of PjBL often fail to capture higher-order thinking (HOT), especially in transfer contexts. This study introduces "design problems" (DPs): concise, scenario-based prompts that require applying project concepts in new situations, to address this gap. We examined instructor perceptions, the ability of large language models (LLMs) to generate DPs, and student experiences. Surveys of 31 instructors, evaluation of 80 LLM-generated DPs, and student performance data showed that while instructors value DPs, creation effort is a barrier. LLMs helped by producing high-quality prompts with strong expert agreement. Students rated DPs from different LLMs similarly, and their performance on DP tasks showed negligible correlation with traditional project grades, suggesting DPs may capture distinct aspects of HOT. Keystroke data also suggested deeper cognitive engagement of students through planning and revision behaviors. Overall, DPs appear to be a useful complement to traditional assessments, especially in situations where AI use or collaboration may undermine individual learning.
- PaperComputers and Education: Artificial Intelligence30 Jun 2026
Enhancing domain adaptation of LLM via model composition in solving medical exam questions
Yicong Liang, Di Zou, Haoran Xie, Fu Lee Wang
This study applies a model composition method (CALM) to adapt a large language model for medical exam questions, creating MedCALM. MedCALM uses cross-attention between the anchor and augmented models to improve multiple-choice question answering. Results show that connecting the last few layers yields the best performance.
Original abstract
The advent of LLMs has sparked interest among educators across various fields, particularly in medical education. However, the current pretrained large language models (LLMs) can not be directly applied to answer medical exam questions, as they require adaptation to specific domains via further fine-tuning. Fine-tuning a pretrained large language model requires intensive computing resources, making it impractical for injecting new capabilities into the LLM. In this work, we investigate the framework of applying the approach Composition to Augment Language Models (CALM) for medical domain adaptation in answering real-world exam questions. In particular, we augment a pre-trained large language model by further fine-tuning it on an open-ended question-answering dataset related to medical exams to inject medical domain knowledge. We develop the model MedCALM to enhance medical question-answering performance by incorporating another individual LLM with a medical-specialized augmenting model. The experiment is conducted on a dataset related to real-world medical exam questions, and the experimental results demonstrate that MedCALM outperforms the compared baselines. The results show that utilizing the multi-head cross-attention module to connect two large language models improves performance in answering multiple-choice questions. In addition, selecting the position of layers for connection has a significant impact on the effectiveness when composing the anchor model and the augmenting model. Our experiment suggests that cross-attention between the last few layers of the two models achieves promising performance.
- PaperarXiv — Language & NLP (cs.CL)27 May 2026
VLMs May Not Globally Enhance Human Alignment over LLMs During Natural Reading
Jinzhou Wu, Zhengwu Ma, Jixing Li, Baoping Tang et al.
Comparing matched large language models (LLMs) and vision-language models (VLMs) under text-only conditions reveals that multimodal pretraining does not provide a universal advantage in aligning with human neural and eye-tracking data during natural reading. However, VLMs show selective improvement for sentences with strong visual semantic content, indicating that language-internal representations remain the primary driver of human-like text processing.
Original abstract
Large language models (LLMs) have become increasingly useful computational models of human language processing, but it remains unclear whether vision-language learning makes text representations more human-like during natural reading. Here, we address this question by comparing tightly matched LLM and vision-language model (VLM) pairs under a strictly text-only setting, allowing us to isolate the effect of multimodal training history from online visual input or cross-modal fusion. We evaluate model alignment with a human natural-reading dataset that includes whole-cortex fMRI responses and synchronized eye-tracking saccades. Our findings demonstrate that multimodal pretraining may not confer a uniform, global advantage in human alignment during natural reading, indicating that language-internal representations remain the key factor for modeling human text processing. However, the VLM advantage could emerge more selectively when sentences contain stronger visual semantic content, with converging evidence from both fMRI and eye-movement alignments. Together, our findings provide a controlled in silico framework for testing how visual learning history shapes model-human alignment of language processing, suggesting that multimodal pretraining contributes selectively rather than globally to human-like language representations during natural reading.