Evaluating Mild Cognitive Impairment using large language models: A comparative analysis

This study evaluates whether fine-tuned large language models (LLMs) can support preliminary screening of Mild Cognitive Impairment (MCI) from transcribed Chinese speech. Using the Chinese subset of the TAUKADIAL/TalkBank connected-speech dataset, we compared Llama3, Gemma, Qwen, and Qwen1.5 models under supervised fine-tuning, LoRA, QLoRA, different quantization settings, automatic speech recognition systems, and prompt styles. The experiments were evaluated with subject-level five-fold cross-validation and reported with mean and standard deviation for accuracy, F1-score, precision, and recall. Among the tested configurations, Llama3-8B with 4-bit QLoRA achieved the strongest performance, with accuracy and F1-score of approximately 0.86. To place these results against transparent alternatives, we added the official TAUKADIAL feature-based benchmarks, which use acoustic eGeMAPS features and interpretable linguistic measures such as token count, type count, type–token ratio, lexical density, and verb and pronoun ratios. Because the official benchmark uses a different multilingual train/test protocol, it is treated as an external reference rather than a head-to-head ranking. The findings suggest that LLM-based transcript analysis may be useful as a low-cost prescreening aid, but the small, language-specific dataset and lack of matched external validation require cautious interpretation.