BanglaDiaHallu: Hallucinations of large language models and the effect of guideline-grounded retrieval-augmented generation in Bangla diabetes question answering

Large language models (LLMs) are increasingly consulted for health advice, but they can produce fluent statements that are false or unsupported, known as hallucinations, and this risk is poorly characterised in low-resource languages such as Bangla. We measured hallucinations of locally deployable open-weight LLMs in Bangla diabetes question answering and tested whether retrieval-augmented generation (RAG) grounded in Bangladeshs National Guideline on Diabetes Mellitus reduces them. Forty patient questions spanning six clinical sub-domains were each answered twice, under zero-shot prompting, by Qwen3.5-9B, TigerLLM-9B, Qwen3.6-35B-A3B and Gemma-4-26B-A4B, and by BanglaDiaRAG, which adds hybrid dense (BGE-M3) and sparse (BM25) retrieval from a physician-reviewed Bangla guideline knowledge base to Qwen3.5-9B. Two physicians rated the 400 de-identified responses by consensus for medical accuracy, completeness, safety, readability and reliability on a 0-1 scale, and responses with accuracy below 0.70 were classified as hallucinated. Hallucination rates were 2.5% for Gemma-4-26B-A4B, 3.8% for TigerLLM-9B, 22.5% for Qwen3.6-35B-A3B and 77.5% for Qwen3.5-9B. Guideline grounding lowered the rate of Qwen3.5-9B to 11.3% (difference -66.3 percentage points; 95% CI -76.3 to -55.0) and raised every criterion by 0.24 to 0.38 points (Holm-adjusted p < 0.001). BanglaDiaRAG outscored Qwen3.6-35B-A3B on all five criteria and matched or exceeded TigerLLM-9B, but it remained below Gemma-4-26B-A4B in safety, readability and reliability, and its residual errors were specific, confident and potentially harmful. Guideline grounding can turn a weak general-purpose model into a much safer Bangla diabetes assistant, but the choice of base model mattered as much as retrieval and no configuration was free of hallucination, so clinical oversight remains essential before patient-facing use.