Comparison of the Accuracy of NLP Tools and Manual Investigation for Identification, Extraction, and Classification of Biomarkers in Colorectal Cancer

Background and Aim: The rapid expansion of biomedical literature has made the manual identification and extraction of biomarker information increasingly time-consuming. Natural language processing (NLP) tools may facilitate large-scale literature screening and support the generation of research hypotheses. However, their ability to broadly identify diverse biomarker types without predefined target terms remains insufficiently characterized. Materials and Methods: Fifty original articles on colorectal cancer biomarkers published between 2015 and 2025 were manually reviewed, and reported biomarkers were extracted to establish a reference dataset. The same articles were processed using two biomedical NLP models: d4data/biomedical_ner_all and en_ner_bionlp3cg_md from the SciSpaCy framework. Biomarkers extracted by each model were compared with the manually curated reference set. The proportion of manually identified biomarkers recovered by each model was calculated, and extracted entities not present in the reference set were also assessed. Statistical analyses were performed using an independent-samples t-test.