According to the Academic Radiology journal, a Natural Language Processing (NLP) system aided to recognize lumbar spine findings, offering considerable gains in model sensitivity. Researchers from many states asked to assess an NLP system developed with open-source tools for recognition of lumbar spine imaging findings related to low back pain on magnetic resonance and x-ray radiology reports from four health systems.
The researchers utilized a limited data set tested from lumbar spine imaging reports of a prospectively accumulated group of adults. A total of 871 reports were randomly chosen from 178,333 available reports, in which 413 were x-rays and 458 were MR reports. Four spine analysts, using standardized criteria, defined the presence of 26 finding, where 71 reports defined by all four analysts and 800 were each annotated by two experts. The researchers measured inter-rater concord and finding prevalence from defined data. The defined data was randomly split into development, with 80 percent and 20 percent testing sets. The researchers built an NLP system from both rule-based and machine-learned models. The system was verified using precision metrics like sensitivity, specificity, and area under the receiver operating characteristic curve.
In the testing sample, rule-based and machine-learned anticipations both had comparable average specificity, 0.97 and 0.95, apiece. The machine-learned approach had a higher average sensitivity, with 0.94, compared to 0.83 for rules-based, and a higher overall AUC, with 0.98, compared to 0.90 for rules-based. The researchers concluded that their Natural Language Processing system acted well in recognizing the 26 lumbar spine findings, as benchmarked by reference-standard annotation by medical experts. Machine-learned models offered substantial gains in model sensitivity with slight loss of specificity, and overall higher AUC.