Analysis of article screening and data extraction performance by an AI systematic literature review platform - Cassell, Kelsie; Ologunowa, Abiodun; Rastegar-Mojarad, Majid; Chun, Bianca; Huang, Yi-Ling; Wang, Dong; Cossrow, Nicole (Frontiers in Artificial Intelligence, 2025)
BackgroundSystematic literature reviews (SLRs) are critical to health research and decision-making but are often time- and labor-intensive. Artificial intelligence (AI) tools like large language models (LLMs) provide a promising way to automate these processes.MethodsWe conducted a systematic literature review on the cost-effectiveness of adult pneumococcal vaccination and prospectively assessed the performance of our AI-assisted review platform, Intelligent Systematic Literature Review (ISLaR) 2.0, compared to expert researchers.ResultsISLaR demonstrated high accuracy (0.87 full-text screening; 0.86 data extraction), precision (0.88; 0.86), and sensitivity (0.91; 0.98) in article screening and data extraction tasks, but lower specificity (0.79; 0.42), especially when extracting data from tables. The platform reduced abstract and full-text screening time by over 90% compared to human reviewers.ConclusionThe platform has strong potential to reduce reviewer workload but requires further development.