AUTHOR=Pamova Anastasia Petrovna , Vasilev Yuriy Aleksandrovich , Bobrovskaya Tatyana Mikhaylovna , Vladzimirskyy Anton Vyacheslavovich , Omelyanskaya Olga Vasilyevna , Arzamasov Kirill Mikhailovich 

TITLE=Implementation of a net benefit parameter in ROC curve decision thresholds for AI-powered mammography screening

JOURNAL=Frontiers in Big Data

VOLUME=Volume 8 - 2025

YEAR=2025

URL=https://www.frontiersin.org/journals/big-data/articles/10.3389/fdata.2025.1690955

DOI=10.3389/fdata.2025.1690955

ISSN=2624-909X

ABSTRACT=BackgroundThe rapid integration of artificial intelligence (AI) into mammography necessitates robust quality control methods. The lack of standardized methods for establishing decision thresholds on the Receiver Operating Characteristic (ROC) curves makes it challenging to judge the AI performance. This study aims to develop a method for determining the decision threshold for AI in screening mammography to ensure the widest possible population of women with a breast pathology is diagnosed.MethodsThree AI models were retrospectively evaluated using a dataset of digital mammograms. The dataset consisted of screening mammography examinations obtained from 663,606 patients over the age of 40. Our method estimates the decision threshold using a novel approach to net benefit (NB) analysis. Our approach to setting the cutoff threshold was compared with the threshold determined by Youden's index using McNemar's test.ResultsReplacing the Youden index with our method across three AI models, resulted in a threefold reduction in false-positive rates, twofold reduction in false-negative rates, and twofold increase in true-positive rates. Thus, the sensitivity at the cutoff threshold determined by NB increased to 99% (maximum) compared to the sensitivity determined by Youden's index threshold (72% maximum). Correspondingly, the specificity when using our method decreased to 48% (minimum), compared to 75% (minimum) with the Youden's index method.ConclusionsWe propose using AI as the initial reader together with our novel method for determining the decision threshold in screening with double reading. This approach enhances the AI sensitivity and improves timely breast cancer diagnosis.