Back to 2025 Abstracts
POSITIVE NET BENEFIT OF MACHINE LEARNING: PREDICTING POSTOPERATIVE PANCREATIC FISTULA AFTER PANCREATICODUODENECTOMY
Rachel L. Wolansky
*1, Tyler Zander
1, Joseph Sujka
1, Richard A. Jacobson
1, Timothy M. Nywening
1, Eric M. Toloza
2, Paul Kuo
1,3, Melissa A. Kendall
11Surgery, USF Health, Tampa, FL; 2Moffitt Cancer Center, Tampa, FL; 3C W Bill Young Department of Veterans Affairs Medical Center, Bay Pines, FL
Postoperative Pancreatic Fistula (POPF) is a devastating complication following pancreas surgery. Studies have compared the performance of machine learning (ML) models to predict POPF. However, the use of decision curve analysis (DCA) to assess superiority and clinical utility of ML models, expressed as net benefit, has not been studied. We aim to develop clinically interpretable and applicable ML models to predict POPF after pancreaticoduodenectomy (PDD) for pancreatic malignancy using DCA.
The National Surgical Quality Improvement Program Pancreatectomy database (2017-2022) was queried for adults with pancreatic malignancy who underwent elective PDD performed on the day of admission. Patients with >5% missing information were excluded. Univariable analysis compared patients who developed clinically significant POPF (+POPF) to those who did not (-POPF). AutoML, XGBoost, and LightGBM models were developed using a 60/20/20 train/validation/test split to predict POPF and SHapley Additive exPlanations (SHAP) were used for interpretation of feature importance of predictors. DCA was used to evaluate the clinical net benefit of the models.
8,478 patients were included: 920 (10.9%) +POPF and 7,558 (89.1%) -POPF. +POPF patients were more likely to be male than female (57.3 vs 42.7%, p < 0.01). T1 disease was more common in the +POPF group (22.6 vs 20.8%, p < 0.01). +POPF patients were less likely to have received neoadjuvant chemotherapy (35.5 vs 50.9%, p < 0.01) or radiation (7.6 vs 15.5%, p < 0.01), or have a biliary stent (57.8 vs 64.6%, p <0.01). An AutoML gradient boosting machine was top performing. Lift curve analysis demonstrated that our model identifies +POPF 3.6 times more effectively than random guess (AUC 0.74, Brier Score 0.08, F1 maximizing score 0.35). The top 5 influential factors for +POPF predictions in the models were soft gland texture, lack of diabetes, greater weight, small duct size, and lack of neoadjuvant therapy. DCA revealed positive net benefit for all models in a similar threshold range for treating patients predicted as +POPF compared to the "treat all" vs "treat none" strategies (Figure 1).
Gland texture, diabetes, weight, duct size, and neoadjuvant therapy are influential in +POPF predictions following PDD in patients with pancreatic malignancy. DCA reveals clinical utility of ML models expressed as positive net benefit to predict +POPF and implement treatment. In addition to standard performance metrics, DCA should be applied to ML models to highlight their utility as an adjunct to clinical decision-making.
Back to 2025 Abstracts
