(UroToday.com) Mr. Patrick Doyle and colleagues from Vanderbilt University held an excellent presentation on the use of machine learning models in predicting stone recurrence episodes. Although 24-hour (24H) urine studies are currently recommended by AUA Guidelines as an informative measure for the metabolic profile of high-risk stone formers, there exist significant limitations in testing utility, where the value of the test in predicting recurrent stone events is yet to be established. Additionally, further limitations may be present depending upon patient factors and test processing. Provided this discrepancy, Mr. Doyle et al. aimed to evaluate the feasibility of applying machine learning (ML) models to predict symptomatic stone recurrence at 2 and 5 years, as well as assessing which features are utilized by such models.
These models are a form of artificial intelligence to automatic analytical model building, which allows them to uniquely handle non-linear variables. Mr. Doyle and colleagues have previously demonstrated the utilization of ML in the prediction of stone composition and 24H abnormalities. In this study, they retrospectively analyzed a cohort (n=1231) at a single institution, in which patients were manually identified from an index symptomatic episode defined by pain, acute kidney injury, recurrent infections attributed to a kidney stone identified in an outpatient clinic or emergency department, or stones that require surgical intervention. All patients underwent 24H urine testing (Litholink, Chicago, IL). Recurrence was defined by outpatient treatment, surgery, or ER visit for stone evaluation 90 days or more following the index event. Following this, electronic health-record (EHR) data was extracted in addition to the 24H urine data. Three different ML models (least absolute shrinkage and selection operator regression [LASSO], random forest [RF], and gradient boosted decision tree [XGBoost]) were used to predict symptomatic recurrence from EHR features and the 24H urine data. EHR features included demographics, medical history, and medication; while lab-based features incorporated a 24H urine analysis and stone composition. Comparison between models was accomplished using logistic regression. Performance was analyzed using area under the receiver operating curve (AUC-ROC), and predictors for each model were identified with a Shapely Additive Explanation.
Interestingly, Mr. Doyle et al. found 2- and 5- year symptomatic stone recurrence rates of 25% (308) and 31% (381), respectively. The top-preforming LASSO model was the top-preforming model for symptomatic stone recurrence prediction (2-yr AUC: 0.62, 5-yr AUC: 0.63) (Figure 1). In comparison, there was modest performance with LR (0.585, 0.618), RF (0.570, 0.608), and XGBoost (0.580, 0.621). The prioritized features for all models included age, diabetic status, stone composition, and urine pH. Of note, the LASSO model included BMI and history of gout. Altogether, Mr. Doyle shared fascinating demonstrations of ML models and recurrence prediction.
Following a productive conversation between various members in the audience and the moderators, Mr. Doyle concluded his presentation with these lasting thoughts:
- Demonstrate feasibility of models for prediction of symptomatic recurrence
- Features utilized reflect known risk factors for recurrence
- Future work may explore the optimization of models, curating a robust dataset, and evaluating software for automated feature extraction from EHR
Presented by: Patrick Doyle, Vanderbilt University Medical Center
Written by: Mariah Hernandez, Department of Urology, University of California, Irvine, @mariahch00 on Twitter during the 2023 American Urological Association (AUA) Annual Meeting, Chicago, IL, April 27 – May 1, 2023