(UroToday.com) The 2024 Society of Urologic Oncology (SUO) annual meeting held in Dallas, between December 3 and December 6, 2024, was host to the Prostate Cancer Session I. Dr. Okyaz Eminaga discussed risk stratification using AI-Driven Digital Biomarkers.
Dr. Eminaga began his presentation by emphasizing that prostate cancer presents a broad spectrum of risk profiles due to its diverse biological behaviors, varying stages at diagnosis, metastatic patterns, and tumor burden. This complexity necessitates a risk-adjusted approach to decision-making in clinical practice.
Unfortunately, while risk calculators provide numerical outputs, they don't make decisions for us, as much as we might wish they could. While continuous values can be accurate, they often lack the decisiveness needed in clinical decision-making. To overcome this, we use thresholding as a tool to abstract information. By setting mental thresholds, we simplify complex data into actionable insights that help guide the decision-making process.
Dr. Eminaga posed a critical question: how do we define actionable thresholds for AI-driven risk stratification in decision-making? This remains a major challenge, as AI tools offer sophisticated data analysis but don't inherently define the exact cutoff points at which clinicians should take action. These thresholds must be carefully determined through clinical judgment, statistical analysis, and real-world data, ensuring that they align with outcomes and treatment goals. AI can support this process, but ultimately, it’s up to clinicians to interpret these thresholds and apply them in practice. Identifying the right actionable thresholds will be a key step in fully integrating AI into personalized treatment strategies.
Dr. Eminaga introduced an AI-driven risk stratification system developed through a large collaborative effort across several hospitals in Canada and the US. The system utilized multiple cohorts, including the CPDCBN cohort, PROCURE cohort, and the PLCO trial cohort. One cohort served as the training set for the algorithm, while another cohort was used for in-training validation.T he development cohort included 600 RP cases from two institutions.1
Dr. Eminaga emphasized that for AI-driven risk stratification, it is crucial to have clear boundaries and a well-calibrated prediction model. This was the first step in their large multinational, multi-institutional collaboration. Additionally, the validation of the risk stratification concept on independent cohorts is vital. Without proper calibration and validation across different patient populations, an AI-driven risk stratification system may yield significantly different results.
He noted that the current risk stratification system uses five categories: very low, low, intermediate, high, and very high risk groups. However, AI proposed in this validation set only four risk scores through an unsupervised decision tree after examining transition patterns at the microscopic level. The reason for this discrepancy remains to be fully understood. For this model, multiple validations were conducted, with a correlation between the grade severity and the risk scores. These correlations were color-coded, as shown at the top of the figure.
Dr. Eminaga briefly discussed that Al-Driven four risk groups appeared to be prognostic to cancer-specific survival on the PLCO cohort
Furthermore, Al-driven risk stratification was correlated with the development of castration-resistant prostate cancer in BCR (39.3%)/ in the highest risk group as shown in the graphic below.
Dr. Eminaga highlighted a common issue in the literature: the overuse of the concordance index (c-index) as a metric for model evaluation. While the c-index measures a model's ability to distinguish between two subjects in terms of risk, it does not address several critical aspects of clinical decision-making, particularly in survival models. Specifically, it falls short in answering questions about the validity of the model, the relevance of its findings, the strength of its predictions, and its ability to accurately group risks and estimate survival outcomes. Therefore, Dr. Eminaga suggested that other metrics should be explored to evaluate models more comprehensively, as the c-index alone does not provide a full picture of a model's clinical utility.
Dr. Eminaga emphasized the importance of specific metrics in evaluating the prognostic value and clinical relevance of predictive models. These metrics include:
- Goodness-of-Fit – This metric assesses how well a model's predicted outcomes align with the actual outcomes. It helps to determine how accurately the model reflects the observed data.
- Net Benefit Measures – This evaluates the model's overall benefit by comparing the potential benefits of making a prediction (true positives) with the potential harms (false positives), often factoring in the cost of misclassification.
- Decision Curve Analyses – This method analyzes the clinical utility of a model by determining whether the decision to act based on the model's predictions results in a positive net outcome across different thresholds.
Lastly, Dr. Eminaga highlighted that the goodness-of-fit for the novel AI-derived risk group classification is comparable to that of the full model when predicting cancer-specific survival. This suggests that the AI model can perform at a similar level of accuracy in identifying survival outcomes, making it a promising tool for clinical decision-making.
Dr. Eminaga concluded his presentation with a call to action for collaboration and innovation, encouraging a collective disruption of current treatment and diagnostic paradigms in order to achieve breakthroughs beyond what we currently imagine
Presented by: Okyaz Eminaga, MD, PhD, Urologist, Data/AI Scientist, and Physician at Stanford University School of Medicine, California, USA.
Written by: Julian Chavarriaga, MD – Urologic Oncologist at Cancer Treatment and Research Center (CTIC) Luis Carlos Sarmiento Angulo Foundation via Society of Urologic Oncology (SUO) Fellow at The University of Toronto. @chavarriagaj on Twitter during the 2024 Society of Urologic Oncology (SUO) annual meeting held in Dallas, between the 3rd and 6th of December, 2024.
References:- Eminaga, O., Saad, F., Tian, Z. et al. Artificial intelligence unravels interpretable malignancy grades of prostate cancer on histology images. npj Imaging 2, 6 (2024). https://doi.org/10.1038/s44303-023-00005-z