(UroToday.com) The 2024 SUO annual meeting included a kidney cancer session, featuring a presentation by Dr. Nicholas Kavoussi discussing artificial intelligence in RCC diagnosis and treatment. Dr. Kavoussi started his presentation by emphasizing that artificial intelligence is about prediction. These models are data driven, have flexibility for model fitting, and have low interpretability, whereas conventional statistics are model driven, have explicit features, and rely on inference. There are several different types of artificial intelligence models:
The advantage of artificial intelligence models is that we are able to assess complex, high-dimensional data. The hope for artificial intelligence is that it can make us better and smarter, and that it can stop us from getting things wrong. Due to the vast swaths of data that we generate from clinical care, there are lots of applications and projects of how we will see these in our clinical practice:
The clinical need for artificial intelligence and RCC diagnosis is to improve treatment selection and risk stratification. Additionally, predicting treatment response is important, including whether increased detection of early stage RCC improves cancer specific survival, accurate prediction of pathology, and accurate staging. Operationally, there is data input followed by a training set sample and finally an output:
In terms of diagnosis, this currently relies on human characterization of a renal mass, using quantitative and qualitative characteristics to assess size, anatomic complexity, contrast enhancement, and invasion:
However, Dr. Kavoussi notes that this can be quantified by machine learning, allowing percentages of grade, pathology, and stage to be computed. Machine characterization of a renal mass includes categorization and labeling of visual data (pixels/voxels), which is then associated with an outcome:
For example, the group from Cleveland Clinic showed they could create a model that could automatically perform nephrometry scoring and predict pathologic outcomes similar to experts.1 They further demonstrated that their model could potentially augment renal scoring by using continuous components extracted from the model’s analysis over ordinal components, suggesting that there may be novel ways to evaluate imaging using machine learning based tools, thus extracting more clinical information:
This type of scoring and feature extraction is the basis for data input for all sorts of imaging classification tasks:
Over the last decade, there have been an increasing number of studies on applying artificial intelligence to renal mass characterization:
Summarizing these studies, Dr. Kavoussi highlighted the following performance characteristics (c-statistics):
- Pathologic subtype: 0.75-0.96
- Staging: 0.80-0.90
- Grade: 0.60-0.94
- Prediction of recurrence: 0.84-0.92
With regards to assessing deep learning and the ability to distinguish benign from malignant renal lesions based on routine MRI, Dr. Xi and colleagues assessed 1,162 MRIs (655 malignant, 507 benign) from 5 institutions.2 Compared with 4 experts, the deep learning model had higher test accuracy (0.70 versus 0.60, p = 0.053), sensitivity (0.92 versus 0.80, p = 0.017), and specificity (0.41 versus 0.35, p = 0.450). Moreover, compared with the radiomics model, the deep learning model had higher test accuracy (0.70 versus 0.62, p = 0.081), sensitivity (0.92 versus 0.79, p = 0.012), and specificity (0.41 versus 0.39, p = 0.770):
Dr. Kavoussi highlighted that there are few studies assessing treatment. Preliminary studies suggest that there may be some use in understanding and analyzing the steps of a case automatically and the kinematic chain, but these are small studies and early in their development. With regards to response to treatment, Dr. Margue et al.3 developed a model for individual postoperative disease-free survival prediction using machine learning on real-world prospective data. Using the French UroCCR database, the predictive performance of the machine learning model was evaluated on the test dataset and compared with the usual risk scores. There were 3,372 patients included, with a median follow-up of 30 months. The best results in predicting disease free survival were achieved using Cox proportional hazard models that included 24 variables, resulting in an AUC of 0.81 (CI 95% 0.77-0.85). Of note, the ML model surpassed the predictive performance of the most commonly used risk scores while handling incomplete data in predictors.
Dr. Kavoussi’s group has been looking at integrating clinical information intraoperatively and adding deformation modeling to better track the anatomy and outcomes during partial nephrectomy, highlighted as follows4:
However, Dr. Kavoussi notes there are several current limitations:
- Black box nature
- Lack of data standardization
- Lack of validation
Importantly, Dr. Kavoussi emphasized that we should be using artificial intelligence reporting guidelines, including the STREAM-URO Framework,5 which is designed to promote quality, enhance reproducibility, comparability, and interpretability, as well as improve engagement and literacy. There are 26 included items in this framework, including model rationale, code sharing, and comparison to a reference standard. Finally, the FDA has published a Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning-Based Software as a Medical Device statement, highlighting the following points:
- Clear expectations on quality systems: high-quality data with meaningful features
- Premarket assessment: bias and fairness
- Reporting algorithm changes: robust training models
- Transparency and real-world performance monitoring: explainability, monitoring, and maintenance
Dr. Kavoussi concluded his presentation by discussing artificial intelligence in RCC diagnosis and treatment with the following take home messages:
- We are early in the evaluation
- We have the feasibility to evaluate things we could not before
- Our responsibility is to evaluate and usher in these tools
Presented by: Nicholas Kavoussi, MD, Vanderbilt University, Nashville, TN
Written by: Zachary Klaassen, MD, MSc – Urologic Oncologist, Associate Professor of Urology, Georgia Cancer Center, Wellstar MCG Health, @zklaassen_md on Twitter during the 2024 Society of Urologic Oncology (SUO) annual meeting held in Dallas, between the 3rd and 6th of December, 2024.
References:
- Heller N, Tejpaul R, Isensee F, et al. Computer-generated R.E.N.A.L. Nephrometry Scores Yield Comparable Predictive Results to Those of Human-Expert Scores in Predicting Oncologic and Perioperative Outcomes. J Urol. 2022 May;207(5):1105-1115.
- Xi IL, Zhao Y, Wang R, et al. Deep Learning to Distinguish Benign from Malignant Renal Lesions Based on Routine MR Imaging. Clin Cancer Res. 2020 Apr 15;26(8):1944-1952.
- Margue G, Ferrer L, Etchepare G, et al. UroPredict: Machine learning model on real-wrld data for prediction of kidney cancer recurrence (UroCCR-120). NPJ Precis Oncol. 2024 Feb 23;8(1):45.
- Cannon PC, Setia SA, Klein-Gardner S, et al. Are 3D Imaging Guidance Systems Ready for Use? A Comparative Analysis of 3D Image Guidance Implementations in Minimally Invasive Partial Nephrectomy. J Endourol. 2024 Apr;38(4):395-407.
- Kwong JCC, McLoughlin LC, Haider M, et al. Standardized reporting of machine learning applications in Urology: The STREAM-URO Framework. Eur Urol Focus. 2021;7:672-682.