Renal cell carcinoma (RCC) is a common cancer that varies in clinical behavior. Clear cell RCC (ccRCC) is the most common RCC subtype, with both aggressive and indolent manifestations. Indolent ccRCC is often low-grade without necrosis and can be monitored without treatment. Aggressive ccRCC is often high-grade and can cause metastasis and death if not promptly detected and treated. While most RCCs are detected on computed tomography (CT) scans, aggressiveness classification is based on pathology images acquired from invasive biopsy or surgery.
CT imaging-based aggressiveness classification would be an important clinical advance, as it would facilitate non-invasive risk stratification and treatment planning. Here, we present a novel machine learning method, Correlated Feature Aggregation By Region (CorrFABR), for CT-based aggressiveness classification of ccRCC.
CorrFABR is a multimodal fusion algorithm that learns from radiology and pathology images, and clinical variables in a clinically-relevant manner. CorrFABR leverages registration-independent radiology (CT) and pathology image correlations using features from vision transformer-based foundation models to facilitate aggressiveness assessment on CT images. CorrFABR consists of three main steps: (a) Feature aggregation where region-level features are extracted from radiology and pathology images at widely varying image resolutions, (b) Fusion where radiology features correlated with pathology features (pathology-informed CT biomarkers) are learned, and (c) Classification where the learned pathology-informed CT biomarkers, together with clinical variables of tumor diameter, gender, and age, are used to distinguish aggressive from indolent ccRCC using multi-layer perceptron-based classifiers. Pathology images are only required in the first two steps of CorrFABR, and are not required in the prediction module. Therefore, CorrFABR integrates information from CT images, pathology images, and clinical variables during training, but for inference, it relies solely on CT images and clinical variables, ensuring its clinical applicability. CorrFABR was trained with heterogenous, publicly-available data from 298 ccRCC tumors (136 indolent tumors, 162 aggressive tumors) in a five-fold cross-validation setup and evaluated on an independent test set of 74 tumors with a balanced distribution of aggressive and indolent tumors. Ablation studies were performed to test the utility of each component of CorrFABR.
CorrFABR outperformed the other classification methods, achieving an ROC-AUC (area under the curve) of 0.855 ± 0.0005 (95% confidence interval: 0.775, 0.947), F1-score of 0.793 ± 0.029, sensitivity of 0.741 ± 0.058, and specificity of 0.876 ± 0.032 in classifying ccRCC as aggressive or indolent subtypes. It was found that pathology-informed CT biomarkers learned through registration-independent correlation learning improves classification performance over using CT features alone, irrespective of the kind of features or the classification model used. Tumor diameter, gender, and age provide complementary clinical information, and integrating pathology-informed CT biomarkers with these clinical variables further improves performance.
CorrFABR provides a novel method for CT-based aggressiveness classification of ccRCC by enabling the identification of pathology-informed CT biomarkers, and integrating them with clinical variables. CorrFABR enables learning of these pathology-informed CT biomarkers through a novel registration-independent correlation learning module that considers unaligned radiology and pathology images at widely varying image resolutions.
Medical physics. 2024 Oct 24 [Epub ahead of print]
Indrani Bhattacharya, Karin Stacke, Emily Chan, Jeong Hoon Lee, Justin R Tse, Tie Liang, James D Brooks, Geoffrey A Sonn, Mirabela Rusu
Department of Radiology, Stanford University, Stanford, California, USA., Sectra, Linköping, Sweden., Department of Pathology, Stanford University, Stanford, California, USA., Department of Urology, Stanford University, Stanford, California, USA.