Determining the optimal sequence of therapies for patients with prostate cancer (PC) is challenging. Artificial intelligence (AI)-based tools that can analyze the complex medical literature have the potential to assist clinicians at a time when the literature is expanding quickly. This study describes INSIDE PC, an AI-based platform designed to help clinicians query the literature on therapeutic sequencing in PC (Figure 1). Importantly, this study also develops new practices for evaluating outputs of AI-driven support platforms.
Click to Enlarge
Figure 1. Visual representation of the INSIDE PC methodology (top) and INSIDE platform (top; accessible at https://inside.dimensions.ai/ ). Data are current as of February 2023. On the top, the figure demonstrates the layers of search inquiry needed to focus in on literature related to therapeutic sequence and how PubMedBERT was used for the semantic analysis. On the bottom is an image of the INSIDE PC dashboard including its search and filtering functionality, tables of treatment sequences and their corresponding publications. The Sankey diagram on the far right can be viewed as part of the dashboard. It provides a visual interpretation of the first treatment the user would like to search for followed by a subsequent treatment.
Three blinded PC experts compared the relevance of publications retrieved by INSIDE PC and PubMed for three test questions related to therapy sequencing in metastatic castration-resistant PC (mCRPC).
- For the first question, which focused on a novel hormonal therapy (NHT; also called androgen receptor pathway inhibitor or ARPI) followed by another NHT (the example being enzalutamide followed by abiraterone), INSIDE-PC outperformed PubMed over the top 5, 10, and 20 publications (Figure 2).
- For the second question, which focused on an NHT followed by a poly-ADP ribose polymerase inhibitor (PARPi), INSIDE PC and PubMed performed similarly for the top 5, 10, and 20 publications.
- For the third question, which focused on an NHT or PARPi followed by 177Lu-PSMA-617, INSIDE PC outperformed PubMed for the top 5, 10, and 20 publications, although the differences were more modest than for the first question.
Figure 2. Comparison of the search performance of INSIDE PC and PubMed for the first test question (NHT-NHT sequencing; enzalutamide followed by abiraterone). Publications were scored for relevance using a 3-point Likert scale (0, not relevant; 1, somewhat relevant; 2, very relevant). Performance was summarized by computing the normalized discounted cumulative gain (nDCG) scores. nDCG scores sum relevance scores after weighting (discounting) them by a function of their ranking in the search, so that the relevance score of documents lower in the search results were counted less than the score of documents ranked higher.
This study applied INSIDE PC to develop standards for evaluating the performance of AI-based literature extraction tools. In this initial evaluation, INSIDE PC performed competitively with PubMed suggesting it can assist clinicians in determining optimal therapeutic sequencing for patients with PC.
Written by: Arnulf Stenzl,1 Andrew J. Armstrong,2 Andrea Sboner,3 Jenny Ghith,4 Lucile Serfass,5 Christopher S. Bland,4 Bob J. A. Schijvenaars,6 Cora N. Sternberg7
- Department of Urology, University Hospital Tübingen, Tübingen, Germany.
- Department of Medicine, Duke Cancer Institute Center for Prostate and Urologic Cancer, Duke University, Durham, NC, USA.
- Department of Pathology and Laboratory Medicine, Englander Institute for Precision Medicine, Meyer Cancer Center, Weill Cornell Medicine, New York, NY, USA.
- Pfizer Inc, Collegeville, PA, USA.
- Pfizer Oncology, Paris, France.
- Digital Science, London, UK.
- Department of Medicine, Englander Institute for Precision Medicine, Meyer Cancer Center, Weill Cornell Medicine, New York, NY, USA.