Bob Schijvenaars: My name is Bob Schijvenaars, and I'm one of the authors of the article we are discussing: Artificial Intelligence to Support Informed Decision-making for Improved Literature Analysis in Oncology, that has been published in the Journal of European Urology Focus. Our study introduces the Inside Prostate Cancer Initiative, which is designed to help clinicians who are increasingly faced with information overload. We specifically focused on addressing whether an AI based tool inside prostate cancer could help clinicians query the literature on therapy sequencing. Our study is relevant because it contributes to the development and standardization of AI systems, as well as timely, because it addresses therapeutic sequencing, a topic that requires deep understanding of complex information by subject matter experts.

Typical search engines struggle with the different ways in which therapeutic sequencing is described in scientific publications. Our hypothesis is that using AI to detect mentions of sequences in scientific texts will provide more relevant results than a standard search engine, which focuses on the presence of individual terms rather than semantics. To test this hypothesis, we built a classifier that aims to distinguish sequencing mentions from other co-mentions. For instance, when comparing efficacy of two mono treatments. This classifier was developed by customizing a large language model. Detected sequences can then subsequently be used to rank and aggregate publications for relevance.

In order to test the hypothesis, we chose, as use cases, three groups of treatment sequences and constructed for each of these, a list of publications ranked according to the frequency of relevance sequence mentions detected. This is a very simple ranking method compared to the typical engine ranking methods. We compared our rank list with the results of a PubMed search for these sequences and had a panel of domain experts judge the relevance of each paper in both result lists.

The graphs showed a performance using nDCG, a standard metric for expressing relevance in a ranked search result list. The approach outperformed PubMed in certain areas, particularly for queries on novel hormonal therapies followed by NHT, and for NHT or PARPI, followed by lutetium. The two provided more relevant results than PubMed in these cases, demonstrating its effectiveness in literature analysis. This shows that INSIDE PC is competitive in searching for sequencing literature, supporting clinicians in therapeutic decision-making for prostate cancer. It demonstrates the added value of AI assisted semantic analysis, and highlights the need for structured evaluation and the potential of AI in medical literature analysis.

To demonstrate the capabilities, we developed a publicly accessible dashboard with some features demonstrating that added value. For instance, the sankey diagram visual gives you an overview of the number of publications mentioning a certain treatment sequence. You can filter the mono treatments and you can filter on the Z setting mentioned in the publication. I'm now going to focus on MCRPC, for instance. To focus on a particular sequence. You can pick the first, second, or both mono treatments. Additional filters allow for categories that approximate level of evidence. At the bottom, you see the list of publications ranked by the number of sequences detected in the text.

So concluding, the Insight Initiative proves that systems designed for a semantic Q & A analysis are effective and are a valuable addition to standard search engines, especially where the language used to express certain concepts, like therapeutic sequencing, has so much variation. Thank you for listening to this explanation, and I hope you found it useful.