The findings demonstrate a moderate concordance between the AI model and oncologists (Kappa = 0.43, p < 0.001) It suggests that while ChatGPT 4.0 may complement human expertise, it cannot replace it. Specifically, ChatGPT's recommendations for treatment continuation or cessation were aligned with oncologists' decisions in many cases, but key discrepancies arose in scenarios requiring a more nuanced understanding of patient-specific factors, such as comorbidities and prior treatment responses. One particularly encouraging aspect of the study was the significant time saved in pre-charting and decision-making, emphasizing the efficiency gains that AI can offer to reduce clinician workload. The average charting time for oncologists was 42 minutes compared to just less than a minute when using ChatGPT, highlighting the potential for substantial improvements in workflow efficiency.
However, the discrepancies noted in treatment decisions illustrate a key limitation: the difficulty AI faces in replicating the nuanced clinical judgment of experienced oncologists. These discrepancies were particularly evident in cases involving complex therapeutic decisions, where considerations such as patient tolerance to therapy, side effect profiles, and evolving biomarkers were critical. While ChatGPT shows promise as a support tool for structured and well-documented cases, more development is needed to enhance its ability to integrate unstructured clinical data, interpret patient histories, and consider multifaceted clinical parameters. This limitation highlights the need for AI models to evolve beyond pattern recognition to incorporate deeper clinical reasoning and contextual understanding.
Other studies have continued to evaluate the efficacy of AI tools like ChatGPT in oncology and other medical fields. For instance, a study1 demonstrated that advanced AI models could predict skin cancer outcomes with high accuracy, highlighting the importance of integrating multimodal data inputs, such as genomic sequencing, imaging, and electronic health records.1 This finding reinforces the importance of integrating diverse types of patient data to enhance the accuracy and applicability of AI in clinical decision-making. Other research has explored the use of AI in generating personalized treatment recommendations in complex cancer cases and found that AI could provide valuable insights, particularly when used in conjunction with a multidisciplinary team of oncologists.2 These studies collectively highlight the evolving role of AI in augmenting the capacity of healthcare professionals to make informed, patient-specific decisions.
Recent literature has also emphasized the importance of explainable AI (XAI) in improving clinician trust and the adoption of AI tools. For instance, a recent study discussed the role of XAI in making AI systems more transparent and interpretable, which is crucial for their adoption in clinical settings.3 The ability of AI models to provide interpretable insights is crucial, particularly in fields like oncology where treatment decisions are complex and high-stakes. Incorporating XAI methodologies could help bridge the gap between AI-generated recommendations and clinician acceptance, facilitating a more synergistic human-AI collaboration. The need for transparency in AI decision-making processes is especially relevant in telemedicine, where patients may be relying on remote consultations without the opportunity for immediate, in-person clarification.
Future research should explore the broader generalizability of AI models across different cancer types and patient demographics, particularly focusing on diverse populations that may present distinct biological and socio-economic challenges. Additionally, the integration of real-time learning capabilities, such as reinforcement learning from real-world clinical data, could enhance the adaptability and reliability of AI decision-making. Studies should also investigate the incorporation of multimodal data inputs, including imaging, genomic data, and patient-reported outcomes, to improve the depth of AI-assisted decision-making. Moreover, focusing on human-AI collaboration frameworks where AI acts as an assistant rather than a standalone decision-maker can help mitigate the risks associated with AI errors in clinical practice.
Ultimately, our study illustrates the transformative potential of AI while highlighting that its best application currently lies in collaboration with, rather than a substitution for, human medical professionals. Future studies should aim to establish robust frameworks for AI integration that prioritize patient safety, clinical efficacy, and the enhancement of healthcare delivery. In addition, ongoing advancements in explainable AI and multimodal data integration will likely play a critical role in shaping the future landscape of AI-assisted telemedicine. With continued research and development, AI has the potential to become an indispensable component of personalized, efficient, and accessible healthcare.
Written by: Emre Dal, MD, Huntsman Cancer Institute, University of Utah, Salt Lake City, UT
References:
- Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017 Feb 2;542(7639):115-118. doi: 10.1038/nature21056. Epub 2017 Jan 25. Erratum in: Nature. 2017 Jun 28;546(7660):686. doi: 10.1038/nature22985. PMID: 28117445; PMCID: PMC8382232.
- Krittanawong C, Zhang H, Wang Z, Aydar M, Kitai T. Artificial Intelligence in Precision Cardiovascular Medicine. J Am Coll Cardiol. 2017 May 30;69(21):2657-2664. doi: 10.1016/j.jacc.2017.03.571. PMID: 28545640.
- Tonekaboni, S., Joshi, S., McCradden, M. D., & Goldenberg, A. (2019). What clinicians want: Contextualizing explainable artificial intelligence for clinical end use. The Lancet Digital Health, 1(3), e157-e158.