(UroToday.com) The 39th World Congress of Endourology and Uro-Technology included a session regarding new technology and infection related to clinical stones, featuring work from Dr. Shaan Setia and colleagues presenting results of their study assessing the efficacy of computer vision techniques to automatically segment kidney stones during ureteroscopy. During stone removal procedures, it is critical that the endoscope field of vision remains clear to properly visualize stones. During fragmentation using laser lithotripsy, retropulsion, debris, and blood can impact image quality and make tracking stones difficult. Therefore, this study sought to address this issue by utilizing computer vision techniques to segment stones in real-time and improve tracking ability as a result.
Using digital ureteroscopes (Karl Storz Flex XC), twenty separate videos of ureteroscopy were collected. Frames from each video were extracted and manually annotated by researchers to identify stones in each frame. Three established image segmentation computer vision models, which included U-Net, U-Net++, and DenseNet, were trained using the annotated frames. The data was split up for different purposes: 80% was used to train the models to differentiate stone from other material, 10% was used for validation, and 10% was used for actual testing. Outcomes for the study included Dice similarity coefficient (DSC), accuracy (per pixel), and area under the receiver operating characteristic curve (ROC-AUC).
Figure 1 shows a summary of the statistical outcomes gathered for each model. The non-parenthetical values represent the average scores from all frames in the test set. The reported value in parentheses denotes the maximum value recorded from each baseline’s validation during training. Emphasized in bold are the highest performances when comparing the models. U-Net++, as a modeling system, achieved the highest scores in each metric on the test set. U-Net had the same test Dice score as U-Net++ but lower scores in all other metrics. DenseNet had lower performance for all metrics. Only the ROC curves from test set performances are included.
Figure 2 depicts an example of automated segmentation and the comparisons between measurements. As shown in side-by-side comparison, the manual annotation and U-Net++ predicted segmentation are very similar visually. The heat map may provide another angle from which to approach tracking and complements other visualization methods. When put to the test in the operating room, the models were capable of intaking real-time video at 30 fps and annotating live, which provides hope for augmenting this technology into standard practice.
In concluding the presentation, Dr. Setia commented on the promise of these findings. If these models can segment and track stones during stone laser treatment accurately with consistency, this suggests the feasibility of computer stone tracking during ureteroscopy and the improvement of patient outcomes through utilizing this technology.
In opening the floor for discussion, one member of the audience inquired if it was possible to determine the smallest size fragment that was detectable. Dr. Setia proceeded to state that while the models were capable of tracking stones smaller than the fiber themselves, the exact smallest size of those fragments is unknown, which provides a goal for future study to be able to track stones and measure size in real time. As of now, their best estimation is that fragments are less than 250 microns, as that was the size of the laser fiber used.
Another question was raised about the ability to differentiate plaques against stone. Dr. Setia conceded that this was one of the limits of this study, as the model is proficient at distinguishing stone from non-stone material but is currently incapable of identifying the nature of that non-stone material.
A final question was asked about the types of stones and their effect on the accuracy of measurement. According to Dr. Setia, the stones during ureteroscopy varied widely in stone type and it is difficult to make statements on whether the computer models performed better or worse based on stone composition. Anecdotally, however, the performance was mostly similar, although further study is warranted to further elucidate the utility of this technology for more applications.
Presented by: Shaan Setia, MD – Vanderbilt University Medical Center
Written by: Kelvin Vo, BS, Assistant Research Specialist, Department of Urology, University of California Irvine, @kelvinvouci on Twitter during the 39th World Congress of Endo urology and Uro-Technology (WCET), Oct 1 - 4, 2022, San Diego, California.
References:
- S Setia, Z Stoebner, I Oguz, N Kavoussi. Automated Segmentation of Kidney Stones During Ureteroscopy Using Computer Vision Techniques [abstract]. In: 39th World Congress of Endourology and Technology, October 1-4, 2022, San Diego, CA