Read the Full Video Transcript

Ruchika Talwar: Hi, everyone. Welcome back to UroToday's Health Policy Center of Excellence. As always, my name is Dr. Ruchika Talwar, and I'm from Vanderbilt in Nashville, Tennessee. I'm excited to be joined today by Dr. Nathan and Dr. Smith, who are both from UNC Chapel Hill. And they'll be sharing some really interesting insights they recently published from the CISTO trial. Thank you both for making the time to chat. We really appreciate it.

Angela Smith: Thanks for having us.

Krupa Nathan: Yeah, thank you so much for that wonderful introduction. Like Dr. Talwar mentioned, my name is Krupa Nathan, I am a urology resident at UNC Chapel Hill. Thank you so much for the opportunity to talk about our new paper that we had published in Cancer, titled "The Comparison of Intravesical Therapy and Surgery as Treatment Options for Bladder Cancer or CISTO Study: Lessons learned about management and patient enrollment in a large, pragmatic, patient-centered trial". This paper actually came from Dr. Smith's idea of highlighting the operational tactics of the CISTO study to maximize engagement and combat challenges in study accrual, which is a common barrier to the success of many large clinical trials.

So first, we'll talk about the CISTO study and how what we refer to as our lessons learned paper came about. Then we'll pull three lessons learned that we'd like to highlight in this talk. We'll review some data figures. And last but not least, review some take home messages that we'd like to underscore for this paper.

So the CISTO study stands for the Comparison of Intravesical Therapy and Surgery as Treatment Options for Bladder Cancer, which is a large, PCORI-funded, patient-centered outcome study spanning 38 clinical sites around the country that utilizes surveys from patients and caregivers to investigate various patient-centered outcomes, including urinary function, sexual function, quality of life, and other outcomes of radical cystectomy versus intravesical BCG for the management of non-muscle invasive bladder cancer.

One of the largest barriers to success for multi-institutional clinical trials is patient accrual or enrollment. And the way that our lessons learned paper arose was through wanting to highlight the strategies that we used to maximize accrual through study site engagement. Two of those strategies that we described in our paper were CISTOquestion email, which was a centralized email that allowed research coordinators from various CISTO sites to directly ask the principal investigators questions about patient eligibility and screening at their individual sites.

Another engagement strategy that we had employed was all sites meetings, which were monthly virtual meetings involving research coordinators and CISTO staff from all sites—ergo, all sites meetings—which served primarily to provide study updates, check for understanding for eligibility and screening criteria, and then also facilitate communication and engagement among our sites.

So we talk about seven different lessons in our paper. But for this talk, we'd like to highlight three. The first of them being that the introduction of CISTOquestion email was followed by increased and sustained patient screening and enrollment over time, which we'll discuss in detail in the following slides. Additionally, sites that had used CISTOquestion frequently tended to also have fewer post hoc exclusions, or patients who were retroactively deemed ineligible after having been enrolled, which presumably was from research coordinators being more equipped to screen patients after being in regular communication with the team through CISTOquestion. And our third lesson that we wanted to emphasize was that based on a feedback survey that we had administered to CISTO research coordinators, CISTOquestion was rated as a much more valuable tool than all sites meetings were, and received an overwhelmingly positive response over all sites meetings.

So going back to our first lesson, the introduction of CISTOquestion in April 2021, which was approximately two years after the start of the CISTO study and about one year after the COVID-19 pandemic had hit the United States, we had observed a rise in enrollment that was sustained over quite a long period of time. And then also, the sites that had utilized CISTOquestion email the most also tended to have higher enrollment.

When we looked at the top eight enrolling sites, the top five with the highest enrollments were both, one, enrolled in CISTO before the introduction of CISTOquestion, and two, had sent more than the average of 16 email inquiries per site, indicating that CISTOquestion was critical to those sites’ success in enrolling patients.

And so, given the observations that we had made about the CISTO study strategies for success, we have three take home points. First of them being that accessible communication, or CISTOquestion in this case, was critical in CISTO’s success in achieving accrual goals, especially during the COVID-19 pandemic. However, there is a nuance to that, being that a large part of the success of CISTOquestion email was the responsiveness to email inquiries, which required relatively low lift. Additionally, we wanted to pose the question, how can we utilize accrual strategies like CISTOquestion moving forward? Or how can CISTOquestion be adapted to other studies?

And CISTOquestion is something that we feel is generalizable to other large, pragmatic studies, but does require a degree of investment or designated personnel who can respond to email inquiries. And from our feedback survey from research coordinators, it received incredibly favorable responses from the sites that had used it. On that note, we thank you so much for your time and wanted to take the opportunity to thank the rest of our team at the University of Washington, as well as the folks listed in the slide for their invaluable contributions to this CISTO study.

Ruchika Talwar: Thank you so much. First of all, I want to applaud the entire team for publishing this type of paper. Because I think, too often, lessons learned in the process of these sorts of trials really go under-recognized and they're not openly discussed. I think there's a lot of benefit, particularly to young investigators, who endeavor to conduct these large trials of their own.

So kudos to the whole team for spending the time to create this sort of study. That was what excited me so much and what made me want to feature it and share some of these lessons learned. So just backing up a bit and thinking about the CISTO trial, in general, even outside of some of the lessons learned here, Dr. Smith, I'm curious if you would be able to share some of the biggest challenges that you and the whole team faced through the planning and execution phase.

Angela Smith: Yeah, great question. I think there were a few challenges that I would probably highlight. Predominantly being one, just the sheer size of the study. Thirty-eight sites, just a lot of different moving parts, a lot of different institutions trying to use a centralized IRB. All of those were challenges. But I think more so the type of disease that we were studying.

A mentor once told me, if you're studying something new, there's probably a reason someone hasn't studied it before. It's probably because it was hard. There's something about it. And I think that was true here because we know that BCG unresponsive disease is really complex. It's complicated. It's not simple. It's very hard to categorize it, even though we do our best and we do have categories. And that made it very hard and challenging to understand the inclusion criteria.

And then the third part was just we started during COVID. So we had all of the additional challenges of COVID, trying to go virtual, not being able to be in-person, all those other things layered on with many staffing challenges. So I will say that with every challenge comes an opportunity. And I think this lessons learned paper may not have been published had those challenges not been there for us to really work through. And so that really created the impetus for the paper.

Ruchika Talwar: Yeah, great points. Dr. Nathan, I'm curious, of the lessons learned that you presented and that are discussed in this paper, to you, which do you think was the most surprising?

Krupa Nathan: Yeah, I think after going through all of the data and seeing how people had responded—and particularly by people, I mean the research coordinators—had responded to the feedback survey that we had administered. What was surprising to me was how many research coordinators didn't perceive the all sites meetings being as helpful as CISTOquestion. And I think part of that is our association with labor or investment of labor as giving us the reward that we want.

And there were a lot of moving parts, like Dr. Smith had mentioned with the all sites meetings and coordinating attendance and asking poll questions. And there were a lot of moving parts in that as well and trying to coordinate everyone to come and to attend those meetings. So essentially, during those meetings, what would happen is that we would pose a poll question in which we would present a sample patient case and then poll the group about whether that patient was eligible for the study and which eligibility category that patient fell into. And each eligible patient would fall into a category based off treatment, timing, recurrence, et cetera.

And there were some difficulties with having poll respondents answer accurately on a consistent basis. And I think just based off of that association with the investment of labor with reward, I would presume that the all sites meetings would have been perceived as very helpful, but that wasn't the case. And CISTOquestion was actually perceived as way more helpful because it was something that could be individualized and tailored to the patients at that particular site. And that's something that the research coordinators really valued was that every patient was different. And so they loved being able to ask about a specific patient at their site and get a response very quickly.

Ruchika Talwar: Yeah, great point. And I mean, the other thing about coordination, like Dr. Smith mentioned, 38 sites—that's a lot of schedules to try to coordinate and have consistent attendance. And so I imagine that definitely would have been a barrier, too. Along those lines, Dr. Smith, I'm wondering, in addition to all sites meetings, CISTOquestion, we're in 2024, almost 2025, technology is booming. And so looking into the future, what are some ways that you think technology can potentially help investigators boost site engagement, enrollment, et cetera?

Angela Smith: Such a great question. And I'm smiling because I feel like any technology-related question, of course, the answer is AI. But I do think there is a potential ability to facilitate AI. I think one of the reviewers for this paper—I thought it was a really great question—talked about, well, how onerous was it for the PIs to have to be answering these CISTOquestion emails? And, I mean, it wasn't a lot of time, but it was time.

And I suspect that with AI, those could be easily answered over time. You can put in some of the inputs, understand—I really could see that even in a year's time, being a really helpful adjunct. Because I do think, even listening to Dr. Nathan's answer there, I wonder if they worked in tandem, the all sites meetings and the CISTOquestion. Like at the beginning, maybe some polls get you to just learn it a little bit. It's almost like an education. Once you've had that, you don't necessarily need to go to the meetings. Now, you need personalized content. And maybe that's where AI could be helpful.

Ruchika Talwar: Yeah, absolutely. So as we wrap up here, I'll ask you each—and I'll start with Dr. Nathan first—what advice do you have for other junior investigators, who might be interested in undertaking this sort of a trial? I mean, just going through the exercise of conducting this project, what were the big takeaways for you?

Krupa Nathan: Yeah, I think you had highlighted this earlier, that even trials that might not have a definitive objective for having a paper, I think learning as you're going through the trial and then writing about that is an important lesson in and of itself to take away from doing studies like these. I think what's not highlighted enough is doing operations-based research and how we can optimize the operations of certain studies. And that was a lesson for me as well.

And I think a lot of how we learn about research and get engaged with research is through these objective measures of the outcome of the research itself and not actually the operation side of things. So I think thinking about things like that in a microscopic level is something that I would push other people to do as well and that I learned a lot from this study.

I think I learned a lot about data management and all of the different components of the CISTO study and all the people engaged with the study, including all of the community practices and the role of patient advocates and how they played such a big role in this study. So it made me really value how we can apply patient-centered outcome studies to other fields as well outside of oncology.

Ruchika Talwar: Yeah, absolutely, great points. Dr. Smith.

Angela Smith: Yeah, I thought that was a great answer, and I agree with it completely. I'll just add that for any junior investigator doing this kind of work to be creative and flexible and also use all of the inputs around you. I still forget that I have to remind myself that I don't always have to have the solution myself.

Some of these ideas, like the all sites meetings and the CISTOquestion, came from just asking research coordinators and asking our patient advocates and asking a site PI—just ideas. It's like we're having these challenges. What could we do to help facilitate learning, help facilitate screening, et cetera? And it doesn't have to be just you. So that's just creativity, flexibility, and the ability to ask for help.

Ruchika Talwar: Yeah, absolutely. I love that. And again, kudos to both of you for pushing this work forward. I think this is such an important conversation. There's going to be so many takeaways here that are going to help guide future studies like these. And we're encouraged by these sorts of patient-centered outcome trials that really help us drive clinical decision-making in real time in complex disease states like BCG unresponsive bladder cancer. So thank you both again for taking the time to do this.

Angela Smith: Thank you for having us.

Krupa Nathan: Yeah, thank you so much for having us.

Ruchika Talwar: And to our audience, thanks for tuning in. We'll see you next time.