Read the Full Video Transcript

Ruchika Talwar: Hi everyone. Welcome back to UroToday's collection of health policy articles. We're going to continue by recapping some interesting studies that were presented at the 2023 AUA annual meeting. Today I'm joined by Dr. Ray Tan from UNC Chapel Hill, who'll be chatting with us about one of the studies that he presented. Dr. Tan, thanks so much for joining us today.

Ray Tan: Thank you. It's awesome to be here.

Ruchika Talwar: So I wanted to chat a little bit about your project, the effective visual and numeric risk information on surgeon behavior, a randomized clinical vignette experiment. Tell me a little bit about the background of this specific study question.

Ray Tan: Yeah, I think as most people are well aware of there's a great deal of variation in the care that we provide for patients with urologic cancers. Depending on the clinical situation, the range can be as much as 0% to a hundred percent for specific providers providing a certain type of treatment for people. And so based on that, it seems that treatment is driven in large part by the physicians they see as opposed to the disease they may have.

And there's tons of literature out there that shows this to be the case. And a lot of that I think comes down to how people perceive the risk to be for a specific disease entity or for specific surgery. And I think people are generally just trying to make the best decision they have with the information they have on hand, and there's really a lot that goes into that in terms of experience, knowledge of what's latest and new.

And so with that as sort of the backdrop, we are interested in how do we convey information to people to really allow people to be the best version of themselves and to make the best decision and recommendations for their patients. And something that's been really going on in urology as well as other fields have been a lot of work really in trying to develop these risk prediction models and tools.

There's probably hundreds of publications that are out there, but at the same time, there is a lot of other work that shows that one, they're not really used with regularity and two, they may not have much impact. And so the objective of our study was can we develop a way of communicating that type of information in a way that can actually influence what a urologist may recommend for their patient?

Ruchika Talwar: Yeah, interesting. I think there's a lot of variation in the things that providers recommend for an index patient. There's also a lot of variation in what a specific provider recommends patient to patient at similar scenarios. So this is a really cool space, I think. Definitely an interesting focus of investigation. So tell me a little bit about the methods of your study.

Ray Tan: So I would describe the study as being mixed methods, user centered. Essentially what we did is that we proceeded with ... so what we are essentially are testing in the study is whether this visual risk display or a visual way of communicating information is any different from providing people just purely the numeric information versus sort of the control.

And so it's a randomized experiment of these six clinical vignettes, two in prostate cancer, two in bladder cancer, two in kidney cancer, and randomizing the vignettes plus or minus one of these exposures. In order to create the visual exposure, we actually went about in this user-center design process. And so user-centered design is something that's been used kind of throughout different industries, especially well used in tech to really place the user at the center of the design process, as we were going through it to get their feedback. Trying to adjust their specific needs and preferences as part of it as opposed to more of a top-down approach.

So a top-down approach would be creating the prediction tool and just the people who are creating it saying, "Okay, well I wanted it to be in a bar graph format," and just putting in a bar graph format and that's your product. And so what we did is that we convened ... we actually ... well first we did a national mixed method study in partnership with the American Urological Association, followed by interviews of urologists throughout the country. Try to get a sense of how they would want that information conveyed to them.

And then we convened a user panel of urologist, seven in total throughout the United States who saw patients with urologic cancers. And then went through a bunch of different tasks and design activities to really try to boil down and come up with the design. And that's what we ultimately tested in our experiment.

Ruchika Talwar: Okay. Interesting. So tell me what you found.

Ray Tan: So we found a bunch of different things and we're still going through the data right now, but what was interesting is that we found that there was a lot of variation in terms of what the perceived risk were to these different scenarios. So if you look at these scatter plots as an example, when we asked people to estimate what the 10-year risk of dying from a cause other than the cancer, the estimates really ranged all over the place.

So we saw that the perceived risk or perceived benefits of these different scenarios varied widely by urologist, which is pretty consistent with what the literature has reported out so far. What we found is that both visual and numeric information seemed to significantly reduce the variation and perceived risk, at least in what was answered and provided to these clinical vignettes compared to the control arm.

But what we found is that there wasn't really too much difference with respect to the visual and numeric information. There were certain instances where you do see maybe a potential difference, but really by and large, we didn't see much difference between the visual and numeric information. What we also found is that even when you present people with information in a way that they had asked for, so as an example, people wanted information that was multifaceted. They wanted information that was multiple domains.

So for the surgical decision, people don't just care about the cancer outcome risk, they care about the life expectancy, the cancer outcome, the outcomes without treatment, the complications from treatment. So when you provide all that holistically, it still didn't make a difference between whether a person was likely to recommend surgery or not recommend surgery. And so it seems that these risk prediction tools, even when you optimally design them, and I probably wouldn't say ... there's probably rooms to improve the design, but when you sort of design it in a way that's responsive to what surgeons want, doesn't really seem to budge whether they're going to recommend surgery or not.

It does seem to maybe have a little bit of impact on what services or what additional support care they may want the patient to have before a surgery. And so that's something that we did find. But in the bottom line, recommending surgery versus not recommending surgery didn't seem to budge much.

Ruchika Talwar: Interesting. So we do have our biases then I guess, and it's hard even when we use an objective risk calculator to try to sway perhaps what we recommend for a patient surgery versus not. It's interesting that it seems that we've made up our minds even before inputting that information, a little bit of the eyeball test, a little bit of us just making up our mind on chart review. So that leads me to the question, do these risk calculators help us make the decision or perhaps help us justify our decision to the patient? What do you think about that?

Ray Tan: I think in the current state, it's probably more the latter. I think this study, to me and my interpretation of it really confirms how intuitive decision making is. How much is it based on our intuition, our prior experience, our practice habits? And so I think it provides confirmation to those behavioral psychological aspects of surgical decision making.

Some prior work that we've done have identified thematically that surgeons who use respiration tools really do it for confirmation to make sure they're on the right track. Sometimes we'll use it to calibrate their own estimates or risk, but they also use it a lot for communicating with patients. So yeah, I think that's where risk prediction tools in their current form have their greatest function for surgeons.

And so I think that is a potential benefit of risk prediction tools. But I think if we're trying to use risk prediction tools to move, for example, people towards more active surveillance for certain situations, I think other approaches may be more effective or combination of different approaches may be more effective.

Ruchika Talwar: Yeah. What's next in this space, both for your specific future analyses, but also where do you see these risk prediction tools moving towards in the future?

Ray Tan: Yeah, that's a great question, Ruchika. I think there is a lot ... as I mentioned, I think people use these tools really for communication or that's one of the main functions that people deploy these tools for. And so what we're really interested in is seeing how trying to take this approach, how does it influence physician communication in the QuantX?

When they're talking to patients, if people have a chance to view this information in this format, is there a difference in how they communicate to patients and family members and other people who are across for them? The other thing is seeing how patients respond to it. Is this something that patients are responsive to? Is it something that is easily understandable to them and helps them in their decision making process?
And how does it work as we're approaching and optimizing the decision making from both angles? There's been a lot of work on the patient side and there's been a lot of work used for developing, decision aids and such. And this is something that's been a little bit more physician surgeon focused and so it'd be interesting to see what happens in the middle.

Ruchika Talwar: Yeah, I totally agree. Well, thanks for spending some time chatting with us about your analysis that you presented at the AUA. I think it's great to see that we are pushing the limits a little bit of how these tools have already been integrated into our practice, and I think this is really important work that'll guide the way that urologists continue to integrate such tools into their routine communication with patients.

Ray Tan: Oh yeah. No, thank you for the opportunity to share. And thanks to all of my colleagues in the urology community who helped participate in these surveys and interviews and everything. It's really great to have such awesome colleagues around the country who are willing to help out a fellow urologist in doing this stuff.


Ruchika Talwar: Yeah. All right. Thanks for joining us here and we'll continue to spotlight some interesting health policy findings from the annual meeting.