Institute for Clinical and Economic Review: BCG Unresponsive Disease

The ICER Process

To address the importance of high-value care in the context of affordability and access, the Institute for Clinical and Economic Review (ICER) an organization whose mission is to conduct evidence-based reviews of health care interventions, independently reviews evidence, free from financial conflicts of interest, to understand an intervention’s ability to extend or improve life, a fair price based on clinical evidence, and how stakeholders can translate evidence into real-world insurance coverage to improve patient outcomes. The ICER process is multi-fold, rigorous, inclusive and systematic, based upon a Value Assessment Framework conducted in 5 steps.¹

Step 1: ICER notifies selected stakeholders and creates a draft scoping document to inform an understanding of the disease and available treatments. This document is informed through public input and scoping calls with clinical experts, patients, pharma, and payers.

Step 2: A draft evidence report is created through a formal literature search and a subsequent public comment period. This draft report includes background of the disease, patient and caregiver perspectives, evidence review and preliminary cost-effectiveness analyses. Importantly, the report is not intended to make policy recommendations, but can certainly inform them.

Step 3: Following a four-week public comment period, a revised evidence report is drafted. This report includes “health benefit price benchmarks,” a price range that is intended to align fairly with a treatment’s benefits over a patient’s lifetime.

Step 4: ICER holds a public meeting to present findings of the revised evidence report. During this meeting, an overview of the findings is described followed by speakers who give oral public comments. A roundtable of stakeholder experts then vote about how best to apply the evidence and impact policy.

Step 5: A final report and meeting summary are published, released 3 weeks following the meeting.

ICER recently convened to evaluate two new agents for BCG unresponsive disease: nadofaragene firadenovec and oportuzumab monatox. The goal of this report is to discuss the findings of the ICER report, ICER recommendations, limitations of the analysis and comments about the implications and future directions.

BCG unresponsive analysis: what was modeled?  General assumptions and conclusion

The patient population addressed in ICER’s review is BCG-unresponsive high risk non-muscle invasive bladder cancer (NMIBC) which includes 1) CIS ± Ta/T1; or high grade (HG) Ta/T1 without CIS. Patients whose cancers did not respond to a reasonable course of treatment with BCG and patients whose cancers recurred after treatment within a short period of time (6-12 months) were also included.  ICER developed cost-effectiveness models to evaluate two new agents for BCG unresponsive disease: nadofaragene firadenovec and oportuzumab monatox. Pembrolizumab and intravesical gemcitabine ± docetaxel were also included in the analysis for comparison.

Models were developed based on available efficacy and toxicity data, then extrapolated longitudinally since long term follow-up does not exist for most agents. Information for nadofaragene firadenovec, oportuzumab monatox, and pembrolizumab was primarily derived from pivotal prospective studies while information for intravesical gemcitabine ± docetaxel was retrospective. ICER concluded that the lack of comparative data limits the ability to compare these new agents to one another and to other therapies. Specifically, differences in study population, design and outcomes were felt to be too great for meaningful comparisons. Differences in the outcome assessed and patient censoring can impact comparability across trials. For example, even among the new agents, the nadofaragene firadenovec study has a mandated biopsy requirement at 12-months while studies of oportuzumab monatox and pembrolizumab do not.

Model Assumptions

With any hypothetical model, assumptions are required for analysis. In many cases, the assumptions result from lack of data, whereas other assumptions are required to simplify modeling. In this ICER study, examples of reasonable assumptions include “patients who are disease-free or who have metastatic disease will not have a cystectomy” and “utilities for the metastatic state originating from other cancers are similar to the utilities for metastatic bladder cancer.”  However, the study also assumed that patients progress stepwise between bladder cancer stage states (i.e., disease free to NMIBC to MIBC to metastatic disease). Unfortunately, this assumption is not always correct. Urologists will see patients “jump” stages, such as the patient with history of high risk NMIBC who has a negative three-month cystoscopy only to show cT2/3 disease, N+ or M+ disease in a subsequent CT scan. Failure to account for such scenarios in which the window of opportunity is missed will impact model results through underestimation of utilities. Similarly, the assumption that “states of persistent or recurrent NMIBC have similar utilities and costs” is inconsistent with clinical practice.  A patient with BCG-unresponsive disease who tried a new therapy and has persistent disease at the three-month cystoscopy will be addressed differently by both the patient and provider than a patient who remains disease free after starting new therapy until the 12-month cystoscopy.  Whether these assumptions lead to meaningful alterations in model outputs is unknown, but it is important when reading such studies to recognize and understand the model assumptions.

In addition to model assumptions, the model inputs for clinical outcomes, cost and health utilities determine the results. The ICER analysis was able to adjust the probability of moving from disease-free to recurrence/progression for nadofaragene and oportuzumab using data from clinical trials.  Since the rate of failure over time is non-linear, allowing separate probabilities at different three-month evaluations is important for the model.  The lack of comparator therapy, placebo or randomization from the clinical trials of BCG-unresponsive disease limited the ICER analysis’ ability to directly compare treatments.  As a result, the researchers compared the therapies to a hypothetical treatment with base efficacy of 0% and without inclusion of any costs or disutility associated with adverse events from the hypothetical treatment.  Since neither nadofaragene nor oportuzumab are commercially available currently, the analysis could not provide cost data for the base analyses. The cost of nadofaragene was set to the annual price of pembrolizumab, and oportuzumab cost was set at $150,000 based on estimates provided from communications with Sesen Bio.  The true cost is yet to be determined.  Indirect costs and loss productivity as a result of treatments were also not considered in the analysis.   Finally, health utility for cystectomy and post-cystectomy state is important when treatment success avoids cystectomy.  Unfortunately, such data are limited for modeling.  In the ICER analysis, health utility was estimated from a study that “derived the utility of an uncomplicated, post-cystectomy health state from a standard gamble involving 25 urologists and urology trainees at our institution.”²  Clinicians’ perception of post-cystectomy health state of individuals likely over-estimates the true health utility of an individual.  Considering the >50% complication rate following cystectomy, use of an uncomplicated case as the standard for utility assessment also over-estimates the true health utility. The reader should carefully look for model inputs for any hypothetical model and acknowledge the limitations when interpreting the results.  Sensitivity analyses are an important part of these studies allowing for varying of input. 

ICER Findings:

• Nadofaragene firadenovec and oportuzumab monatox appear to have few serious side effects and given their intravesical administration, may be safer than pembrolizumab that is given systemically. Nevertheless, as new therapies, potential side effects of nadofaragene firadenovec and oportuzumab monatox will require longer term evaluation in more patients.

• Nadofaragene firadenovec and oportuzumab monatox show one year response rates that are similar to or better than currently available treatments. Efficacy over longer time periods remain uncertain.

• Most patients receiving nadofaragene firadenovec or oportuzumab monatox progress or recur over time, it is possible that by delaying potentially curative cystectomy these treatments may lead more patients to develop metastatic disease or die from bladder cancer.

• A number of chemotherapeutic drugs, such as intravesical gemcitabine ± docetaxel, have been used for patients with BCG-unresponsive NMIBC. Despite differences in patient populations and study design making any direct comparisons exceedingly difficult, similar outcomes and expected lower costs suggest that trials comparing these older chemotherapeutic drugs with newer agents are warranted.

For cost-effectiveness assessment, ICER concludes:

• Compared to the hypothetical treatment, nadofaragene firadenovec is cost-effective in both the CIS with or without HG Ta/T1 group and the non-CIS HG Ta/T1 group while oportuzumab monatox is cost effective only in the HG Ta/T1 group, and pembrolizumab is cost-effective in the CIS with or without HG Ta/T1 group.

• Compared to the hypothetical treatment, gemcitabine ± docetaxel are cost-effective and dominates in both the CIS with or without HG Ta/T1 group and the non-CIS HG Ta/T1 group.

• The health benefit price benchmark price range for nadofaragene firadenovec is $158,600-$262,000 per year, and for oportuzumab monatox it is $92,800 to $162,100 per year.

ICER recommendations

ICER recommendations are provided with a focus on payers, manufacturers, patient advocacy organizations, providers, clinical and specialty societies, regulators and researchers.

From a payer perspective they note that trials of nadofaragene firadenovec and oportuzumab monatox included only patients with BCG-unresponsive NMIBC so use should be limited to these patients. ICER noted that the evidence is too limited to be able to distinguish the clinical effectiveness among nadofaragene firadenovec, oportuzumab monatox, pembrolizumab, and standard chemotherapy options (e.g., gemcitabine/docetaxel).  Despite lack of comparative evidence,  they raise the question whether payers should consider “economic” step therapy to seek cost savings since there are significant cost differences between treatment modalities, yet acknowledge that “analysts believe that only a minority of payers (10-20%) will ultimately implement step therapy for this population.  On the other hand, they recognize important differences in the types of side effects, regimen complexity, and location of treatment that would lead patients to have strong preferences for certain treatment options. As such, requiring treatment with certain drugs like intravesical chemotherapy prior to use of other agents in unlikely to occur.

Recommendations to manufacturers include acknowledging that “single-arm trials usually fail to provide the kind of evidence that is needed to help patients, clinicians, and insurers understand the comparative clinical effectiveness and value of new treatments. Manufacturers developing new treatments for BCG-unresponsive NMIBC should therefore use randomized trials as the basis for regulatory approval.”  ICER notes that bladder cancer has a significant impact on patients and families. Most patients will recur despite therapies for BCG unresponsive disease and whether delaying cystectomy is worth the risk of metastatic disease and disease-related mortality is unknown. ICER also recommends that “manufacturers should set prices for new therapies based on their demonstrated added clinical value over lower-cost clinically appropriate regimens.”

ICER applauds patient groups advocating for bladder cancer and encourages these groups to “speak explicitly about the impact of the high cost of treatments for BCG-unresponsive NMIBC.”

ICER recommends that providers should engage in a shared decision-making process with their patients and not let their treatment recommendations be unduly swayed by the perverse incentives that often pay clinicians more for administering more expensive treatment options.  Providers should be protected from the cost of buying these expensive drugs and at the same time, paying providers based upon a percentage of the drug’s cost would create perverse incentives for their use.

Regulators have an important role to play in how new therapeutics enter clinical practice. The lack of a clear consensus on “standard care” for BCG-unresponsive NMIBC provides no justification for the FDA’s failure to require randomized trials comparing emerging therapies to active regimens. The FDA should no longer accept single-arm trials as the basis for regulatory approval of NMIBC therapies.  They also advocate that researchers should compare nadofaragene firadenovec and oportuzumab monatox to other therapies in randomized trials of patients with BCG-unresponsive NMIBC.

Limitations with analysis

ICER develops cost effectiveness models based on available data.  The optimal data for modeling includes prospective, randomized studies with long term follow up and discrete data fields including side effects, costs and survival.  However, in assessing new technologies this data is often lacking especially with heterogeneous populations and rare diseases.  The limitations in analysis for patients with BCG unresponsive disease are multifactorial.   The FDA approved the use of single arms studies to evaluate new therapies in large part due to the failure to accrue to randomized trials which had significantly hindered introduction of any therapies for these patients for close to two decades.³  As such, it is not surprising that this was the approach selected in developing nadofaragene firadenovec, oportuzumab monatox, and pembrolizumab.  ICER did not have a good comparator to use in their model because valrubicin was the only approved agent for CIS unresponsive to BCG in this space (and rarely used in clinical practice at that) until approval of pembrolizumab in January of 2020, both of which were also approved following non-randomized trials.  The use of retrospective data on intravesical gemcitabine with or without docetaxol was not a “fair” comparison due to differences in quality of data, relatively short term follow-up and relatively low numbers of patients who met strict criteria for BCG unresponsive disease in these studies.  For example, in the recent multicenter retrospective study on intravesical gemcitabine and docetaxol, only 38% of patients were BCG unresponsive compared to 100% of BCG-unresponsive in prospective studies of nadofaragene and oportuzumab. 

The model developed by ICER focused on outcomes and cost.  Comparing outcomes between prospective studies is easier since data are available at various time points but challenges remain.  For example, some trials had mandatory biopsies while others do not.  A mandatory biopsy is more likely to identify carcinoma in situ than waiting for a lesion to appear due to inherent limitations with white light cystoscopy and cytology.  Furthermore, not every patient undergoes cystoscopy exactly at 3 or 6 months so determining how to estimate times to event can have an impact.  More importantly, determining long-term events makes a significant difference when trying to model over 5 years or a lifetime if trials only have robust data for 12 to 18 months. 

Another important consideration is cost of drug toxicity.  There are important differences in toxicity of non-systemic intravesical therapies when compared to systemic therapy. In the nadofaragene trial, less than 5% of patients experienced grade 3–4 drug-related adverse events and there were no treatment-related deaths.  Similarly, oportuzumab was generally well-tolerated with 52% of patients experiencing treatment-related adverse events (AEs), the majority being grade 1-2, and only 3% of the patients discontinued treatment due to AEs.

By comparison, pembrolizumab therapy is a systemic therapy, and 12.7% of patients had grade 3–4 treatment-related adverse events and over 20% of patients experience immune-related adverse events (colitis, pneumonitis, and hypothyroidism). ICER’s cost-effectiveness assessments did not include the costs or quality of life implications of some of these toxicities which could have significant cost-effectiveness implications.  Finally, it is difficult to model patient preferences in terms of type of treatment they prefer and the quality-of-life implications of choosing cystectomy versus a novel therapy.  These patient-related issues were not included in the analysis and quality of life implications are important in patient and physician shared decision-making regarding treatment for BCG unresponsive disease.

Finally, it is not clear how many urologists or patients were aware of  the public comment period. Since ICER recommendations can have an impact on access to care, it is important that more education on the process is available to all stakeholders.  

Our Commentary: future directions, need for randomized vs single arm trials, patient perspective on cost vs effectiveness

Patients with high grade NMIBC who are unresponsive to BCG face a challenging dilemma. They are at risk for progression and death from bladder cancer.  The most effective therapy remains radical cystectomy but results in life-altering surgery, a high rate of morbidity and a risk for peri-operative mortality.  The current alternative options include intravesical chemotherapy either FDA approved (Valrubicin) or non-approved (gemcitabine +/- docetaxel) and systemic immune therapy (pembrolizumab).  The option of conservative management (i.e., observation) is ineffective and not one that patients or providers willingly pursue. 

There were two new therapies under evaluation by ICER including nadofaragene firadenovec and oportuzumab. The focus of ICER recommendations is cost-effectiveness and yet physicians and patients may have different perspectives than the ICER analytic perspective of the US Healthcare System. Physicians and patients wish to extend life and quality of life. However, cost is often a secondary consideration with the exception of scenarios in which cost limits access to care. While the concern for a life-long management strategy is important, it is often difficult to make decisions for the next 5 to 10 years when considering therapies with trials that report 12-to-18-month outcomes.  A patient often wants to know how likely a therapy will achieve a complete response and how long will it last.  Also, what are the side effects of treatment and what are the odds that if they pursue this treatment their disease will progress to an incurable state.   

The goals of ICER to encourage randomized trials and cost-conscious treatment undoubtedly have merit. We recognize that bladder cancer is one of the most expensive cancers to treat from diagnosis to death.⁴ On the other hand, current drugs were developed with FDA guidance as single arm trials. Once several agents are approved, the option of single arm trials will likely disappear. Future agents will need to demonstrate superiority over approved regimens as is common for other disease states.  Cost considerations are also important for the health care system. One comparison that ICER did not perform was the cost of cystectomy as an initial option for all patients.⁵ We recognize that some patients will follow recommendation to undergo cystectomy for BCG unresponsive disease, but it is also important to have multiple available options for those patients who are ineligible for cystectomy or unwilling to proceed.

The CISTO comparative-effectiveness research study will provide valuable information on the risks and benefits to each approach (cystectomy vs. alternative medications) NCT03933826.  There will also be important information that will help in the understanding of true costs of these medication with further longitudinal data from the pivotal trials.  Also, many patients may undergo sequential therapy which will impact overall cost of care.

At the end of the day, patients with BCG unresponsive disease need effective options and some will likely prefer to try one or two therapies prior to undergoing cystectomy.  Nadofaragene and oportuzumab represent important therapeutic options for patients with BCG unresponsive NMIBC. To facilitate decision-making regarding sequential therapy in this challenging disease space, future studies must focus on comparative effectiveness research design to highlight meaningful differences in efficacy, patient-reported outcomes and cost effectiveness. Meaningful long-term comparisons will enable patients and providers to ultimately choose the right treatment for the right patient at the right time.

Yair Lotan¹, Jonathan L Wright⁴, Angela B Smith^3,4
1Department of Urology, University of Texas Southwestern Medical Center
²Department of Urology, University of Washington School of Medicine, Seattle, Washington
³Department of Urology, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina.
⁴Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina.