APCCC 2021: What Can Meta-Analyses, Network Meta-Analyses and Subgroup Analyses Tell Us About Treatment Options: Pearls and Pitfalls

(UroToday.com) The first session of the Advanced Prostate Cancer Consensus Conference 2021 which was hosted virtually in the context of the COVID-19 pandemic focused on the management of patients with newly diagnosed metastatic hormone-sensitive prostate cancer. In this context, Dr. Claire Vale discussed analytic issues including the role of meta-analysis, network meta-analysis, and subgroup analyses to inform care for patients in this disease space.

Dr. Vale began by highlighting the purpose of evidence synthesis which, as she describes it, is three-fold:

#1. Evidence synthesis collates high-quality research evidence;
#2. Through this, evidence synthesis guides clinical policy and practice; and
#3. Evidence synthesis should reduce research waste and duplication of research efforts by avoiding potentially unnecessary trials where a robust answer has already been derived.

However, she highlighted that there are numerous potential pitfalls for evidence synthesis. These including the choice of methodology which may be inappropriate or simply lack transparency. Further, there are issues with the quality of included data which may range from reported aggregate effect measures to individual patient data. Further, beyond the analytic concerns, there may be issues of the interpretation of findings or duplication of systematic review efforts. Issues of interpretation of meta-analyses may be particularly troublesome where these analyses are performed outside the context of a systematic review. In this case, the synthesized data may be incomplete and provided a biased summation of the literature.

Citing an editorial from Sionitis and colleagues in The BMJ, she suggested that “while some independent replication of meta-analyses is possibly useful, the overall pictures suggests a waste of efforts with multiple overlapping meta-analyses”. She then cited the sample of two different topics in metastatic castration sensitive prostate cancer in which she, or her group, had undertaken meta-analysis: the role of docetaxel treatment intensification and a network meta-analysis of abiraterone, docetaxel, and standard of care. By mid-2016, she noted 6 publications assessing the first of these questions with broadly similar results. In the context of the network mtea-analysis, 13 total overlapping studies were published with differing eligibility and statistical modeling approaches. However, she did not provide specific guidance on how we should address this duplicative effort, except to infer that it may not be worthwhile undertaking meta-analysis where others have done so.

In framing the context of best practice for meta-analysis, she emphasized that a pre-specific protocol should be used with a systematic review forming the backbone of any subsequent meta-analysis. Further, careful consideration should be undertaken as to what data will be combined. In particular, she suggested that it may be worthwhile to get “bespoke summary data” to have more comparable data between trials. Further, while individual patient data may be best, it is both resource intensive to acquire and analyse these data. Finally, she suggested that it may be worthwhile to undertake systematic reviews and meta-analyses in a collaborative fashion including clinicians and trialists, though she acknowledged that this is not a universally held view and some may argue that the inclusion of trialists may unduly influence the design of systematic reviews.

Moving next to a discussion of network meta-analyses, all of these same quality considerations continue to apply. However, we must further consider issues of transitivity. That is, network meta-analyses assumes that there are no systematic difference between the studies being used in the network apart from those treatments being compared. Therefore, systematic differences in study cohorts, follow-up, interventions, etc. may invalidate the assumptions necessary for a network meta-analytic indirect comparison. Further to this point, we may empirically assess the validity of these assumptions by comparing the direct and indirect treatment effects.

Finally, Dr. Vale moved to discussing subgroup meta-analyses. She emphasized that, when considering the rationale and validity of these analyses, we must consider whether the groups have been sensibility design and whether, in both the initial trials and in the evidence synthesis, the subgroup analyses were pre-specific or post-hoc. If subgroup analyses are not included in the protocol, then any results should be considered exploratory. Finally, from an analytic perspective, Dr. Vale emphasized that we should be pooling test for interactions, rather than simply pooling the results from each subgroup as is commonly done.

In summary, Dr. Vale emphasized that the methodology is as important as the results of a meta-analysis. As with more research, data quality is key and not all studies are therefore created equally. To best design, execute, and act on the results of systematic reviews and meta-analysis, Dr. Vale suggested that communication between trialists, systematic reviewers, and policy makers is critical.

Written by: Christopher J.D. Wallis, University of Toronto, Twitter: @WallisCJDduring the 2021 Advanced Prostate Cancer Consensus Conference, Saturday, October 9, 2021.