Introduction and Objectives
Interrater reliability (IRR) of the pressure flow study (PFS) in multicenter trials is difficult to achieve.
Specific criteria were developed to assist reviewers in determining PFS validity prior to interpretation. These included assessments of properly functioning pressure measuring systems at baseline and maximum flow. Precise guidelines were provided to assist the reviewers in determining parameters of pressure and flow and detailed descriptions were developed for classification of voiding mechanism and of EMG relaxation during voiding. We now report on the PFS IRR between central reviewers (CR) and the local physician reviewers (LR) using these modified guidelines.
Methods
65 PFS tracings from 13 UITN-certified testers and 9 centers that had previously been interpreted by the LR and entered into the network’s database were randomly selected for this study. Each CR was assigned 16 or 17 tracings to interpret using the same set of guidelines that the LR had used. The guidelines instructed the reviewers to assess the validity of the study prior to interpretation. Only those studies that were classified as valid were reviewed. Intraclass Correlation Coefficients (ICC) were calculated for numerical variables. The percentage of cases where both the LR and the CR agreed and a kappa statistic (k) were calculated for the voiding pattern. Acceptable agreement was defined as ICC or k 0.6. Pvalues were obtained as a test of the hypothesis that reviewer means are equal.
Results
Of the 65 cases that were centrally reviewed, the LR and CR agreed on whether the study was valid in 56 (86%) and disagreed in 9 (14%) for a Kappa of 0.64 (0.42, 0.85). The IRR for continuous and qualitative PFS parameters was calculated for those cases that were considered valid by both the LR and CR (n=44). Excellent IRR was demonstrated for all pressure and flow parameters with correlation coefficients above 0.9 for all variables. The agreement for qualitative parameters was not adequate demonstrating overall agreement of 50 % for voiding mechanism (kappa 0.11) and agreement of 66% for whether the urethra relaxed during voiding (kappa 0.50).
Conclusions
Excellent IRR between central and local reviewers can be achieved for quantitative variables obtained in multicenter PFS when they are performed with a standardized protocol and interpreted with standardized guidelines. Agreement remains poor for qualitative variables such as voiding mechanism and what constitutes EMG relaxation during voiding.
Keywords
Urinary incontinence, urodynamics, pressure flow study, interrater reliability