Using the bootstrap to establish statistical significance for relative validity comparisons among patient-reported outcome measures
UMass Chan AffiliationsDepartment of Quantitative Health Sciences
Document TypeJournal Article
Analysis of Variance
Outcome Assessment (Health Care)
Quality of Life
Renal Insufficiency, Chronic
Health Services Research
MetadataShow full item record
AbstractBACKGROUND: Relative validity (RV), a ratio of ANOVA F-statistics, is often used to compare the validity of patient-reported outcome (PRO) measures. We used the bootstrap to establish the statistical significance of the RV and to identify key factors affecting its significance. METHODS: Based on responses from 453 chronic kidney disease (CKD) patients to 16 CKD-specific and generic PRO measures, RVs were computed to determine how well each measure discriminated across clinically-defined groups of patients compared to the most discriminating (reference) measure. Statistical significance of RV was quantified by the 95% bootstrap confidence interval. Simulations examined the effects of sample size, denominator F-statistic, correlation between comparator and reference measures, and number of bootstrap replicates. RESULTS: The statistical significance of the RV increased as the magnitude of denominator F-statistic increased or as the correlation between comparator and reference measures increased. A denominator F-statistic of 57 conveyed sufficient power (80%) to detect an RV of 0.6 for two measures correlated at r = 0.7. Larger denominator F-statistics or higher correlations provided greater power. Larger sample size with a fixed denominator F-statistic or more bootstrap replicates (beyond 500) had minimal impact. CONCLUSIONS: The bootstrap is valuable for establishing the statistical significance of RV estimates. A reasonably large denominator F-statistic (F > 57) is required for adequate power when using the RV to compare the validity of measures with small or moderate correlations (r < 0.7). Substantially greater power can be achieved when comparing measures of a very high correlation (r > 0.9).
SourceHealth Qual Life Outcomes. 2013 May 31;11:89. doi: 10.1186/1477-7525-11-89. Link to article on publisher's website
Permanent Link to this Itemhttp://hdl.handle.net/20.500.14038/46652
Related ResourcesLink to article in PubMed
© 2013 Deng et al.; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.