Pressure Ulcer Risk and Prevention: Examining the Inter-Rater Reliability of the National Database of Nursing Quality Indicators® (NDNQI)

Abstract

ABSTRACT Measuring and reporting performance have become the norm. The purpose of this descriptive multi-site (N = 36 NDNQI-participating hospitals) study was to examine the reliability of the National Database of Nursing Quality Indicators® (NDNQI®) pressure ulcer (PrU) risk and prevention measures. This is the first known study to examine the inter-rater reliability of these measures. Data for Part 1 of this two-part study were extracted from 1,637 patient records by 120 raters. One rater at each hospital was considered the "expert". Agreement between the expert and non-expert raters was calculated for the risk measures. Among the patients, 530 were "at risk" for PrU, and included in calculations of agreement for the prevention measures. In Part 2, raters completed an online survey about the methods they use to collect these data. Cohen's kappa values varied widely within and across hospitals. Because most patients were assessed for PrU risk, and those at risk received prevention, the prevalence of a "Yes" response was high suggesting prevalence-adjusted kappa (PAK) may be a better estimate of inter-rater reliability than Cohen's kappa. PAK values for: Skin assessment, PAK = .977, 95% CI [.966 - .989]; Risk assessment, PAK = .978, 95% CI [.964 -.993]; Time since last risk assessment, PAK = .790, 95% CI [.729 - .852]; Risk assessment scale, PAK = .997, 95% CI [.991 - 1.0]; Risk status, PAK = .877, 95% CI [.838 - .917]; Any prevention, PAK = .856, 95% [.769 - .943]; Skin assessment documented, PAK = .956, 95% CI [.904 - 1.0]; and Pressure-redistribution surface use, PAK = .839, 95% CI [.763 - .916] indicated substantial to near perfect agreement. PAK values for: Routine repositioning, PAK = .577, 95% CI [.494 - .661]; Nutritional support, PAK = .500, 95% CI [.418 - .581]; and Moisture management, PAK = .556, 95% CI [.469 - .643] indicated moderate agreement. Results provide support for the reliability of all (5) PrU risk measures, and three of six prevention measures. Areas of disagreement between the expert and non-expert raters should direct education to improve reliability. Results of the online survey suggest raters need further training on the NDNQI guidelines for PrU data collection.