The purpose of this study is to develop a psychometrics of standard-setting for the Bookmark standard-setting procedure. Using simulated and real data, the error variance of Bookmark cutscores is modeled as a function of both within-group and between-group variance. Fully Bayesian methods are then used to estimate the total error variance associated with Bookmark cutscores. The results of the study suggest that the estimates of error variance for the proposed method tend to be larger than those calculated using traditional methods. Consequently, the confidence intervals were more likely to include the “true” cutscore in the simulation study. Because the result of a standard-setting study is almost always a recommendation rather than a final cutscore, one goal is to provide policymakers with a more accurate representation of the amount of uncertainty around a panel’s recommendation.
The University of Kansas prohibits discrimination on the basis of race, color, ethnicity, religion, sex, national origin, age, ancestry, disability, status as a veteran, sexual orientation, marital status, parental status, gender identity, gender expression and genetic information in the University’s programs and activities. The following person has been designated to handle inquiries regarding the non-discrimination policies: Director of the Office of Institutional Opportunity and Access, IOA@ku.edu, 1246 W. Campus Road, Room 153A, Lawrence, KS, 66045, (785)864-6414, 711 TTY.