Reliability and validity of PAEA end of rotation™ examination scores for predicting performance on the physician assistant National Certification Examination

Purpose: In 2013, the Physician Assistant Education Association (PAEA) launched its End of Rotation™ examinations for student assessment of core clinical rotation content. Although being similar to the National Board of Medical Examiner's subject examinations ("shelf exams"), these examinations have not been characterized for reliability or validated against outcomes such as the Physician Assistant National Certification Examination (PANCE). This study's objective was to determine whether a composite scale based on all 7 PAEA End of Rotation examinations can be used as an effective measure of comprehensive medical knowledge for physician assistant (PA) students in their clinical year. Methods: This study was a retrospective analysis of deidentified data gathered by 3 PA programs. Scores for all 7 PAEA End of Rotation examinations and for the PANCE were available for all students in the class of 2014 for all programs, with an overall N = 134. SPSS version 21 was used for descriptive and inferential data analysis. Results: The overall Cronbach alpha for the 7 End of Rotation examination scores was 0.871. Pearson correlation coefficients for the End of Rotation examinations and the PANCE ranged from 0.47 to 0.68 and were statistically significant. Simple regression of a composite 7-item End of Rotation examination average gave an R of 0.810 and an adjusted R = 0.654 (P < .0005), explaining 65% of the variance in PANCE scores. Conclusions: This study provides solid evidence for the reliability of the composite 7-item End of Rotation score and the validity of using this composite score for the purpose of predicting whether the medical knowledge base in clinical PA students is adequate to pass the PANCE.