Analyzing and adjusting for variables in a large-scale standardized-patient examination

Structuring a clinical performance examination that uses standardized patients (SPs) for large groups of examinees often involves the use of two or more parallel forms of the examination with different SPs portraying the same case on the different forms. In addition, each form may be administered more than once on different days and/or in different locations. Method. To determine the effects of critical variables, such as day of examination, time of day (am/pm), which of two simultaneous forms were taken, and sequencing effects, a univariate nested factorial analysis of variance was conducted for each of four annual SP examinations (1990-1993) at the University of Texas Southwestern Medical School. The examinations were given to approximately 200 second-year students per year at the end of their Introduction to Clinical Medicine course, and were graded on a pass/fail basis. Results. Statistically significant differences were found for the following variables: (1) time of day (am or pm) and day were significant but were inconsistent and of small magnitude; (2) sequencing for the first two stations was significant in each form of the examination and in all four years; and (3) form-within-case differences (i.e., differences between SPs) were significant between the two forms of the examination in each year of administration. To minimize the impacts of these variables, two mean equating formulas were applied to the scores. Few examinees’ pass/fail status would have been affected by either adjustment. Conclusion. The parallel-forms examination format is minimally affected by the variables evaluated and is a fair pass/fail assessment of a student’s performance. Mean equating is a valuable tool in minimizing the possibly unfair impact of variables on pass/fail decisions for homogeneous student populations.