Can we predict "problem residents"?

Purpose: This study investigates whether data available at the time of residency application can be used to predict more accurately future problems of performance, both during and after residency. Method: The authors identified all residents with reported problematic behavior across 20 years (1987-2007) at a single residency program and created a set of matched controls. Problems were further divided into "major" (leading to significant disruptions of performance and disciplinary action) and "minor" (remediable and resolved). Application materials were then reviewed for United States Medical Licensing Examination (USMLE) scores, evidence of academic failures, interviewer ratings, negative interviewer comments, negative comments in the dean's letter, and negative comments in letters of recommendation. Results: The presence of any negative comments in the dean's letter yielded significant correlations with future problems. Further, those applicants with future major problems had significantly more negative comments in the dean's letter than did those with future minor problems. Other factors such as USMLE scores, failed courses, letters of recommendation, and interviewer ratings and comments did not predict future problems. Conclusions: Most of the factors the authors assessed in prospective applicants did not predict future problems, with the exception of negative (even mildly so) comments in the dean's letter. The authors suggest that more attention should be paid to the use of the dean's letter to assess risk among applicants, and prospective study of this assessment should be performed.