Effect of an evidence-based medicine seminar on participants' interpretations of clinical trials: A pilot study

Purpose. To evaluate the effect of evidence-based medicine (EBM) education on physicians' short-term and long-term understanding of research methods and statistics. Method. Twenty-four gastroenterology (Gl) fellows attended a three-day seminar about evidence-based medicine and the critical appraisal of medical literature. Attendees completed the same 14-item test on this material at the start of the seminar, at the conclusion of the seminar, and six months after the seminar. A Student's t-test and chi-square analysis were performed to determine the differences between test scores by testing date and performance on test items. Results. Seminar attendees improved their test scores between pre-seminar and post-seminar tests (mean test score: 57% ± 16% versus 82 ± 14%, respectively; p < .001) and between pre-seminar and six-month post-seminar tests (mean test score: 57% ± 16% versus 78% ± 13%, respectively; p < .001). Seminar attendees showed significant improvement in frequency of correct answers with individual questions on concealment of allocation, relative risk reduction, and meta-analysis trial methods. Conclusions. In this pilot study, the critical appraisal skills necessary to practice EBM were taught to Gl fellows in a seminar format that led to significant improvement in their understanding of research methods and statistics. Data from this pilot study justify a definitive trial examining the educational value of EBM seminars for physicians.