QJM Advance Access published online on October 8, 2008
QJM, doi:10.1093/qjmed/hcn131
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Teaching critical appraisal and statistics in anesthesia journal club
From the 1Department of Anesthesiology, 2Research and Development Center and 3Department of Community Medicine, Tehran University of Medical Sciences, Sina Hospital, Tehran, Iran
Address correspondence to M.R. Khajavi, MD, Sina Hospital, Tehran-11367-46911, Iran. email: mohammadreza.khajavi{at}gmail.com
Received 21 April 2008 and in revised form 9 September 2008
 |
Summary |
|---|
 Top Summary IntroductionMaterial and methodsResultsDiscussionAcknowledgementReferences |
|---|
Background: This study was designed to assess the effectiveness of a series of journal clubs held for anesthesiology residents in promoting their awareness of research methods and statistical analysis, as well as their skills in critical thinking and appraisal.
Material and methods: Twenty-four journal club sessions were held between September 2006 and August 2007 for 16 residents of anesthesiology. A 31 multiple-choice question (MCQ) was taken as pretest and posttest to evaluate the participants’ level of awareness in research methodology and statistical analysis. Their competence in critical thinking and appraisal was also evaluated by evaluating a randomized controlled trial paper using the CONSORT checklist before and after the course.
Results: Residents’ awareness in the application of information improved (P = 0.012), as well as research methodology (combined study design and application of information, P = 0.017). Their ability in critical appraisal did also significantly rise at the end of the course (P < 0.001).
Conclusion: Journal clubs can enable residents to develop the knowledge, expertise and enthusiasm needed to undertake research plans and can also enhance their ability in critical thinking and scientific reading.
|
Introduction |
|---|
|
TopSummaryIntroductionMaterial and methodsResultsDiscussionAcknowledgementReferences |
|---|
Residency training represents an ideal time when clinicians may learn and practice the necessary skills for searching, evaluating and applying medical knowledge. The journal club may play an important role in this regard.1 The present study was designed to assess the effectiveness of a regular journal club in teaching anaesthetic trainees research methodology and critical appraisal skills.
|
Material and methods |
|---|
|
TopSummaryIntroductionMaterial and methodsResultsDiscussionAcknowledgementReferences |
|---|
Twenty-four journal-club sessions (biweekly for 1 h) were held between September 2006 and August 2007 for residents of anesthesiology (years 1–3). Attendance was closely monitored. In each session, a single article relevant to anesthesiology and critical care medicine was discussed with respect to statistical analyses and research methodology (type of study, biases and methods). Suitable articles were identified by Medline and EMBase English searches and were given to the participants 4 days beforehand. At the beginning of each session, a senior resident presented a 10-min summary of the article including its objectives and findings. Then, a senior clinician focused in detail on a single methodological issue according to a prepared syllabus (principles of statistics and research; concepts of risk, exposure, odds and bias; block randomization; case–control studies, cohort studies, randomized controlled trials and use of the CONSORT checklist).
In the early sessions, theoretical issues were discussed in a didactic style. As the knowledge of the residents improved, the sessions became mainly student-centered. To assess the overall improvement in the participants’ awareness level, an identical pretest and posttest including 31 multiple-choice (MCQ) questions was administered (Chronbach-
score = 0.7); their mean values were compared using paired samples test. Subgroup analyses were also performed for research methodology (22 MCQs, 11 for study design and 11 for application of information) and statistical analysis (nine MCQs), using Student's t-test. In addition, to evaluate the participants’ competence in critical thinking and appraisal, they were asked to appraise a single randomized clinical trial (RCT) paper, using the CONSORT checklist, before and after the course. Two professors were also asked to do the same test to provide the reference answers; in case of any discrepancy, a third professor was asked to do the test. Participants were instructed to give points to each of the 22 individual items in the CONSORT checklist between 0 (not mentioned in the article at all) and 1 (discussed in the article). Then, the scores of individual items on every answer sheet were summed to yield a total score for its respondent. The mean values of these total scores before and after the course were compared using Paired Samples Test.
|
Results |
|---|
|
TopSummaryIntroductionMaterial and methodsResultsDiscussionAcknowledgementReferences |
|---|
There was a significant improvement in the participants overall awareness level (Table 1) and this was clearly for ‘application of information’ but not for ‘study design’ or ‘statistical analysis’. After combining study design and application of data (method-ology), the study revealed a significant difference between the pre and posttest scores (P = 0.017).
|
The pretest score on critical thinking and appraisal was deliberately considered zero as all residents declared a lack of knowledge in this area. However, their posttest appraisal total score (mean ± SD, 27.06 ± 2.98) showed significant improvement (Table 2). However, when we segregated the total 22 items of the CONSORT checklist into 14 well-discussed items (19.56 ± 1.82) and 8 poorly-discussed items (7.50 ± 1.58), there appeared a significant difference between the two compartments (mean difference = 12.06; 95% CI 10.90–13.21; P < 0.001 using unpaired t-test).
|
|
Discussion |
|---|
|
TopSummaryIntroductionMaterial and methodsResultsDiscussionAcknowledgementReferences |
|---|
The potential use of a journal club in improving reading habits1 and awareness of biostatistics2 has already been described. Other authors have also indicated that teaching the principles of critical appraisal may also be achieved by this means.3–5 Another review of six studies confirmed the journal club's significant role in promoting critical appraisal skills6 but this view is not widely held.7
Our study similarly suggested an improvement in critical appraisal competency and in epidemiologic awareness level of the residents. We concur with Kuhn et al.8 that the unavailability of trained faculty members in the very areas, lack of sufficient time and budget shortages are the factors which can limit the effectiveness of the journal club. We also found that the participants were unaware of their lack of knowledge of epidemiology and critical appraisal skills, a situation that improved by the end of the sessions.
In summary, we conclude that the journal club represents a simple but effective means of effectively engaging residents in developing their research and critical appraisal skills. We recommend that further research we control groups and larger sample sizes should be performed in order to fully exploit the teaching potential of the journal club.
|
Acknowledgement |
|---|
|
TopSummaryIntroductionMaterial and methodsResultsDiscussionAcknowledgementReferences |
|---|
We acknowledge the support of the Research and Development Center of Sina Hospital.
|
References |
|---|
|
TopSummaryIntroductionMaterial and methodsResultsDiscussionAcknowledgementReferences |
|---|
1. Linzer M, Brown JT, Delong ER, Siegel WC. Impact of medical journal club on house-staff reading habits, knowledge and critical appraisal skills: a randomized, control trial. J Am Med Assoc (1988) 260:2537–41.
2. Cheatam ML. A structured curriculum for improved resident education in statistics. Am Surg (2000) 66:585.[Web of Science][Medline]
3. Kellum JA, Rieker JP, Power M, Powner DJ. Teaching critical appraisal during critical care fellowship training: A foundation for evidence-based critical care medicine. Crit care Med (2000) 28:3067–70.[CrossRef][Web of Science][Medline]
4. Seeling CB. Affecting residents’ literature reading attitudes, behaviors and knowledge through a journal club intervention. J Gen Intern Med (1991) 6:330–4.[Web of Science][Medline]
5. Cramer JS, Mahoney MC. Introducing evidence based medicine to the journal club, using a structured pre a post test: a cohort study. BMC Med Educ (2001) 1:6.[CrossRef][Medline]
6. Ebbert JO, Montori VM, Schultz HJ. The journal club in post graduate medical education: a systematic review. Med Teach (2001) 23:455–61.[CrossRef][Web of Science][Medline]
7. Bennett KJ, Sackett DL, Haynes RB, Neufeld VR, Tugwell P, Roberts R. A controlled trial of teaching critical appraisal of the clinical literature to medical students. J Am Med Assoc (1987) 257:2451–4.
8. Kuhn GJ, Wyer PC, Cordell WH, Rowe BH. A Survey to determine the prevalence and characteristics of training in evidence – based medicine in emergency medicine. J Emer Med (2005) 28:353–9.[CrossRef]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||