QJM vol. 97 no. 9 © Association of Physicians 2004; all rights reserved.
Commentary |
Critical thinking and critical appraisal: the chicken and the egg?
From the 1Department of Epidemiology and Public Health, Queens University, Belfast, and 2Royal Victoria Hospital, Belfast, UK
 |
Introduction |
|---|
 Top Introduction Cognitive concepts and their...Strategies for problem solving...Analysing argumentsReferences |
|---|
Having attended the first National Workshop on Teaching Evidence-Based Medicine (Oxford, 1995), one of us actually has the T-shirt declaring his critical appraisal credentials. But although trained in clinical epidemiology, both before and since the workshop, the same person continues daily to doubt his critical thinking skills. The profusion of mini-guides and short courses in Critical Appraisal and Evidence-Based Practice has enriched our curricula for over a decade,1 but perhaps basic skills of critical thought should take precedence in developing our day-to-day practice, which is rather less informed by the published evidence than we would care to admit.
Although the term critical thinking was used in the General Medical Council's 1993 edition of Tomorrow's Doctors, it has now been dropped in favour of phrases such as: an ability ‘to gain, assess, apply and integrate new knowledge ... ‘; or an ability ‘to integrate and critically evaluate evidence ... ’.2 While a shift towards problem-based learning has occurred in some UK medical schools, the proof that such curricula can enhance critical thinking may be elusive.3 However, many of the contributions to the teaching of critical thinking have not come from medical schools, but rather from philosophy and psychology departments, and there are now regular International Conferences on Critical Thinking.4 Perhaps it is time to learn some lessons from the ‘professionals’. Below we describe a few local initiatives to promote critical thinking at an undergraduate and post-graduate level. We give a brief overview of some material covered in a one semester course (for third-year medical students), and briefly describe a few take-home messages that postgraduates have also found helpful. Space permits description of only a small selection of this material.
|
Cognitive concepts and their application to clinical judgment |
|---|
|
TopIntroductionCognitive concepts and their...Strategies for problem solving...Analysing argumentsReferences |
|---|
To be human is to be fallible. Further, most of us prefer to expend the least possible effort in achieving our goals, including thinking about how to achieve them. In the words of epidemiologist Carol Buck, ‘If our motor engines were as erratic as our deliberate intellectual efforts, most of us would not get home for supper’. Our thinking processes are often either: (i) hasty, with insufficient investment in deep processing or examination of alternatives; (ii) narrow, with a failure to challenge assumptions or consider other points of view; (iii) fuzzy, or imprecise and prone to conflation; or (iv) sprawling, or disorganized with a failure to conclude. While educationalists have designed programmes to remedy these deficiencies,5 few clinical texts highlight strategies to deal with them.
Kassirer and Kopelman's Learning Clinical Reasoning is one of the few, and it uses a rich compendium of clinical cases to illustrate strategies for better clinical thinking,6 drawing from a vast literature on the structure of medical knowledge and the nature of expertise.7 For example, the potential bias induced by availability and representativeness heuristics (i.e. faulty ‘rules of thumb‘, when saliency and typicality may affect estimates of probability) is well documented,8 as are the effects of the ‘illusion of control’ and ego bias that can lead to overestimation of the probability of good outcomes when we ourselves are making choices9 (for example, surgeons often overestimate the chances of success when they are in charge). A frequent and pervasive cognitive bias affecting everyday practice is ‘confirmation bias‘, where we invest more time in seeking to confirm (rather than refute) hypotheses. The same type of process governs our tendency to associate physical signs with disease processes, when we have not sought out the correct denominator10 (Figure 1). An excellent recent mini-series in the Canadian Medical Association Journal also reminded us of a slightly different set of cognitive biases that must be overcome when interpreting patients’ histories.11 A couple of simple examples are the context dependency of recall and the persistence of beliefs about their illness, often about coincidences, such as the fact their arthritis is made worse by certain weather patterns.
|
Roy Poses has concisely summed up some of the proven strategies that can subvert the most important cognitive biases affecting clinical judgment,12 including: (i) obtaining feedback on the judgment process and outcome of judgment; (ii) overt accountability and justification for decision making; and (iii) the necessity to focus on alternatives. In our guide to critical thinking, we emphasize the need to avoid perceptual biases, by reframing problems, seeking descriptions from multiple references points and the need to search for neglected information, particularly information contrary to the proposition. The latter points are illustrated by problems of premature closure in several of the clinical cases discussed by Kassirer and Kopelman.5
|
Strategies for problem solving and hypothesis testing |
|---|
|
TopIntroductionCognitive concepts and their...Strategies for problem solving...Analysing argumentsReferences |
|---|
The reasoning that we apply at the bedside is usually of three types: (i) probabilistic, when we correctly appreciate the information value of signs and symptoms; (ii) causal, which requires an understanding of anatomical and physiological processes; and (iii) rule-based, which requires patterns to be recognized first, before algorithms can be applied. Although the importance of each varies according to the type of problem to be solved, it is humbling to consider that in the substantial literature on the nature of expertise, it is suggested that what experts are doing is not so much solving problems as remembering previous solutions.13
Accumulating evidence now points to the importance for clinicians of ‘chunking’ knowledge and memory into meaningful parts. The philosopher Susan Haack uses a crossword metaphor to describe how we then elaborate our knowledge to promote problem solving: ‘How reasonable an entry in a crossword is depends on how well it is supported by the clue and any other intersecting entries ... and how much of the crossword has been completed ... How well evidence supports a proposition depends on how much the addition of the proposition improves its explanatory integration’.14 Though the principles of adequacy, parsimony, coherence, robustness, primacy and prediction may guide us when evaluating diagnostic hypotheses,15 we live in a complex non-linear world in which small changes in initial conditions may lead to large and unpredictable effects.16 We need tools to help structure problems at and beyond the bedside, before we even attempt their solutions. Without a systematic approach to structuring the problem, the focus of the question will be too vague or misdirected, or we may permit the problem statement to be assumption- or solution-driven.
Tools have emerged that can help us structure our problems and improve our thinking towards their solution. Miriam Hunink's ProACTIVE framework17 (Figure 2), bears many similarities to predecessors.18 Essentially, to move towards a solution, the problem must be defined in the correct terms, the objectives of any solution stated clearly, the possible alternatives and their consequences listed; in so doing the tradeoffs may become apparent. Finally, the solution to one problem may derive consequences for others and so past decisions have to be integrated with the current problem and possible future decisions. Only then can a solution be properly be evaluated.
|
We must not forget the power of a variety of visual tools and diagramming techniques that can help both structure problems and point toward solutions. For decades, decision analysts have used simple tree diagrams to help structure and solve clinical problems and improve the underlying reasoning processes.19 At an earlier stage, thinking maps20,21 (which, as ‘concept maps‘, have proved effective in medical education)22 and causal flow diagrams can be promoted and enriched with an assortment of software tools. Mapping out the links in a causal chain of the circumstances leading to ‘near misses’ or sentinel events is proving a simple and effective technique to understanding and preventing medical error.23,24 At the same time, a growing number of clinicians are seeing the value of borrowing from recent developments in management science and finding applications for the ‘theory of constraints’ (TOC) in solving bedside and health service problems.25 While many of the tools in TOC involve diagrams, their most important contribution is in helping us distinguish necessary from sufficient factors that determine the outcome of a problem, allowing us to identify our underlying assumptions and the core constraints in the system.26
|
Analysing arguments |
|---|
|
TopIntroductionCognitive concepts and their...Strategies for problem solving...Analysing argumentsReferences |
|---|
For millennia, philosophers have taught their pupils the skills of argument. While doctors may believe in the ascendancy of deductive certainty, with a veneration of canons of scientific induction, modern texts on critical thinking take a more practical view of rationality as exercised in argument.27–30 In simple terms, this boils down to asking the right questions. Figure 3 illustrates the main ones. An awareness of the more common fallacies (Figure 4) in an argument may certainly assist in identifying flaws, although such checklists should not be used slavishly. Depending on the complexity of the argument, it is often more revealing and important to ensure that its structure has been mapped correctly. A pioneer of this type of analysis was Stephen Toulmin who, in The Uses of Argument (1958) sought a new method for laying out the structure of arguments, and saw that real-world argument had an organic structure that consisted not only of logical operators, premises and conclusions, but also of various claims on a probabilistic basis, and of what we take to be more or less warranted beliefs. Since then, while there seems to be an increasing market for a variety of argument mapping tools, researchers have analysed the power of visual techniques for inference. Figure 5, however, uses a much simpler software package called Reason!able (available at [http://www.goreason.com/]) to diagram an argument, the specific example being of biomedical interest to undergraduates, namely the legacy-of-slavery hypotheses to explain the higher prevalence of hypertension in African Americans vs. Caucasians. Essentially, the tool assists the user in exposing all the assumptions, in a sequential process that traces the steps from the argument's conclusion back to the supporting premises, clarifying the underlying structure and making clearer any counterclaims about the premises or problems with the logical structure of the argument. The experience of others in using these techniques is encouraging.31
|
|
|
One might well ask whether any of these approaches are of provable worth. In the recent International Handbook of Research in Medical Education, Norman calls for a moratorium on further efforts to teach and assess critical thinking, arguing largely from the low correlation between performance in critical thinking measures (such as the Watson-Glaser Critical Thinking Assessment)32 and the fact that so much of clinical expertise is context-dependent.33 However, we find modest correlation at best between students’ performance in Epidemiology courses and their eventual success in, say, Endocrinology or Orthopaedics, but the GMC is unlikely to argue for the abandonment of courses in Epidemiology (although some of our colleagues might!). Norman's view also seems to pre-suppose that the majority of day-to-day thinking and decision making occurs at the bedside. A moments thought could produces evidence to the contrary: How should I prioritize my time today ? How should I prioritize the patients on my waiting list ? How can we devise the best strategy for my specialty in this Trust, that best serves the local population? What are the implications for the hospital of expanding the renal unit or introducing mobile coronary care in the community? What would the consequences be of making all sexually transmitted diseases notifiable conditions? The list is endless. It is far better, in our view, to provide our students with a set of practical tools that they might learn to employ in tackling, if not solving, many of the complex problems that modern health professional face.
|
Acknowledgments |
|---|
FK's work developing an undergraduate Clinical Reasoning and Critical Thinking Module was funded by a Queens University Teaching Award. While there are no conflicts of interest to declare, the views expressed are those of the authors and not their employing institutions.
|
Footnotes |
|---|
Address correspondence to Professor F. Kee, Department of Epidemiology and Public Health, Queen's University Belfast. Mulhouse Building, Grosvenor Road, Belfast BT 12 6BJ. e-mail: f.kee{at}qub.ac.uk
|
References |
|---|
|
TopIntroductionCognitive concepts and their...Strategies for problem solving...Analysing argumentsReferences |
|---|
1. EBM: Unmasking the ugly truth. Clinicians for the Restoration of Autonomous Practice Writing Group. Br Med J 2002; 325:1496–98.
2. Tomorrow's Doctors: Recommendations undergraduate medical education. GMC, July 2002.
3. Norman G. Problem solving skills, solving problems and problem based learning. Med Edu 1988; 22:279–86.
4. [http://www.criticalthinking.org/eventsisc/24conf.htm] (accessed 20th April 2004).
5. Perkins D, Grotzer T. Teaching intelligence. American Psychologist 1997; 52:1125–33.[CrossRef]
6. Kassirer JP, Kopelman RI. Learning clinical reasoning. Baltimore, Williams and Wilkins, 1991.
7. Posner M. What is it to be an expert ? In: Chi M, Glaser R, Farr MJ, eds. The nature of expertise. Hillsdale NJ, Lawrence Erlbaum Associates, 1988.
8. Elstein A. Heuristics and biases: selected errors in clinical reasoning. Acad Med 1999; 74:791–4.[Web of Science][Medline]
9. Poses R, McClish D, Bekes C, Scott E, Morley E. Ego bias, reverse ego bias and physician's prognostic. Critical Care Med 1991; 19:1533–9.[Web of Science][Medline]
10. Christensen-Szalanski JJ, Bushyhead JB. Physicians use of probabilistic information in a real clinical setting. J Exp Psychology 1981; 7:928–35.
11. Redelmeier DA, Tu JV, Schull M, Ferris L, Hux J. Problems for clinical judgment 2. Obtaining a reliable past medical history. CMAJ 2001; 164:809–13.
12. Poses R. One size does not fit all: questions to answer before intervening to change physician behaviour. J Qual Improvement 1999; 25:486–94.
13. Regehr G, Norman G. Academic Medicine 1996; 71: 988–1001 .[Web of Science][Medline]
14. Haack S. Manifesto of a passionate moderate. Chicago, University of Chicago, 1998.
15. Richardson WS, et al. User's guide to the medical literature. How to use an article about disease probability for differential diagnosis. JAMA 2000; 281:1214–19.
16. Wilson T, Holt T, Greenhalgh T. Complexity science: Complexity and clinical care. Br Med J 2001; 323:685–8.
17. Hunink M. Presidential address: Proactive decision making. Med Decis Making 2002; 22:182–6.
18. Hammond JS, Keeney RL, Raiffa H. Smart choices: a practical guide to making better decisions. Boston, Harvard Business School Press, 1999.
19. Gigerenzer G, Hoffrage U. How to improve Bayesian reasoning without instruction : frequency formats. Psychological Rev 1995; 102:684–704.[CrossRef]
20. Hyerle D. Visual Tools for constructing knowledge. Alexandria VA, ASCD, 1996.
21. McDermott P, Clarke DN. Mind Maps in Medicine. Edinburgh, Churchill Livingstone, 1998.
22. West D, Pomeroy JR, Park J, Gerstenberger E, Sandoval J. Critical thinking in medical education. A role for concept mapping assessment. JAMA 2000; 284:1105–10.
23. Vincent C, Taylor-Adams S, Stanhope N. Framework for analyzing risk and safety in clinical medicine. Br Med J 1998; 316:1154–7.
24. McNutt R, Abrams R, Hasler R, Rosen R, Brill J, Dimou C, Reiner C, et al. Determining medical error. Three case reports. Eff Clin Practi 2002; 5:23–8.
25. Hunink M. In search of tools to aid logical thinking and communicating about medical decision making. Med Decis Making 2001; 21:267–77.
26. Goldratt EM, Cox J. The Goal: A process of ongoing improvement. Great Barrington MA, North River Press, 1992.
27. Halpern D. Thought and knowledge. An introduction to critical thinking, 3rd edn. New Jersey, Lawrence Erlbaum Associates, 1996.
28. Browne MN, Keekey S. Asking the right questions. A guide to critical thinking, 6th edn. New Jersey, Prentice Hall, 2001.
29. Fisher A. Critical thinking. An introduction. Cambridge University Press, 2001.
30. Thompson A. Reasoning. A practical introduction, 2nd edn. London, Routledge, 2002.
31. Twardy C. Argument maps improve critical thinking. Teaching Philosophy 2004; 27:95–116.
32. Watson GBN, Glaser EM. The Watson-Glaser Critical Thinking Appraisal Manual. San Antonio CA, The Psychological Corporation, 1980.
33. Norman G. Critical Thinking and Critical Appraisal. In: Norman G, Van der Vleuten CPM, Newble DI, eds. International Handbook of Research in Medical Education. Dordrecht, Kluwer Academic Publishers, 2002:277–98.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  Minerva BMJ, August 28, 2004; 329(7464): 520 - 520. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||