Q J Med 2002; 95: 643-645
© 2002 Association of Physicians
Editorial |
Randomized trials in alternative/complementary medicine
School of Biology, University of Newcastle upon Tyne e-mail: bruce.charlton{at}newcastle.ac.uk
The increasing trend for applying randomized trials to alternative medicine is part of a larger social project to ‘integrate’ orthodox and alternative medicine—an aspiration of doubtful merit1—and much hope has been pinned upon randomized trials to evaluate the effectiveness of alternative medicine.2 Thanks to the excessive zeal of the ‘evidence-based medicine’ movement, it is commonly believed that randomization procedures confer automatic validity on an investigation, and consequently, that randomized trials will be able to generate objective evidence concerning therapeutic effectiveness.
This viewpoint misunderstands both science and randomized trials. Randomized trials may serve as a stepping-stone by which individual treatments from alternative medicine are incorporated into the science-based system by which orthodox medicine treats pathologies. But trials alone cannot establish the validity of systems of alternative medicine because their hypotheses lack the precision and formal complexity required to be testable and generalizable.1,3,4
Alternative/complementary/fringe medical systems are frequently discussed, but seldom defined—perhaps because there are so many of them. But the difference between alternative and ‘orthodox’ medicine is clear enough—orthodox medicine is based upon the scientific study of disease processes (or is working towards this goal), while alternative medical systems have non-scientific approaches based on spiritual, mystical, or otherwise intuitive insights.3
In orthodox medicine, explanations of diseases and symptoms are constrained by scientific concepts from the biological sciences, chemistry and physics—knowledge of illness is embedded in this vast and inter-linked background. Of course, individual items of scientific knowledge may turn out to be wrong, and individual interventions of alternative medicine may turn out to be useful. As a consequence, conventional medicine is predicated upon self-correction, which acknowledges the inevitability of error, while alternative medicine aspires to the timeless ‘truth’ of religion, and some systems boast of their ancient lineage and unchanging validity.1,3,4
The fact that systems of alternative medicine eschew scientific approaches means that their validity is untestable since it is only within a framework of science that general theories may be evaluated.
If any of the alternative medical systems were obviously effective, there would be no call for randomized trials to prove the point. However, despite many decades or even centuries of experience, there is no clear-cut instance in which any alternative therapy is unequivocally effective and indicated for any particular disease or symptom. In other words, there are no cures of the otherwise incurable—nobody dragged-back from certain death in the way that has happened many millions of times with antibiotics and steroids. Severed limbs are not reattached to bodies, nor diseased internal organs extracted, nor can reliable anaesthesia be induced (despite the propaganda-based claims for acupuncture4).
Lacking any clear-cut and replicable evidence of therapeutic effectiveness, the only positive indicators comprise a few randomized trials—for example, the famous Lancet report of better-than-placebo effectiveness of a homoeopathic remedy in hay fever.6 However, well-conducted positive trials of homoeopathy have only been reported for conditions which are distinctive in having a highly unpredictable prognosis—conditions such as hay fever, rhinitis, asthma, eczema, back pain, arthritic pain, migraine, chronic fatigue, post-operative ileus, multiple sclerosis, and so on7–9—and the same applies to other less frequently investigated alternative systems. In conditions with such widely variable natural histories, it takes a great deal of research using many methodologies to establish the effectiveness even of orthodox scientific treatments (and their value often remains uncertain despite huge investments of resources).3
The potential clinical usefulness of a trial arises when its results may be generalizable.10 What makes a result potentially generalizable is that it can be interpreted in the context of a recognized theoretical basis of already existing scientific knowledge. A well-controlled therapeutic trial is a type of scientific experiment, and like any other scientific experiment, it may be understood as testing an hypothesis.11 But a hypothesis must have certain formal qualities if it is to be both useful and testable.12 In particular, the hypothesis must be part of a larger and systematic theory. A scientific theory is like a map—a theory tells you where to look and what to expect; enabling observations to act as tests of these predictions.13
So there is no single definitive test of any scientific theory—rather, the truth of a theory is decided on the basis of a range of observations, preferably done by independent investigators (including some who are genuinely but honestly trying to refute the theory), and using a variety of methodologies. A theory becomes established when other scientists incorporate and build upon it in their work.12
Questions of the general form ‘Is X an effective treatment?’ are not true scientific hypotheses, but pseudo-hypotheses with a misleading similarity to the genuine article.10,11 For example, it would not make sense to ask something as vague as whether beta-blockers are an ‘effective treatment’, since—although statistically precise—the question lacks the requisite structure to be testable. A great deal more would need to be specified. For example, ‘effective’ in what dose, for what condition, and with what outcome? Without such knowledge, the benefits of treatment cannot be evaluated.
So hypotheses need to be not just numerically precise, but also sufficiently theoretically complex as to entail numerous observable consequences. Lacking a basis in scientific pathology, such hypotheses cannot be generated by alternative medicine systems. This makes randomized trials or meta-analyses uninterpretable when they examine pseudo-hypotheses such as ‘Is homeopathy a placebo response?’ or ‘Are the clinical effects of homeopathy placebo effects?’.5,6 Such questions simply do not have the formal complexity required of scientific hypotheses—they do not entail numerous observable consequences.
Trials addressing such questions merely end-up measuring outcome variables in comparison groups. It is then left to statistical procedures (and statistical assumptions) to try and make sense of the results. Such attempts invariably degenerate into a morass of numerical logic-chopping that leads nowhere.5,8
Some alternative medical treatments are clearly without significant biological effects—others almost certainly do have effects, for instance some herbal medicines. But merely knowing that some herbal medicines do something is not an adequate basis for their therapeutic use—because all effective treatments have significant risks.14
Before they can be clinically deployed, interventions need to be embedded in a scientific theory and incorporated into the orthodox medical system. Even orthodox treatments of proven effectiveness but lacking an acceptable scientific explanation tend to be regarded as disreputable (e.g. the perennial unpopularity of electroconvulsive therapy), and may become neglected to the point of extinction (for example the rise and fall of lithium in nineteenth century psychiatry).15 Observations become facts only when they are invoked to support a theory, and a convincing theory will usually require only minimal observational confirmation before being accepted.5,16
When a truly effective intervention emerges within an alternative medical system, its benefits need to be re-explained using ‘orthodox’ theories in order that it can be evaluated. A topical example would be the almost-certainly-effective antidepressant herbal medication—St John's Wort. This agent has been slotted into orthodox medicine by treating it as essentially interchangeable with conventional antidepressants such as the SSRIs—in other words, its effectiveness is assumed to be due to its (as yet undetermined) pharmacological action, and not to the fact that it is a ‘herb’.17 However, the current lack of an agreed scientific theory to explain the action of St John's Wort has apparently deterred conventional psychiatrists from using what is probably an important drug. (Ironically, the widely accepted neurochemical explanations of orthodox antidepressant action are scientifically flimsy, and owe their success to marketing needs rather than scientific discoveries.18)
In other words, the effectiveness of an alternative intervention can only formally be established by first incorporating that specific intervention into the scientific explanatory framework of orthodox medicine. But measuring the effect of specific alternative interventions using randomized trials does not help decide on the validity of systems of alternative medicine. The effectiveness of St John's Wort when used like an SSRI does not validate alternative systems of herbal medicine, but to the contrary converts an individual alternative treatment into an orthodox one.
Useful randomized trials come at the end of a long process of therapeutic development, because a randomized trial cannot be designed until it is known what needs to be controlled.19 Knowledge must be accumulated to answer such questions as: what dose of a drug should be given, what are the side-effects, should the trial include both men and women and of what age, how specific does the diagnosis need to be, what severity of disease should be recruited, do other drugs or diseases interfere with the outcome, what outcomes should be measured, and so on. Randomization is in fact one of the least important aspects of a useful randomized trial, in the sense that for a well-controlled experiment, randomization is merely the icing on the cake.11
When randomized trials are used in alternative medicine, the proper process of therapeutic development is turned on its head. Instead of being at the end of therapeutic evaluation, trials are placed at the beginning, and used in isolation. Not only is there no accumulated scientific knowledge with which to design a trial, the very possibility of any scientific context is excluded, and by definition. And performing randomized trials outside of the scientific context is simply an absurdity—as was wittily illustrated by the publication of a ‘positive’ randomized trial of the power of remote retroactive intercessory prayer in the treatment of blood infection.20
When the constraints of randomized trials are properly understood, so too is the irrelevance of ‘positive’ trials in alternative medicine. Performing more such trials is merely evidence of intellectual confusion. Instead of resolving questions of effectiveness, randomized trials simply surrounds alternative medicine with a pseudo-scientific aura that is seldom appropriate or deserved.
Acknowledgments
Many thanks to Neville Goodman for his help in surveying the literature.
References
1. Fitzpatrick M. Put alternative medicine back in its box. Spiked [www.spiked-online.com/Articles/00000006D949.htm]. Accessed 18.7.2002.
2. Ernst E. What's the point of rigorous research on complementary/alternative medicine? JRSM2002; 95:211–13.
3. Charlton BG. Philosophy of medicine: alternative or scientific? JRSM1992; 85:436–8.
4. Skrabanek P. Acupuncture: past, present and future. In: False Premises: False Promises. Glasgow, Tarragon Press, 2000:29–52.
5. Vandenbroucke JP. Homoeopathy trials: going nowhere. Lancet1997; 350:824.[ISI][Medline]
6. Reilly DT, Taylor MA, McSharry C, Aitchison T. Is homoeopathy a placebo response? Controlled trial of homoeopathic potency, with pollen in hayfever as model. Lancet1986; ii:881–6.
7. Linde K, Clausius N, Ramirez G, et al. Are the clinical effects of homoeopathy placebo effects? A meta analysis of placebo-controlled trials. Lancet1997; 350:834–43.[ISI][Medline]
8. Kleijnen J, Knipschild P, Rietter G. Clinical trials of homoeopathy. Br Med J1991; 302:316–23.[ISI][Medline]
9. Taylor MA, Reilly D, Llewellyn-Jones RH, McSharry C, Aitchison TC. Randomised controlled trial of homoeopathy in perennial allergic rhinitis with overview of four trial series. Br Med J2000; 321:471–6 and Letters Br Med J2001; 322:169–71.
10. Charlton BG. The scope and nature of epidemiology. J Clin Epidemiol1996; 49:623–6.[ISI][Medline]
11. Charlton BG. Mega-trials: methodological issues and implications for clinical effectiveness. J Roy Coll Phys London1995; 29:96–100.[ISI][Medline]
12. Hull DL. Science as a Process. Chicago, Chicago University Press, 1988.
13. Ziman JM. Real Science: what it is, and what it means. New York, Cambridge University Press, 2000.
14. Ernst E. Herbal remedies: where is the evidence. Br Med J2000; 321:395–6.
15. Healy D. The Creation of Psychopharmacology. Cambridge MA, Harvard University Press, 2001.
16. Vandenbroucke JP. Medical journals and the shaping of medical knowledge. Lancet1998; 352:2001–6.[ISI][Medline]
17. Stevinson C, Ernst E. Hypericum for depression: an update of the clinical evidence. Eur J Neuropsychopharmacol1999; 9:501–5.[ISI][Medline]
18. Healy D. The Antidepressant Era. Cambridge, MA, Harvard University Press, 1997.
19. Charlton BG. Fundamental deficiencies in the megatrial methodology. Current Controlled Trials Cardiovascular Med2001; 2:2–7.
20. Leibovici L. Effects of remote, intercessory prayer on outcomes in patients with bloodstream infection: randomised controlled trial. Br Med J2001; 323:1450–1 and Letters Br Med J2002; 324:10037.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  E. Ernst The "Improbability" of Complementary and Alternative Medicine Arch Intern Med, April 26, 2004; 164(8): 914 - 915. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||