OUP user menu

Judging the judges: the role of journal editors

J.G. Ray
DOI: http://dx.doi.org/10.1093/qjmed/95.12.769 769-774 First published online: 1 December 2002

In societies with a free and open judiciary system, individuals are permitted to challenge a judge's verdict, ability to remain impartial, and conduct. In the first situation, a higher court of appeal typically handles the matter.1 In the second, the judge is disqualified from overseeing a case if an objective observer raises reasonable questions about the judge's impartiality.2 In the third situation, a governing body, such as a judicial council, hears the complaint. To minimize the likelihood of such events, judges are appointed based on their legal knowledge, prior experience, and a historical demonstration of impartiality and fairness.

Although most medical journals and their editors conduct themselves fairly, there are some differences between their practice and that of a judiciary body. In this article, I pose four questions to medical journal editors concerning their impartiality and training as researchers and editors, and the options available to those who disagree with a decision rendered by an editor or editorial board.

Are you free? Editorial freedom versus economic and political pressure. Editors must be free to make decisions about journal policy and the publication of manuscripts without coercion.3 This is the basis of editorial objectivity, and the foundation on which the advancement of both human well‐being and science rest.4 Those who choose (or are chosen) to become medical journal editors tend to do so because they believe in and wish to uphold this fundamental principle.3 However, it has become apparent that editorial decisions are not always independent or unfettered. In fact, external and internal pressures partially influence an editor's selection and acceptance of manuscripts.

Editors have led themselves to believe that they publish works that address the needs of a principal ‘target’ audience. However, in a study comparing the relative importance of topics selected for publication by JAMA, readers agreed with only three of the top ten subjects selected by the journal's editorial board members.5 A similar discordance was found among peer reviewers, experts and the editors of the Annals of Internal Medicine.6 Thus, if editors do not consistently meet the needs of their readers, on what basis do they make decisions about what they publish?

The influence of pharmaceutical and diagnostic test firms on medical research7 and physician practice patterns8 has received much attention. However, there has been little focus on how the private sector bears on medical journal editors and publishers. In a 1995 survey of 221 North American senior medical journal editors, only 57% believed that journals were responsible for ensuring the honesty of pharmaceutical advertisements, while 40% were in favour of subjecting advertisements to rigorous peer review.9 In response to a commentary questioning the ability of editors to judge the appropriateness of drug advertisements, one editor wrote: ‘Given our heavy reliance on openness in the editorial process and an already considerable but appropriate restriction on the claims that are made in drug ads, the fundamental problem with [this] suggestion is that they must surely stem from a belief that we physicians are far too gullible to understand that an ad is an ad, and a belief that editors don't know that they are judged by what they publish.’10

What is now known is that physicians, under the influence of pharmaceutical advertising and promotions, are much more impressionable than was originally believed.11 A systematic review, recently published in JAMA, highlighted the fact that pharmaceutical advertising and promotions influence physician prescribing practices,12 even among those still in training.13 Frequent and intense drug promotion is correlated with increased prescribing volumes, and more expensive and less appropriate prescribing. Drug advertisements often omit negative points about a drug and offer ‘red herring’ surrogate endpoints while using images (including sexual imagery) to appeal to personal desires.11 Moreover, referenced articles in pharmaceutical advertisements are often of poor methodological quality.14 A study of five major Russian medical journals, comprising 207 distinct advertisements, found that only 40% mentioned the generic name of the drug, 45% mentioned the drug's indication, 11% mentioned safety warnings and contraindications, and 2% provided references. These phenomena arose despite the presence of Russian law prohibiting the omission of essential drug information in its advertisements.15 If journal editors once believed that pharmaceutical advertisements were ‘benign’ and their readers impervious to the negative effects of such ads, one must question whether editors themselves have been influenced by pharmaceutical advertising and sponsorship.

Some medical journals benefit from the publication of subsidized symposia under the name of the parent journal. In a study published in 1992, 42% of 625 symposia published within medical journals had a single pharmaceutical sponsor.16 Published pharmaceutical‐sponsored symposia were more likely to use misleading titles and refer to drugs by their brand name, and less likely to be peer‐reviewed with the same degree of rigor as other journal articles.16

Arguments may be put forward that the above examples are rare, or arose in another era, or in countries with less rigorous editorial policies. Nonetheless, they reveal weaknesses in the policy of objectivity claimed by medical journal editors. As recently as 1997, most editors had not yet required their authors to disclose any potential financial conflict of interest. Only 16% of 1396 highly ranked scientific and biomedical journals had a conflict‐of‐interest policy in effect, and less than 1% of articles disclosed any personal financial conflict of interest on the part of their authors.17 In response, several prominent medical editors have taken a more active stance on this issue. Even so, the private interests of the corporate sponsor are satisfied by editorial policy: ‘[A] sponsor should have the right to review a manuscript for a defined period (e.g. 30 to 60 days) before publication to allow for the filing of additional patent protection, if required … but the sponsor must impose no impediment, direct or indirect, on the publication of the study's full results, including data perceived to be detrimental to the product.’18

Why might journal policy permit a private, arms‐length pharmaceutical sponsor to be privy to unpublished data before it can be shared with other clinicians and researchers?19 Why should editors appease the economic interests of corporate sponsors and effectively muzzle scientists, even in a time‐limited fashion?20 Certainly, there is no editor‐industry conspiracy, and researchers themselves delay publishing data to allow for patent application21 but it is clear that medical journal editors are under greater pressure than they are willing to admit.

Another challenge facing medical journal editors is the need to publish studies that improve their journal's profile, vis‐à‐vis its impact factor22,,23 and media exposure.24 This, in turn, augments the sale of journal subscriptions and article reprints.25 Because medical publishing is a competitive business, an editor must consider not only the clarity and scientific validity of a manuscript, but its novelty and timeliness as well.24,,26 One former editor of JAMA labelled this influential factor as the ‘sexiness' of the manuscript.25 For a popular journal, the sale of reprints and syndicated research can turn big revenues,25 particularly in the context of product promotion, as evidenced by an advertisement on the British Medical Journal's website: ‘Reprints are invaluable for direct marketing, exhibitions/seminars and sales support campaigns and for mailing new product information to doctors [and for distribution] to conference delegates and visitors at exhibitions.’27

Beyond their susceptibility to advertisers' and their own economic interests, editors may also face tremendous pressures from the journal's governing body. We have recently witnessed the firing of the chief editors of both JAMA28 and the New England Journal of Medicine.29 Both dismissals arose out of strong philosophical differences between the American Medical Association29 and the Massachusetts Medical Society28 and their respective editors. Even if editors respond to such divergence by claiming freedom and independence,28,,29 there is no guarantee of an effective change in this relationship: the editor of JAMA made this point twice prior to his own dismissal.3,,30

Who made you the boss, and why? Training and appointment of editors. The position of editor‐in‐chief is held in high regard. However, senior editorial appointments often occur independently of a formal, regulated process. For example, in a survey of the editors‐in‐chief of 191 peer‐reviewed clinical journals, 32.5% stated that they were elected or selected by a scientific society, editorial board or research committee, while 21% were nominated by the previous editor.31 Moreover, formal editorial training has been quite limited. In the same survey, 45% of senior editors had no formal training in editing, and 35% had not previously served on the journal's editorial team. When questioned, 69% of editors believed that some form of editorial training would be helpful; surprisingly, 26% did not.31

It has been suggested that medical journal editors should have clinical or research experience in, and a substantial knowledge of, the topics within the scope of their journal.32 However, after a detailed search of the internet, using both PubMed Medline and Google search engines, suggested requirements for training of journal editors as researchers or in the critical appraisal of medical information could not be found. Few individuals in society (perhaps with the exception of politicians) are placed in a position of power and authority without some evidence that they have ‘worked from the ground up’. Those trusted with the selection and dissemination of new and potentially valuable research ideas must be treated no differently.

Does the boss make mistakes? Avoiding errors and editorial malpractice. Applied statistics remains the principal method for analysing medical research data. Because editors often do not understand complex biostatistical methods, most rely on the expertise of statisticians.33 However, in a survey of 114 biomedical journal editors, only a third maintained a policy guaranteeing statistical review for all accepted manuscripts, while 31% of small and 58% middle‐sized journals maintained a statistical consultant on staff.34 Even among journals that do consult with a statistician, the application35 and reporting36 of commonly used statistical methods, even at the most basic levels, remains poor.36,,37 For example, in a random sample of 27 randomized clinical trials published within five major medical journals, 81% explicitly reported a single primary outcome, and only 41% provided confidence intervals for the point estimates of the efficacy of the interventions. The clinical importance of the findings was discussed in 20 studies (74%), of which five (25%) provided justification for their clinical interpretation of the results.36 Simply stated, editors cannot guarantee that basic statistical concepts have been properly evaluated and presented in the manuscripts that they publish.38

To assess the appropriateness and scientific quality of a manuscript, editors rely on peer reviewers9,,39 defined by one editor as ‘professionals whose standing is equal to or above that of the authors’.40 However, the use of expert peer reviewers introduces several problems. First, there is often poor agreement between reviewers about the acceptability of research manuscripts, which was demonstrated by kappa scores ranging from 0.08 to 0.28 among peer reviewers of two neuroscience journals.41 In the same study, comparably low agreement scores were observed for assigning priority to manuscript publication.41 A similar lack of agreement between reviewers was observed in a study of research manuscripts submitted to the Annals of Internal Medicine.6 Country of origin and language of composition also appear to influence a peer reviewer's judgment of a manuscript. For example, in a study of original manuscript submissions to Gastroenterology, US reviewers were significantly more likely than non‐US reviewers to recommend acceptance of papers submitted by US authors.42 Non‐US reviewers also ranked US papers slightly higher than non‐US papers, albeit to a lesser degree than their US counterparts.42 In another study, a non‐authentic but realistic short manuscript was randomly sent to more than 150 Scandinavian referees; one version was written in English, the other in the referee's national language.43 The reviewers gave a significantly higher acceptance rating to the English‐language version.43 Finally, the majority of editors do not believe in querying their reviewers about any potential financial conflict of interest.9

If medical editors depend on a system of peer review that is both imprecise and susceptible to considerable variation and bias,44,,45 editorial decisions may be weakened. If a novel study is submitted to a journal, representing serendipitous but important findings,46 or the ‘controversial or non‐conformist views of sceptics and dissident minorities, ‘will the peer review system be fair, or will ’highly innovative science be stifled by the conflict of interest and reviewer shortcomings underlying the review system?’.47

Can there be another boss too? Judging the judges. Abuse of power and misconduct by medical editors is probably quite uncommon.48 But beyond being accountable to themselves, and, in the case of larger medical journals, editorial board members, editors' decisions cannot be challenged or reversed. Although the reader might be deemed the ultimate judge, they cannot judge that which has not been published.

To resolve disputes between a medical journal editor and a researcher or reader, an appeal process, such as that proposed by Altman in 1994,48 is required. This should comprise an external appeal mechanism (when a dispute cannot be resolved from within), and an international council that would develop a code of conduct for editors with a corresponding taxonomy of inappropriate editorial conduct.48 Surveyed nearly 5 years after this proposal was made, most journal editors agreed with the formation of such a system,49 but little has materialized since then.

Creating a separate ombudsperson for every journal, as the Lancet50 does, is a novel idea. The Lancet's model, initiated in 1996, allows direct contact with the ombudsperson and bypasses the editors entirely. This process, which handled 20 complaints during its first 18 months, has been successful from the perspective of the Lancet's editor.50 For smaller journals, which form the majority of medical journals, individual ombudspersons might be impractical. To address this, the publisher could appoint a non‐partisan individual who covers all the journals it represents. Alternatively, the ombudsperson might be appointed by a group of journals within a certain geographical region or area of speciality.

To conclude, this article was not written to denounce medical journal editors or suggest that they be replaced by another body. Journal editors should remain at the helm of medical journal science, especially since the editors of prominent medical journals have raised a number of the shortcomings described herein. Indeed, the World Association of Medical Editors (WAME) was founded in 1995 for this purpose.32 Nonetheless, because such self‐reflection remains in its infancy and the position of medical journal editor is not without bias, arbiters who are independent of the journal editorial board and the publisher are needed. To generate better feedback to editors and researchers alike, we also require candour about the qualitative aspects of selecting and editing a medical manuscript.51

Peer reviewers, the bedrock of medical journal objectivity, require more training and experience. One simple solution might be for editors to provide them with a short review of best practices along with the checklist of core elements to consider.52,,53 Although peer reviewers are in short supply, it is important that editors, using validated rating instruments,3,,54 only retain those who can do a proper job. Similarly, medical journal editors need more formal training, including a solid grounding in the principles of critical appraisal, practical biostatistics, and, perhaps, the science of arbitration. Formal editorial training55 and a demonstration of previous competence in research should be prerequisites for the appointment of a senior journal editor. Such demands may be high, but it is in their hands that society places its most highly regarded vehicle for advancing health science.56

In setting the stage for future obstacles yet to face medical editors and researchers alike, we may draw wisdom from those intimately tied to both: ‘we need less research, better research, and research done for the right reasons’.57 If we refuse to engage in such a debate, we may forfeit our freedom and independence. ‘Years ago society accorded science the privilege of governing itself. If it finds that its trust continues to be betrayed, the politicians will step in by default.’58


Ms Marian Vermeulen, MHSc, both provided helpful comments and edited this manuscript. Dr Ray was previously an editorial fellow at the Annals of Internal Medicine. He is currently an editorial board member of the journals Current Controlled Trials in Cardiovascular Medicine, and Evidence‐Based Obstetrics and Gynecology. He is also an honorary member of the World Association of Medical Editors. As an active medical researcher, he has had many manuscripts rejected by various medical journals.


View Abstract