OUP user menu

Evidence and consequences of spectrum bias in studies of criteria for liver transplant in paracetamol hepatotoxicity

G.K.A. Ding, N.A. Buckley
DOI: http://dx.doi.org/10.1093/qjmed/hcn077 723-729 First published online: 7 July 2008

Abstract

Objective: In severe paracetamol hepatotoxicity, orthotopic liver transplant (OLT) is a standard treatment in patients judged to have a hopeless prognosis. The most commonly used criteria to make this decision are the King's College Criteria (KCC). We aimed to compare the expected survival for patients who meet the KCC and do not receive transplant and those who receive OLT.

Methods: A systematic review of studies of survival in patients who met the KCC according to whether they were transplanted. Data from these studies was extrapolated to compare long-term survival with and without adjustment for Quality of Life.

Results: The survival of patients meeting KCC and undergoing transplant has not been specifically studied. UK data on transplants for acute liver failure indicate 1 and 10 year survival rates of 65 and 44%, respectively. Survival in those without transplant was documented in 15 studies. The average long-term survival rate was 24.9%. Survival was worse in studies originating in the King's unit (13.8 vs. 30.0%). It was apparent that this may be due to spectrum bias occurring in this much larger unit. There was clear evidence that those with the best prognosis were preferentially transplanted at the Kings liver unit, indicating the criteria may perform significantly worse at predicting death without transplant than previously estimated. Even so, for a 20-year-old meeting KCC, the best estimate of life expectancy with transplant (13.5 years) is no better than without (13.4 years). Adjustment for quality of life made OLT clearly a worse option.

Conclusion: Criteria for OLT that have a much higher positive predictive value (for death without transplant) are required. Such studies must be conducted only on those who would be considered suitable for transplant. Non-orthotopic liver transplant may be a preferred option in such circumstances, although much more data on survival after this procedure are required.

Background

Paracetamol induced liver failure is a serious and potentially fatal poisoning which occurs commonly in the United Kingdom. Approximately 100–150 people die as a result of paracetamol poisoning, and an estimated 40 000 individuals take paracetamol overdoses each year.1 In patients who are otherwise expected to die, orthotopic liver transplant (OLT) is recommended. OLT is performed in specialist centers and because this treatment is only clearly justifiable in patients who would otherwise die, criteria for determining such patients have been established. If such criteria were perfect, all patients who would benefit from transplant would be identified while they were well enough to receive this major surgery and those that would otherwise survive would always be excluded.

The implications of a wrong decision to transplant a person who would otherwise survive are substantial. The patient's long term survival, health and quality of life would be severely adversely affected, there would be substantial short- and long-term monetary costs of treatment and there are opportunity costs: the liver that was transplanted would not be available for the treatment of someone who might derive a substantial benefit. Therefore criteria have been developed to aid clinicians in making a timely decision. However, the researchers developing such criteria have focused on improved sensitivity and negative predictive value of such criteria (reducing the numbers who die without being listed for transplant) and have largely ignored these adverse consequences of false positives.

By far the most commonly used criteria today are the King's College Criteria (KCC) described in 1989 by O’Grady et al.2 (Table 1). We aimed to compare the expected survival for patients who meet the KCC and do not receive transplant and those who receive OLT. We aimed to quantify and compare this expected survival in terms of the total life expectancy and quality-adjusted life years (QALY).

View this table:
Table 1

The King's College criteria.(2)

King's College criteria for transplant in paracetamol induced liver disease
Arterial pH < 7·3 after adequate fluid resuscitation
OR (all of)
Grade, III or IV encephalopathy and
Prothrombin time >100 s and
Serum creatinine >300 μmol/l
In a 24-h period

Methods

We conducted a systematic MEDLINE search from January 1989 to January 2007 using keywords (Liver failure OR liver transplantation OR king's college criteria) AND (paracetamol OR acetaminophen) Limit Human, and an Ovid search from January 1989 to January 2007 using (Liver failure MP or Liver transplant MP) and (Paracetamol MP or Acetaminophen MP) limit human, obvious reference to other articles within included studies were also included. Two investigators were involved in vetting the articles. If a conflict arose regarding the inclusion of an article, a third investigator would be used to arbitrate. Studies were included if data could be retrieved on survival rates for patients who met KCC and were not transplanted. Studies were excluded if data could not be separated by whether patients had received transplant (most commonly because patients who underwent transplant were counted as deaths in some studies). The only criterion applied was whether the raw data could be extracted and no further assessment of the quality of the studies was made. Where there were multiple studies with overlapping cohorts, only the largest study was included. From these studies we determined the chance of survival of patients meeting the KCC and not transplanted (Figure 1).

Figure 1.

Flow chart showing article selection process.

However, there were no published studies specifically on survival after OLT for paracetamol overdose. We judged that the UK data on survival after liver transplant would be the most relevant alternative (as paracetamol poisoning and deaths are highest in the UK). This was obtained from the United Kingdom Transplant Support Service Authority, Liver Transplant Audit 1985–95 which had 1-, 3-, 5- and 10-year survival for all liver transplants. These data are comparable to other data sets we might have used.3

The typical age of paracetamol self-poisoning is 20–25 years.4,5 The average life expectancy in the developed world is ∼80 years. Therefore we modelled the outcome of the decision to transplant a 20-year-old on their survival over the next 60 years. Survival rates for transplanted patients beyond 10 years for which data are available was extrapolated by simple arithmetic regression (the same percentage of survivors dieing each year from complications of their transplant after year 10 as died between year 1 and 10—pale lines in Figure 2). Those who were not transplanted were assumed to have normal health. Both groups were assumed to also be affected by other causes of mortality equally using data on age related mortality in the UK from the WHO Global Burden of Disease Project 2001.6

Figure 2.

Estimated survival curves depending on decision to transplant a 20-year-old with KCC for paracetamol induced liver failure. Pooled data show survival of 24.9% (95% CI 20.8–29.4) in patients meeting KCC who are not transplanted. Liver transplant recipients after acute liver failure by comparison have survival at 10 years of 44% (95% CI 38–50). Thus, the odds ratio for 10-year survival might be estimated as favouring transplant recipients (OR 1.77; 95% CI 1.29–2.40). However, the survival advantage becomes increasingly unfavourable with extrapolation, and even more so when using QALY.

For quality of life assessment, patients surviving without a transplant were assumed to have a normal quality of life. So their life expectancy was also compared with QALY survival in those with a liver transplant. For such purposes, transplanted individuals are estimated to have a quality of life of 0.6 compared to a healthy population.7

Results

The results of the search strategy are summarized in Figure 1. Many articles were rejected as the abstracts indicated they could not possibly contain original data on this subject. Another 51 articles were rejected because on review of the original paper the relevant data was not present (e.g. death and transplant combined, other severity scores combined with King's criteria). Of the 36 articles identified in the two searches, 11 were identified in both searches and this left a total of 25 articles containing relevant data.8–33 Two studies from the King's centre had complete overlap with two larger studies from the same unit and were excluded to prevent major duplication of results.8–11 One study from Birmingham had complete overlap with another and was excluded.12,13 One study from the Scottish Liver Transplant Unit had complete temporal overlap with another and was excluded.28,30 Two studies from the Rigshositalet Copenhagen had complete overlap with other studies and were excluded.14–19 One study from the Rigshositalet Copenhagen had 24 months of 55 months (43%) overlap with another and was excluded.18,32 One Study from the Rigshositalet Copenhagen had 1 year of total 23 years overlap with another (4.3%) and was included.16,18 The data from two articles from The Austin Hospital in Melbourne Australia were republished in another later article and were excluded.20–22 One Study from Cedars-Sinai had complete temporal overlap with another and was excluded.26,31 The two other studies from King's had 3 months of the studies of a total 102 month overlap (2.9%) and were included for analysis.8,9 Data was extracted from a total of 15 studies8,9,12,16–19,22–25,27,28,31,33 (Figures 1 and 3).

Figure 3.

Number of people who lived and died after meeting KCC for paracetamol induced liver failure.8,9,12,16–19,22–25,27,28,31,33 Asterisk indicate Study from King's unit.

From these studies there were 386 patients that met KCC but were not transplanted and of these 96 (24.9%) survived (95% Confidence interval 20.8–29.4). The expected survival is shown in Figure 2. The expected survival benefit calculated as area under curve (AUC) for a 20-year-old with the KCC was similar without a transplant (13.4 years) as with a transplant (13.5 years). The expected survival in Quality adjusted life-years in the transplanted group was 8.1 years further reducing the survival benefit in the transplanted group.

Discussion

The KCC are the best known and most widely used prognostic indicator to determine the need for liver transplantation in patients with paracetamol induced fulminant liver failure. For a 20-year-old with the KCC, the best estimate of survival benefit with transplant (13.5 years) is not much better than without (13.4). Adjustment for quality of life made OLT clearly a worse option. Criteria for OLT that have a much higher positive predictive value (for death without transplant) are required if this procedure is to provide any substantial health gains.

It is apparent that reported survival in those not transplanted is much worse in studies originating in the King's unit (13.8 vs. 30.0%).8,9,12,16–19,22–25,27,28,31,33 This may be due to spectrum bias occurring in studies reported from this much larger unit. Spectrum bias refers to the testing of the validity of a diagnostic (or prognostic) test in a population that is different from the one in which it most usefully would be applied to in practice. In this case the problem occurs when the test is applied to patients who are not fit for transplant even if they meet the criteria. The survival rates of these patients are clearly worse than those who are fit enough and this would thus overestimate the performance of the criteria. The criteria in studies from the King's unit were frequently applied to patients who were too unwell to even be considered for transplant. Such patients included over half the patients meeting the criteria in two studies.8,9 Moreover there is evidence that even among those listed for transplant, those with the best prognosis may have been preferentially transplanted at the Kings liver unit. For example, in one study those not listed for transplant but meeting criteria had a survival of 9% whereas those who met criteria and were listed for transplant but were subsequently not transplanted had a survival of 17%.9 The potential implications of this for those patients and the validity of the criteria are substantial. This is also seen in a study evaluating modification to the Kings criteria by addition of lactate measurement. The transplanted group that were not included in these studies had measurements of lactate intermediate between those who died and those who survived without transplantation.23 While an early lactate concentration of <3.5 mmol/l seemed to clearly separate those surviving from those who subsequently died, the transplanted group who were removed from the analysis had a median lactate concentration of 3·72 mmol/l (range 2.17–18.5). This suggests that patients with liver failure intermediate in severity between survivors and nonsurvivors were transplanted.23 In order to assess how useful criteria are validation studies for transplant criteria must be conducted only on those who would be considered suitable and fit for transplant. Any inclusion of those who are too unwell to even be considered for surgery will inflate the apparent predictive value of the criteria.

It is also possible that the King's unit is selectively reporting only statistics that show their criteria performing well. Another study from their unit, not evaluating their criteria but reporting on the optimal treatment of paracetamol induced liver failure, reported a much higher survival of between 23% and 42% of patients who meet transplant criteria and were not transplanted.29 The lack of raw data in this study means we were not able to include these figures in our analysis. It is also possible that the KCC have been applied differently by the other centers. This could be because at the King's unit additional prognostic factors have been applied that are not explicit in the criteria or that other units measure the component tests at different intervals increasing the chances of less severely ill patients (with a better prognosis) meeting the criteria. It is also possible that other important characteristics of patients with paracetamol induced liver failure are different at the other centres or that better treatment is altering the chances of survival in those not transplanted favourably. There is evidence of an improved overall prognosis of patients with paracetamol hepatotoxicity over the last two decades. This is apparent in the two King's studies where there has been a fall from 43.9% (pre 1990) to 22.6% of patients admitted to the unit meeting KCC.8,9 The difference between these two cohorts may reflect the introduction of N-acetylcysteine into routine practice.34

Those that do not die from liver failure will be expected to have a normal life expectancy and quality of life. If it is not possible to confidently predict which patients will succumb while they are still well enough for a transplant, then a partial orthotopic or heterotopic liver transplant may be a preferred option. Although much more data on survival after this procedure are required some recent evidence is favourable.3,35,36 The potential advantage of such techniques is that it may serve to support the patient until there is native regeneration whereupon immunosuppressive drugs may be discontinued and lifelong transplant dependence avoided.36 Other new treatments such as liver support devices may also help keep patients alive long enough for native liver regeneration to occur, or help to make the decision that death without transplant is inevitable.

The morbidity and reduced life expectancy of an orthotopic liver transplant recipient suggest a very high positive predictive value is required to justify this treatment, particularly if the decreased quality of life is taken into account. To even achieve a break-even position for life expectancy in quality adjusted life years for a 20-year-old would require the chance of survival without transplant to be <14–15%. This compares unfavourably with the result from our pooled data which shows survival of 24.9% (95% CI 20.8–29.4) in patients meeting KCC. To justify the expenditure and use of a liver that might otherwise be used to treat other patients would require the chances of survival without transplant to be substantially lower again. For the costs of OTL are substantial, estimated at between £32 000 and £45 000 in just the first year with high ongoing costs in medication and treatment of complications.37

There are a number of limitations in the sources of our data and our analysis. The data on survival rates in those not transplanted is limited to just 15 published articles.8,9,12,16–19,22–25,27,28,31,33 These vary widely in their results and it is unclear which studies best represent the true expected survival rates. However, even the results of those studies originating from the King's unit8–11,23 do not suggest a substantial gain in survival benefit if a young person is transplanted after adjustment for reduced quality of life.

A large proportion of the articles used in this analysis are from selected series (Table 2). The cohorts were a subgroup of studies that used different or added selection criteria other than Kings to identify their subjects. This implies that some patients from that centre that met KCC and who were not transplanted were not included in these studies. However, there was little difference in survival without transplant between pooled selected and unselected series (24.1 vs. 25.4%; Table 2).

View this table:
Table 2

Articles included in the pooled analysis of outcome without transplant after meeting KCC

ReferenceCountry of originStudy typeType of selected seriesSurvivors/patients meeting KCC but not transplanted
O'Grady et al.8UK Kings’ CollegeProspective cohortNot selected series4/23
Bernal W et al.9UK Kings’ CollegeRetrospective analysis of unit resultsNot selected series9/80
Bernal W et al.23UK Kings’ CollegeProspective cohortSelected series Lactate criteria4/20
Ng et al. 28UK Scottish Liver Transplant UnitRetrospective analysis of unit resultsSelected series Phosphate criteria0/48
Anand et al.12UK Queen Elizabeth Hospital BirminghamRetrospective analysis of unit resultsNot selected series8/30
Shakil et al.25USA University of Pittsburgh Medical CentreRetrospective analysis of unit resultsNot selected series4/21
Detry et al.31USA Cedars Sinai Los AngelesProspective cohortSelected series Bioartificial Liver Support.5/5
Larson et al.33USA ALF Study GroupProspective cohortNot selected series15/34
Chung et al.24USA University of Chicago liver transplant programProspective cohortSelected series Phosphate criteria2/6
Gow et al.20Australia Austin and repatriation medical centreRetrospective analysis of unit resultsSelected series Phosphate criteria7/14
Donaldson et al.27Canada Toronto HospitalRetrospective cohortSelected series Transjugular liver biopsy0/5
Schmidt et al.17Denmark Rigshospitalet, CopenhagenControlled TrialSelected series MARS treatment3/6
Schmidt et al.18Denmark Rigshospitalet, CopenhagenRetrospective analysis of unit resultsSelected series Galactose elimination capacity15/52
Schiodt et al.16Denmark Rigshospitalet, CopenhagenProspective cohortSelected series Gc-globulin criteria4/10
Brandsaeter et al.19Multiple countries Denmark, Finland, Norway, SwedenRetrospective analysis of resultsNot selected series16/32

Our extrapolation of survival figures for those transplanted makes the assumption that the mortality rate due to the transplant will be constant. However, we believe the analysis is conservative in this respect, for while the acute rejection rate is likely to decrease over time, long term adverse outcomes of immunosuppression such as chronic rejection, accelerated cardiovascular disease, weight gain, osteoporosis and increased rates of malignancy are much less likely to cause death in the first 10 years but to become much more important over subsequent decades.

It should also be noted that our analysis of comparative survival benefit is sensitive to changes in a number of patient factors. In particular, the older patients are when transplanted the more likely they are to gain survival benefit. Conversely if patients are selected for transplant from within those meeting the criteria on the basis of being fitter for surgery their gain compared to not receiving a transplant is potentially less.

While being far from definitive, we suggest this analysis provides a basis for re-examining the use of OLT to treat paracetamol induced liver failure. This decision is generally based on the KCC, and we have identified problems with the internal validity of the original and validation studies from this unit which have had a marked spectrum bias. More importantly the external validity of these criteria are even more of a concern, with 30.0% of people who meet these criteria in the 12 articles from other centres surviving without transplant.12,16–19,22,24–25,27,28,31,33 While the King's unit may argue they are doing well using the KCC to make this difficult decision, these results would suggest other units should be much more circumspect in applying these criteria on their patients.

Conflict of interest: None declared.

References

View Abstract