OUP user menu

Risk prediction in patients presenting with suspected cardiac pain: the GRACE and TIMI risk scores versus clinical evaluation

G. Ramsay, M. Podogrodzka, C. McClure, K.A.A. Fox
DOI: http://dx.doi.org/10.1093/qjmed/hcl133 11-18 First published online: 15 December 2006

Abstract

Background: Identifying which patients presenting with undifferentiated chest pain are at risk of major cardiac events is a major clinical challenge. Clinical evaluation may lack sufficient precision, leading to unnecessary admission or inappropriate discharge. It is uncertain whether risk scores derived from ACS populations apply to unselected patients with chest pain.

Aim: To determine the predictive accuracies of the GRACE risk score, the TIMI risk score and clinical evaluation in unselected patients with suspected cardiac pain.

Design: Prospective observational study.

Methods: We recruited 347 sequential patients with suspected cardiac pain presenting to a large teaching hospital. The main outcome measures were death, non-fatal myocardial infarction and emergency revascularization, in hospital and at 3 months. Receiver operating characteristic (ROC) curves were plotted for TIMI and GRACE risk scores and clinical evaluation.

Results: Overall 54 patients (15.6%) experienced a major cardiac event (16 deaths, seven myocardial infarctions (MIs), one emergency revascularization) or emergency re-admission (n = 30) within 3 months. Both GRACE (p < 0.001) and TIMI scores (p < 0.001) predicted death/MI/revascularization (and the composite including re-admission), but the GRACE score was superior to the TIMI score for predicting major cardiac events (z = 2.05), and both scores were superior to clinical evaluation (ROC areas 0.82, 0.74 and 0.55 respectively). The GRACE score predicted an ACS discharge diagnosis (p < 0.001) and duration of hospital stay (p < 0.001).

Discussion: In unselected patients presenting with suspected cardiac pain, the GRACE risk score is superior to the TIMI risk score in predicting major cardiac events, and both risk scores are superior to using ECG and troponin findings at presentation.

Introduction

Chest pain is the most common reason for emergency ambulance call-outs, and accounts for 27% of emergency medical admissions in the UK (149 000 admissions/year).1 Although the age-adjusted incidence of ST elevation myocardial infarction (MI) is falling, the number of people being hospitalized with suspected acute coronary syndrome (ACS) is increasing.2 Differentiating ACS from suspected cardiac chest pain represents a major clinical diagnostic challenge, especially among those without ST elevation on the electrocardiogram (ECG). ST elevation has high specificity but low sensitivity for infarction, and three-quarters of those with acute coronary symptoms do not have ST elevation on presentation.3 Risk stratification cannot therefore rely simply on the presence of ST elevation, and more accurate risk prediction tools are required.

Are clinical features sufficient for risk stratification? A clinical diagnosis of ‘suspected ACS’ has low diagnostic accuracy when based only on the ECG and clinical symptoms.4,,5 Conversely, pains of atypical distribution may herald acute infarction, and up to a third of those who evolve MI do not have typical chest pains.6 Fewer than half of the patients who are admitted with chest pain have a final diagnosis of ACS.7,,8 Conversely, up to 6% of those discharged from the emergency department have a ‘missed’ myocardial infarction,9 and up to a third of those who evolve MI do not have typical chest pains.

Risk stratification in emergency departments can reduce hospital admissions, and the introduction of troponin assays has aided this process.10,,11 However, the negative predictive value of troponin on arrival is poor, because of the time required for efflux of this marker from cardiomyocytes.12 Binary decisions based upon single tests are not appropriate, as a spectrum of characteristics contributes to risk in ACS.13 Thus, a more integrated approach to risk prediction is required.

The Global Registry of Acute Coronary Events (GRACE) is a large multinational registry encompassing the full spectrum of acute coronary disease.14,,15 The population comprises 68 937 patients with a diagnosis of ACS, and a prognostic model which predicts the risk of death and myocardial infarction has been established (c index 0.84 for death).13,,14 The GRACE risk score has superior predictive accuracy, compared with the TIMI or PURSUIT scores, in an ACS population.16 However, as all three risk scores are derived from ACS patients with a high probability of having an ACS event, it is unknown whether they apply to unselected patients with undifferentiated cardiac chest pain.

We aimed to assess whether the GRACE score can predict adverse events in hospital and at 3 months following discharge, in an unselected group of patients presenting to the emergency department with chest pain thought to be of cardiac origin. We also compared the GRACE score and the TIMI risk score in this unselected population.

Methods

Study population

Patients presenting as an emergency to the Royal Infirmary of Edinburgh with a presumptive diagnosis of acute coronary syndrome or suspected acute ischaemic pain were eligible for this study. Participants were aged >18 years, with a permanent address in the Lothian area, and were alive at the time of admission. Those with clear signs of a non-cardiac cause for the chest pain, for example due to trauma or surgery, were excluded. There were no other exclusions. Inclusion did not require ECG evidence of ischaemia, or biomarker (troponin) elevation.

We therefore asked 351 sequential individuals admitted to the accident and emergency department, the combined assessment area, coronary care unit or direct to a cardiology ward, November 2005–February 2006, to participate; 347 provided written consent and the four who refused were not included further.

Data collection

Before the emergency department presumptive diagnosis was made, all participants were asked to give a concise history to one individual researcher. This included their age, history of presenting complaint, co-morbidities, and risk factors. Information from the history was recorded on a standardized data collection form. The researcher then classified the presentation as either typical of acute coronary syndrome or atypical, based upon the history.

Subsequently, blood test results, and initial investigations were recorded from the in-patient notes. The in hospital progress of each patient was monitored and the discharge diagnosis recorded. No patients were lost to follow-up.

Risk stratification

Full details of the design and methods of the GRACE risk score have been previously published.15 This score is calculated from eight individual variables: the age of the patient, admission systolic blood pressure, heart rate, Killip score, baseline creatinine level, cardiac arrest on admission, ST segment deviation on initial ECG and elevation of cardiac biomarkers. From these variables, estimated risk of death or myocardial infarction can be calculated using the GRACE ACS Risk Model 0.36. The TIMI score has been well described elsewhere.17 From these risk scores, the patients were divided into tertiles of risk (Table 1).

View this table:
Table 1

Distribution of patients according to GRACE and TIMI risk scores

Risk bandGRACE score patients (%)TIMI score patients (%)
Low137 (39.5)73 (21.0)
Medium119 (34.3)146 (42.1)
High91 (26.2)128 (36.9)
  • A TIMI score of 1–2 = low risk, 3–4 = medium risk, >5 = high risk. The corresponding figures for the GRACE score were <15, 16–30, and >30, respectively.

Patient diagnosis and follow-up

Patients were followed up at 3 months after initial admission using the electronic patient database (NHS Lothian HEALTHTRAK), which includes information on admissions, out-patient appointments and discharge letters. In-hospital treatment and discharge diagnoses were recorded. The study outcomes (all-cause mortality, emergency re-vascularization, re-ACS event leading to re-admission to cardiology wards, stroke and emergency re-admission for chest pain) were obtained from this system.

Statistical analyses

A continuous outcome within GRACE or TIMI score categories was expressed as the mean ± 1 SE, and differences were tested using the Kruskal-Wallis one-way analysis of variance. Similarly, a categorical outcome within score categories was expressed as percentages or frequencies, and differences tested using a χ2 analysis. p < 0.05 was deemed significant. The GRACE and TIMI risk scores were compared using ROC curves, analysing areas under the curve. The outcome events death/non-fatal MI/emergency revascularization were plotted in the ROC curves. ROC curves were also plotted for all outcome events, including emergency re-hospitalization. Z scores were calculated for the comparison of ROC curves. All statistical analyses used SPSS v. 13.0 (Windows XP).

Ethical approval

The Lothian University hospitals and The University of Edinburgh College of Medicine ethics committee approved this study.

Results

Baseline characteristics

GRACE risk scores were calculated from presentation characteristics. Of 347 patients included, 137 (39.5%) were in the low-risk stratum, 119 (34.3%) were in the medium-risk and 91 (26.2%) in the high-risk stratum (Table 1). Patients in each successive higher risk stratum were significantly older (p < 0.001), and tended to have a higher frequency of prior diabetes, prior transient ischaemic attack (TIA) or cerebral vascular accident (CVA), and more positive troponin measurements, but a lower rate of current smoking (Table 2). High-risk patients had higher heart rates and Killip scores (p < 0.001), but lower systolic blood pressures (p < 0.03).

View this table:
Table 2

Baseline characteristics of study population

CharacteristicLow (n = 137)Medium (n = 119)High (n = 91)pTotal (n = 347)
Mean age (years)57.0 ± 1.0268.7 ± 1.12573 ± 1.096<0.00165.2 ± 0.725
Males (%)6559.765.90.57362.3
Risk factor
Previous IHD (%)47.452.152.70.6951.3
Treated for hypertension (%)46.745.446.20.5845
Treated for dyslipidaemia (%)45.344.547.30.5744.5
TIA/CVA (%)10.215.117.60.2813.6
Diabetes (%)14.617.624.20.2319.6
Current smoker (%)48.941.240.70.2743.1
Presenting characteristics
Mean SBP (mmHg)*141.7 ± 2.1135.5 ± 2.1128.6 ± 2.90.03136.3 ± 1.4
Mean DBP (mmHg)77 ± 1.374.4 ± 1.471.8 ± 1.60.1374.7 ± 0.8
Heart rate (bpm)*74.6 ± 1.481.2 ± 4.483.5 ± 2.00.0379.2 ± 1.7
Positive 12-h troponin levels (%)*9.546.984.9<0.00142.7
Killip score >1 (%)*7.324.458.3<0.00126.5
  • IHD, ischaemic heart disease; TIA, transient ischaemic attack; CVA, cerebral vascular accident; SBP, systolic blood pressure; DBP, diastolic blood pressure. *p < 0.05.

Discharge diagnoses

Overall, 85/91 (93.4%) high-risk patients, 96/119 (80.7%) medium-risk and 70/137 (51.1%) low-risk patients had a cardiac discharge diagnosis. Figure 1 shows the discharge diagnosis of initial presentation plotted against GRACE score. A total of 140/347 (40.3%) patients had a hospital discharge diagnosis of ACS. Patients in a higher risk group at initial presentation were significantly more likely to have an ACS discharge diagnosis than those in a lower risk group (p < 0.001). An inverse relationship (p < 0.001) was seen for GRACE score and cardiac non-ACS diagnosis (including left ventricular failure, stable angina and atrial fibrillation) (Figure 1). No patients in the high-risk stratum had a discharge diagnosis of a gastrointestinal pathology. Four patients with a high score were diagnosed as having a respiratory pathology, one of whom also had a raised troponin.

Figure 1.

Discharge diagnoses according to category of GRACE risk score (low, medium, high). ACS, acute coronary syndrome; Non- ACS cardio, cardiac diagnosis but not ACS; RESP, respiratory; GI, gastrointestinal; OTHER, other diagnoses.

In-hospital events and duration of hospitalization

Five patients (1.4%) died in hospital (all as a consequence of ACS, and all in the high-risk group) and three evolved an acute MI during hospitalization. The mean duration of admission was 4.2 ± 0.3 days, but this varied from 2.3 ± 0.29 in low-risk patients to 7.3 ± 0.91 in high-risk patients (Figure 2). Patients with higher GRACE scores were significantly more likely to have a longer hospital admission (p < 0.001), compared with those having a lower risk score.

Figure 2.

Mean hospital say (days) according to GRACE risk category on admission.

Outcomes at 3 months

At 3 months, there were 24 major outcome events (7.2%) in the entire group: 16 patients died, seven experienced a non-fatal myocardial infarction and one patient required emergency re-vascularization. Both the GRACE (p < 0.001) and TIMI scores (p < 0.001) were predictive of these major outcome events (death, non-fatal MI, emergency revascularization). Table 3 lists the separate outcomes against each band of the GRACE and TIMI scores. In addition, there were 30 patients with emergency re-hospitalization: 17 due to ischaemic chest pain and 13 others. Overall 54/347 (15.6%) experienced an outcome event or emergency re-hospitalization within 3 months. Outcome events (including emergency re-hospitalization) were not limited to patients with a cardiac discharge diagnosis. These data are included in all outcome events (Figure 3a for GRACE score, Figure 3b for TIMI score).

View this table:
Table 3

Distribution of major outcome events at 3 months according to GRACE and TIMI risk scores

Event …NoneDeathMIRe-vascularization
GRACE
Low136010
Medium115220
High721441
Total3231671
TIMI
Low73000
Medium140420
High1101251
Total3231671
Figure 3.

a GRACE score and b TIMI score for all outcome events at 3 months. This includes other emergency chest pain admissions, and emergency re-admission to a non-cardiac unit.

Comparison of GRACE and TIMI risk scores

Using the same patient group, risk distribution differed significantly for the two scores (p < 0.001). The TIMI score classified fewer patients as low risks (21% vs. 39.5% GRACE) and more as high risk (36.9% vs. 26.2% GRACE).

Clinical predictors of outcome

Table 4 shows outcome events vs. ECG changes and biochemical markers (troponin) at first admission. Five of the 16 deaths and three of the seven MIs occurred in patients with neither ST deviation nor troponin elevation during the initial presentation. Furthermore, ST deviation and troponin elevation both failed to predict cumulative outcome events (death/MI or emergency revascularization) at 3 months (p = 0.6).

View this table:
Table 4

Distribution of major outcome events according to ECG and biomarker findings

Event …NoneDeathMIRe-vascularization
Neither ST deviation nor troponin elevation142530
Troponin elevation only49320
ST deviation only42200
Troponin elevation and ST deviation90621
Total3231671
  • All findings are at initial presentation.

Receiver Operating Characteristic (ROC) analysis

The principal outcomes of the study (death/MI/emergency revascularization) were plotted in ROC curves (specificity versus 1–sensitivity, Figure 4a). The GRACE score had an area under the curve (AUC) of 0.82, the TIMI score 0.74. The AUC for the clinical parameters was 0.55, (the latter not significantly greater than due to chance). The ROC curve for the GRACE score was significantly different to that for the TIMI score (z = 2.05).

Figure 4.

Receiver Operating Characteristic Curves. a Death, MI, emergency revascularization. Areas under the curve: GRACE 0.82, TIMI 0.74, clinical parameters 0.55. The ROC curve for the GRACE score is significantly different from that for the TIMI score (z = 2.05). b All outcome events. Areas under the curve: GRACE 0.73, TIMI 0.69, clinical parameters 0.50. The ROC curve for the GRACE score is significantly different from that for the TIMI score (z = 2.34). ‘Clinical parameters’=ST deviation and/or troponin elevation.

ROC curves for all death, MI or emergency revascularization are shown in Figure 4a, and for all outcome events in Figure 4b. For the latter (more diverse) outcomes, AUCs were 0.73 for the GRACE score, 0.69 for the TIMI score, and 0.50 for clinical parameters. Again, the GRACE score differed significantly from the TIMI score (z = 2.34).

Discussion

In this study, both the GRACE and TIMI risk scores were able to predict adverse outcome events in unselected patients presenting with suspected cardiac chest pain, in the 3 months after their initial presentation. The findings provide support for those published by Pollack and by Soiza and colleagues, which suggested that the TIMI risk score could be applied to unselected patients with suspected ACS.17,,18

Many clinicians currently use the electrocardiogram and initial troponin elevation as the basis for deciding management strategy, and this approach was suggested by previous guidelines in 2002.5 However, such triage systems were designed to be applied to patients with a diagnosis of ACS (i.e. with a high prior probability of acute coronary syndrome), rather than unselected patients with chest pain. In this study, the ROC curve based upon ST deviation and troponins did not differ significantly from that due to play of chance in predicting death, MI or emergency revascularization. Five of the 16 deaths and three of the seven MIs occurred in patients with neither ST deviation nor troponin elevation at presentation (Table 4). These findings are consistent with previous studies, in which the likelihood ratios based on individual clinical characteristics were relatively poor as predictors of ACS in patients with undifferentiated chest pain.4 Our data strongly suggest that in this patient group, the systematic application of a risk score more reliably identifies patients at higher risk for subsequent cardiac events, than do ECG and troponin elevation at presentation. Both troponin elevation and ECG abnormality are included within the TIMI and GRACE risk scores, and hence the findings suggest that additional variables contribute to adverse risk.

Clinical symptoms also do not provide a reliable basis for differentiating the risk of subsequent cardiac events.19,,20 In this study, according to systematic analysis by a single observer (GR), 35% of those with clinically typical ischaemic chest pain had subsequent cardiac events, whereas 26% of those without clinically typical ischaemic chest pain had such events (p = 0.2). Hence, the clinical characteristics of the pain in these heterogeneous patients did not provide a reliable basis for risk prediction.

Overall, as anticipated in an unselected group of patients with chest pain thought to be of cardiac origin, the rate of in-hospital myocardial infarction was low (23/347, 6.6%). Nevertheless, these outcome events and other emergency re-hospitalizations were predominantly clustered in the high risk population (by either GRACE score or TIMI risk score) suggesting that the systematic identification of such patients may allow management to be tailored to such patients. This has the potential to modify future outcomes.5

Comparisons of the ROC curves suggest both the GRACE and TIMI risk scores are significantly superior to risk stratification based on the ECG and troponins alone. Further, the GRACE score was significantly superior to the TIMI risk score. These findings are consistent with the study of Goncalves and colleagues,16 which suggested that the GRACE score had higher accuracy than either the PURSUIT or the TIMI risk scores in an ACS population.

The risk of future events in this population depends in part on the factors that contribute to acute ischaemic risk, and in part upon the underlying risk of the patient (including the impact of age, heart failure, renal dysfunction).5,,13 Thus, the most useful risk score will not only provide information on the future risks of death, but also the risks of MI (related to ischaemic risk). The latter may be potentially amenable to anti-thrombotic and revascularization strategies during the index hospitalization, whereas the former may be ameliorated by secondary prevention measures.

Limitations

Although the size of this study was relatively modest, its unselected study group is more likely to represent real-world clinical practice, compared with the highly selected patients recruited to clinical trials. Indeed, risk scores derived from trial populations (e.g. TIMI risk score) may underestimate the impact of co-morbidity.16,,21 Outcome events in this study were analysed to 3 months (with no loss of follow-up), as previous studies have demonstrated that this period encompasses the major risks of adverse events.23 Nevertheless, such patients are at risk of ongoing events beyond the first 3 months. As in other studies in the early phase of acute coronary syndrome, repeat myocardial infarction may have been under-diagnosed, because of the difficulties in differentiating a subsequent troponin elevation from the index episode. Our study relied upon the clinical differentiation of repeat MI (new ECG changes and new marker elevations), and thus re-infarction may have been under-recognized in the early in-hospital phase. However, after an index infarction, such patients have already identified themselves as being at high risk of subsequent events.

Conclusions

Both the TIMI risk score and the GRACE risk score can be applied to unselected patients with suspected cardiac pain to identify those individuals at higher risk of major cardiac events. The GRACE risk score was superior to the TIMI risk score in this population. Both scoring systems were superior to clinical parameters and to predictions based on ST deviation and troponin elevation at presentation. Systematic application of a validated risk score to unselected patients with suspected cardiac pain may assist in the triage and early management of such patients, and may provide a more reliable basis for early discharge, but this requires validation in larger prospective studies.

References

View Abstract