Limited information has been published regarding how specific processes for event adjudication can affect event rates in trials. We reviewed nonfatal myocardial infarctions (MIs) reported by site investigators in the international Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin (Eptifibatide) Therapy (PURSUIT) trial and those adjudicated by a central clinical events committee (CEC) to determine the reasons for differences in event rates.
The PURSUIT trial randomised 10,948 patients with acute coronary syndromes to receive eptifibatide or placebo. The primary end-point was death or post-enrolment MI at 30 days as assessed by the CEC; this end-point was also constructed using site-reported events. The CEC identified suspected MIs by systematic review of clinical, cardiac enzyme, and electrocardiographic data.
The CEC identified 5005 (46%) suspected events, of which 1415 (28%) were adjudicated as MI. The site investigator and CEC assessments of whether a MI had occurred disagreed in 983 (20%) of the 5005 patients with suspected MI, mostly reflecting site misclassification of post-enrolment MIs (as enrolment MIs) or underreported periprocedural MIs. Patients for whom the CEC and site investigator agreed that no end-point MI had occurred had the lowest mortality at 30 days and between 30 days and 6 months, and those with agreement that a MI had occurred had the highest mortality.
CEC adjudication provides a standard, systematic, independent, and unbiased assessment of end-points, particularly for trials that span geographic regions and clinical practice settings. Understanding the review process and reasons for disagreement between CEC and site investigator assessments of MI is important to design future trials and interpret event rates between trials.
Keywords:acute coronary syndromes; adjudication; clinical events committee; end-points; myocardial infarction
MI is a potentially catastrophic event in patients presenting with acute coronary syndromes. In recent trials of antiplatelet and antithrombotic therapies for such patients, prevention of MI was the primary treatment effect assessed [1,2,3,4]. Although MI has been considered a 'hard' end-point, determination of MI end-points in clinical trials can be difficult, just as in clinical practice. Because of conflicting clinical, laboratory, and electrocardiography (ECG) data, physicians often disagree whether a patient has suffered a MI. The importance of low-level enzyme elevations also has been controversial, particularly in asymptomatic patients and in those undergoing percutaneous coronary intervention (PCI) . Although such enzyme elevations are defined as MIs in many trial protocols, physicians in clinical practice do not consistently consider them as such. The MI rates reported by site investigators in recent cardiovascular trials have differed from rates adjudicated by a CEC [6,7,8,9]. The reasons for these differences are unclear.
CECs are widely accepted to adjudicate suspected endpoint events in trials, but only limited information has been published regarding end-point adjudication [10,11,12,13,14,15]. In the PURSUIT trial , a central, independent CEC systematically identified and adjudicated all suspected MIs that occurred up to 30 days after enrolment. The rationale for CEC adjudication was the need for a systematic, unbiased, independent, and standard assessment of MIs in a large international trial. In a previous analysis of the CEC process in the PURSUIT trial, we found that the CEC and site investigator assessments of end-point MI disagreed in ~10% of patients . We retrospectively reviewed these cases, to identify reasons for them and to understand their potential effect on the trial results.
Materials and methods
The PURSUIT trial
The PURSUIT trial examined the role of eptifibatide, a platelet glycoprotein IIb/IIIa antagonist, in patients presenting with acute coronary syndromes without persistent ST-segment elevation. The trial enrolled 10,948 patients in 726 hospitals in 27 countries in North America, Latin America, Western Europe, and Eastern Europe, using previously reported inclusion and exclusion criteria and treatment regimens . The primary end-point was a composite of death or post-enrolment MI (or reinfarction if patients had a MI at enrolment) by 30 days as adjudicated by a CEC. The composite end-point was also constructed using the site investigator determination of MI.
The protocol defined MI as an end-point event based on clinical, ECG, and laboratory criteria (see Appendix); MIs that occurred before or at enrolment were not included in the primary end-point. The site investigators and the CEC used the same MI criteria. These criteria were presented in the protocol, at investigator meetings, and in trial materials and newsletters.
Data were captured on standard case report forms. Information collected for all patients included cardiac enzymes, ECGs (at the time of the qualifying episode, at enrolment, at 24 hours, at initial discharge, and at 30 days), revascu-larisation procedure reports, details of ischemic episodes, clinical complications, medications, and readmission records. All enzyme values for each patient were to be reported, and study monitors verified them against source documents. For patients with suspected MI, site investigators were to submit supporting documents and to include discharge summaries and additional ECGs during the suspected event. An independent, blinded, ECG Core Laboratory read the ECGs and identified suspected MIs, defined as new Q waves ≥ 0.04 s in two contiguous leads.
Clinical events classification process
The structure of the CEC and the event adjudication process have been reported in detail elsewhere . Computer algorithms systematically identified key clinical, enzymatic, and ECG data from the database that could indicate the occurrence of a MI. Each case of suspected MI was reviewed independently by two physicians blinded to treatment. If the physicians agreed that a MI had or had not occurred, the case was classified as resolved. A committee of faculty cardiologists reviewed patients about which the CEC physicians disagreed, for adjudication by consensus.
Disagreements between the site investigators and CEC
A faculty cardiologist (KWM, RAH, BSC) re-reviewed patients with disagreement between the site investigator assessment and the CEC adjudication of end-point MI, to determine reasons for the disagreement. This review occurred after the main trial results were presented, but the reviewers were blinded to patient treatment assignment. Patients were categorised during the re-review in clinically meaningful groups: MI at enrolment (versus post-enrolment MI), MI related to the revascularisation procedure, MI related to clinically evident ischemia, asymptomatic cardiac enzyme elevation post-enrolment, and clinically significant cardiac event resulting in death (versus sudden death without evidence of MI).
Despite rigorous criteria to define MI, some patients required a subjective assessment by CEC physicians if cardiac enzyme, ECG, and clinical information conflicted. For patients with MI identified by the CEC but not by the site investigator (excluding events after PCI or bypass surgery), we therefore assigned a level of clinical certainty. A high clinical certainty was assigned when all clinicians agreed that a MI had occurred; a low clinical certainty was assigned when only some clinicians would agree that a MI had occurred.
Data regarding the number of patients with suspected events and those with disagreements between the site investigators and the CEC were obtained from the entire PURSUIT study population, so the data included patients assigned high-dose eptifibatide, low-dose eptifibatide, or placebo. Comparisons by treatment assignments included only patients assigned to high-dose eptifibatide or placebo to maintain consistency with the primary efficacy analysis in the PURSUIT trial . P values were calculated using the Χ2 test.
Incidence of disagreement
Overall, 5005 (46%) patients with possible or suspected MI were identified and adjudicated by the CEC. The CEC identified more patients with end-point events than did the site investigators (13.6% versus 7.7% for placebo, 12.6% versus 6.2% for eptifibatide). The CEC and site investigator assessments of MI disagreed for 9% of all patients enrolled in the PURSUIT trial. The CEC identified MIs in 1415 of the 5005 patients with suspected MI. The site investigator and the CEC assessments disagreed in 983 (20%) of the 5005 patients. Of these 983 patients with disagreements, 816 patients had a MI identified by the CEC but not the site investigator, and 167 patients had a MI identified by site investigator but not the CEC. The proportion of patients with disagreement was similar among regions.
Reasons for disagreement
Most often, when the site investigator had identified a MI and the CEC had not, the investigator had misclassified a MI at enrolment as an end-point MI, although by protocol these were not end-point events (Table 1). Recurrent ischemic events without elevated cardiac enzymes or ECG evidence of MI were also incorrectly identified by site investigators as MIs (Table 1). In cases where the CEC had identified a MI and the site investigators had not, one-third of those identified were MIs defined by enzyme elevations without clinical or ECG evidence of ischemia or infarction (Table 2). Enzyme elevations after bypass surgery accounted for 25% of these cases. Patients with clinically evident ischemia reported by the site investigator and associated with cardiac enzyme elevations were not reported as MIs by investigators in 27% of the cases of disagreement (Table 2). Few infarctions based on new Q waves without clinical evidence of reinfarction were identified by the CEC and not by site investigators.
Table 1. Myocardial infarctions identified by site investigators but not confirmed by the clinical events committee
Table 2. Myocardial infarctions identified by the clinical events committee but not the site investigator
Table 3 presents the cardiac enzyme elevation for MIs that the CEC identified and the site investigators did not. These data exclude MIs associated with PCI or bypass surgery; by definition, those types of MI required enzyme elevations greater than three or five times the upper limit of normal (ULN), respectively. The ratio of creatine kinase-myocardial band (CK-MB) to its ULN was highest in North America (median [25th, 75th] 1.6 [1.2, 2.9]). Enzymes were similarly elevated between patients with MIs defined only by enzyme elevations (without symptoms or ECG changes) and patients with elevations associated with ECG or clinical evidence of ischemia.
Table 3. Enzyme elevations in patients with myocardial infarction identified by the clinical events committee but not the site investigator*
Table 4 presents the level of clinical certainty for MIs not associated with PCI or bypass surgery, as identified by the CEC but not the site investigators. Overall, 98 (18%) of these patients were assigned a low level of clinical certainty. The proportion of patients assigned a low level of clinical certainty varied among the regions, with the lowest in North America.
Table 4. Level of clinical certainty* for myocardial infarction identified by the clinical events committee but not the site investigator
Effect of disagreement on outcomes
The 30-day CEC rates of MI and death or MI overall are presented in Table 5 by geographic region. The event rates are also shown for the analyses in which the 98 patients assigned a low level of certainty were reclassified as no infarction.
Table 5. Clinical events committee event rates (%) at 30 days
The lowest mortality rates were in patients for whom the CEC and site investigators agreed that no MI had occurred by 30 days (Table 6). The highest rates were in patients for whom there was agreement that a MI had occurred. Mortality was high in the group with MI identified by the site investigators but not the CEC; however, many of these patients had suspected sudden cardiac death as the event identified as a MI by the investigator (Table 1). The absolute increase in mortality between 30 days and 6 months was greater in patients with CEC-determined MIs not identified by the site investigators than in patients with MI identified by the site investigators but not the CEC.
Table 6. Mortality (30 days and 6 months) for cases of disagreement
This analysis of the adjudication process of nonfatal MI by a central CEC in the PURSUIT trial has five key findings. First, post-enrolment MI rates were higher than those reported in prior trials of acute coronary syndromes. Second, the site investigator and CEC assessments of end-point MI disagreed for nearly 10% of all patients enrolled. Third, the site investigators underreported protocol-defined end-point MIs. Fourth, the observed treatment effect was smaller using the CEC-adjudicated MI rates versus site investigator-identified event rates. Finally, in a retrospective analysis that excluded patients with post-enrolment MI who had conflicting clinical, ECG, and cardiac enzyme data (although they met prespecified endpoint criteria), the treatment effect was larger than that seen when such patients were included. These findings, in aggregate, suggest the use of a CEC is important for systematic ascertainment of nonfatal end-points. The definition of MI and its application in CEC event adjudication in the PURSUIT trial may, however, have been too inclusive of MIs defined by low-level enzyme elevations, which either represented 'noise' or clinically unimportant events.
The MI rates adjudicated by the CEC in the PURSUIT trial were higher than those reported in trials of similar patient populations [6,17,18,19]. The reasons for these higher rates have been detailed previously , and include the review of nearly 50% of patients by physicians to identify suspected events, more liberal MI criteria, and rigorous measurement of cardiac enzymes in all patients. Studies that have used only investigator-reported events probably underestimate the true MI rate.
Lack of concordance between site investigator and CEC event rates
Site investigators underreported MIs. A similar lack of concordance between events adjudicated by a CEC and those identified by clinical investigators has been observed in trials in which a similar CEC group adjudicated MIs [1,8] and in other trials [7,9].
The MI definitions in the PURSUIT trial were formulated by the International Steering Committee based on experience and clinical expertise. Because of the broad geographic enrolment planned for the PURSUIT trial, definitions were designed to be applicable in an array of clinical practice situations. The definitions were detailed in the study protocol, in study newsletters, and in study materials so the CEC and site investigators had the same set of criteria to classify MIs.
The reasons for disagreements in MI assessment between the site investigators and the CEC in the PURSUIT trial (Tables 1 and 2) are similar to those seen in the Global Use of Strategies to Open Occluded Arteries in Acute Coronary Syndromes (GUSTO-IIb) and the Integrilin to Minimize Platelet Aggregation and Coronary Thrombosis (IMPACT-II) trials (unpublished personal data). Many of the disagreements reflect physician reluctance to diagnose MI in patients they are treating, particularly when the definition of MI includes events with low-level cardiac enzyme elevations (often called 'enzyme leaks' in clinical practice). This reluctance by physicians to diagnose MI is also apparent for patients undergoing PCI, in which the clinical significance of postprocedural enzyme elevations is controversial, even though such elevations correlate with worse outcome .
The disagreements also may partly reflect the definitions (see Appendix) of MI themselves. These definitions were designed to be applied to a broad set of clinical scenarios, including events after PCI or bypass surgery, and to events occurring early after enrolment, which needed to be differentiated from pre-enrolment or enrolment MIs. Site investigators may have had difficulty applying these criteria, particularly with conflicting cardiac enzyme, ECG, and clinical information. In addition, the enzyme criteria in the PURSUIT trial required only one cardiac enzyme value above normal to provide supportive evidence of MI. Substantial clinical uncertainty exists regarding the need for more than one elevated enzyme value and whether the CK-MB criteria should specify elevations greater than 1 × ULN or 2 × ULN.
The strategy used by a CEC to adjudicate MIs can dramatically influence event rates and the proportion of events with disagreements between site investigators and a CEC. Some trials have confirmed only events reported by the investigators [9,17,18,19,20,21], while other trials have adjudicated all suspected events identified by systematic screening of patient data [1,2,3,7,8,22]. When only events reported by investigators are reviewed by a CEC, the reported event rates will be identical to or lower than the site investigator rates. When events are identified independently by the CEC, the CEC event rates may be higher, lower, or the same as the site investigator-reported rates.
Difference in treatment effect
The absolute difference in the MI rates was 1.6% (6.2% for eptifibatide versus 7.8% for placebo) as assessed by the site investigators, and 1.0% (12.6% versus 13.6%) as adjudicated by the CEC. The higher event rates in both treatment arms using the CEC data, despite the similar absolute difference, reduced the relative treatment effect (7.4% versus 20.5% reduction) as expected. A similar decrease in relative treatment effect has been noted in some trials [1,8] but not others [17,18,19].
The MIs determined by cardiac enzyme elevations without clinical symptoms or ECG changes accounted for 33% of the disagreements in which the CEC identified a MI and the site investigators did not. The median CK-MB elevation in these events was 1.6 × ULN. About 50% of these events were thus defined by CK-MB values between 1 × ULN and 1.5 × ULN, with normal median CK values (0.9 × ULN) (Table 3).
A retrospective but blinded review of MIs identified by the CEC but not the site investigator found that 18% (98/540) of these patients were assigned a low level of clinical certainty. This was because, although the CK or CK-MB elevations met the end-point criteria, the cardiac enzyme data were considered inconsistent or unreliable, or were associated with conflicting clinical and ECG data.
We noted regional differences in the proportion of patients with low clinical certainty. The highest proportions were in Eastern Europe and Latin America, where the observed treatment effect using the CEC definition was negligible. Furthermore, the magnitude of the enzyme elevations (Table 3) parallels discrepancies in the assigned level of certainty. The highest enzyme elevations were observed in North America (median CK-MB elevation, 2.9 × ULN; median CK elevation, 1.6 × ULN), where the treatment effect was mostpronounced. CK-MB elevations in other regions were less striking (median CK-MB elevation, 1.4 × ULN–2.0 × ULN). The regional differences in treatment effect are, however, complex and include differences in patient demographics, the use of cardiac procedures, medications and revascularisation, and the reliability of laboratory data . These findings support the hypothesis that including MI end-points defined by low-level cardiac enzyme elevations or events associated with conflicting clinical and ECG data may dilute the actual treatment effect.
Predictive value of CEC-identified events
The 30-day treatment effect was reduced using CEC-adjudicated end-points versus site investigator assessments, but patients with events adjudicated by the CEC (but not identified by the site investigators) had greater mortality between 30 days and 6 months than did patients with MI reported only by the site investigators (Table 6). In addition, MIs identified by a similar CEC process have been associated with worse long-term outcomes at 3-year follow-up . These data suggest that events identified by the CEC alone are of prognostic importance.
CEC adjudication of suspected nonfatal MI end-point events is important to provide independent, unbiased, standard, systematic assessment. The CEC adjudication does have certain limitations. The criteria used to define MI must be evaluated. The determination of MI may require more clinical judgment in patients with inconsistent clinical history, cardiac enzyme data, and ECG information, or suspect cardiac enzyme data. We recognise that this may decrease the objectivity that is important for event classification, particularly in trials across geographic regions and clinical practice settings. At a minimum, events characterised by a low level of clinical certainty should not be classified as MIs.
Although the absolute difference in infarction rates determined by the CEC versus site investigators was small, the relative difference was more substantial. This caused a substantial impact on the statistical outcome of the trial. This phenomenon must be considered in calculations of sample size and power during design of future trials, and also may influence how clinical and regulatory bodies interpret trial results. In addition, the CEC process used must be considered, among other factors, when performing comparisons of event rates between trials.
Nonfatal MI is an important clinical outcome, and its prevention is a key measure of efficacy in evaluating new therapies for acute coronary syndromes. The determination of MI can be difficult in clinical practice due to conflicting clinical, cardiac enzyme, and ECG data. The CEC adjudication of MI provides a standard, systematic, independent, and unbiased assessment of this end-point in clinical investigations. In the PURSUIT trial, the assessments of MI by site investigators and a central CEC disagreed; more MIs were identified by the CEC than by site investigators. Most disagreements reflected underreporting of procedure-related events and misclassification of enrolment MIs as post-enrolment (end-point) MIs. Understanding the CEC process used in a trial is important in interpreting event rates, particularly when comparing rates between trials. The type of CEC process also has implications in calculations of sample size and power. Our review of the PURSUIT trial CEC experience has highlighted some of these important issues and the need to further evaluate the strategies used for event adjudication.
CEC = clinical events committee; CK-MB = creatine kinase-myocardial band; ECG = electrocardiography; MI = myocardial infarction; PCI = percutaneous coronary intervention; PURSUIT = Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin (Eptifibatide) Therapy; ULN = upper limit of normal.
Appendix: definition for post-enrolment myocardial infarction
1. Myocardial infarction (events without documentation of previous infarction during the admission): creatine kinase-myocardial band (CK-MB) level above the upper limit of normal (ULN) and ≥ 3% of total CK. If CK-MB was unavailable, then total CK >2 × ULN.
2. Myocardial reinfarction (events with documentation of an infarction before or at enrolment):
If <18 hours since previous infarction. Recurrent, severe ischemic discomfort and new, recurrent ST-segment elevation ≥ 0.1 mV in at least two contiguous leads, either persisting ≥ 30 min.
If ≥ 18 hours since previous infarction. Re-elevation of CK-MB to above the ULN (if prior CK-MB was within normal range) or >50% above the prior level (if prior CK-MB was above normal). If CK-MB was unavailable: either total CK ≥ 2 × ULN and increased by ≥ 25%; or ≥ 1.5 × ULN and increased by ≥ 200 IU above prior value.
3. Periprocedural infarction (events occurring during or within 24 hours of percutaneous coronary intervention): CK-MB level ≥ 3 × ULN and >50% above prior nadir value. If CK-MB was unavailable, total CK ≥ 3 × ULN.
4. Perioperative infarction (events occurring during or within 36 hours of bypass surgery): CK-MB ≥ 5 × ULN (or CK, in the absence of CK-MB).
New, significant Q waves or equivalents ≥ 0.04 s in at least two contiguous leads. When enzyme or electrocardiographic (ECG) data were unavailable, an infarction was identified when the bulk of clinical evidence (patient signs, symptoms, ECG changes, and pathological findings) so indicated.
The PURSUIT study was funded by grants from COR Therapeutics, Inc. (South San Francisco, CA, USA) and Schering-Plough Corporation (Madison, NJ, USA).
The Antiplatelet Trialists' Collaboration: Collaborative overview of randomised trials of antiplatelet therapy I: prevention of death, myocardial infarction, and stroke by prolonged antiplatelet therapy in various categories of patients.
Califf RM, Abdelmeguid AE, Kuntz R, Popma JJ, Davidson CJ, Cohen EA, Kleiman NS, Mahaffey KW, Topol EJ, Pepine CJ, Lipicky RJ, Granger CB, Harrington RA, Tardiff BE, Crenshaw BS, Bauman RP, Zuckerman BD, Chaitman BR, Bittl JA, Ohman EM: Myonecrosis after revascularization procedures.
The PURSUIT Investigators: Inhibition of platelet glycoprotein IIb/IIIa with eptifibatide in patients with acute coronary syndromes without persistent ST-segment elevation: a randomized, placebo-controlled, clinical trial.
The SAVE Investigators: Effect of captopril on mortality and morbidity in patients with left ventricular dysfunction after myocardial infarction: results of the Survival and Ventricular Enlargement Trial.
N Engl J Med 1992, 327:669-677. PubMed Abstract
Contr Clin Trials 1997, 18:27-42. Publisher Full Text
Contr Clin Trials 1984, 5:241-244. Publisher Full Text
J Clin Epidemiol 1990, 43:1285-1295. PubMed Abstract
Early Treatment Diabetic Retinopathy Study Research Group: Grading diabetic retinopathy from stereoscopic color fundus photographs – an extension of the modified Airlie House classification. ETDRS report number 10. Early Treatment Diabetic Retinopathy Study Research Group.
Thromb Haemost 1997, 78:1189-1192. PubMed Abstract
Mahaffey KW, Harrington RA, Akkerhuis M, Kleiman NS, Berdan LG, Tardiff BE, Granger CG, Crenshaw BS, DeJong I, Bhapkar M, Lee KL, Deckers J, Simoons ML, Topol EJ, Califf RM, for the PURSUIT Investigators: Systematic adjudication of myocardial infarction end-points in an international clinical trial.
Thrombolysis in Myocardial Ischemia (TIMI) Study Group: Effects of tissue plasminogen activator and a comparison of early invasive and conservative strategies in unstable angina and non-Q-wave myocardial infarction. Results of the TIMI IIIB trial.
Circulation 1994, 89:1545-1556. PubMed Abstract
The RESTORE Study Group: Effects of platelet glycoprotein IIb/IIIa blockade with tirofiban on adverse cardiac events in patients with unstable angina or acute myocardial infarction undergoing coronary angioplasty.
Akkerhuis KM, Deckers JW, Boersma E, Harrington RA, Stepinska J, Mahaffey KW, Wilcox RG, Lincoff AM, Keltai M, Topol EJ, Califf RM, Simoons ML, for the PURSUIT Investigators: Geographic variability in outcomes within an international trial of glycoprotein IIb/IIIa inhibition in patients with acute coronary syndromes: results from PURSUIT.
JAMA 1997, 278:479-484. PubMed Abstract