Article, Emergency Medicine

Inhomogeneity and temporal effects in AutoPulse Assisted Prehospital International Resuscitation—an exception from consent trial terminated early

Original Contribution

Inhomogeneity and temporal effects in AutoPulse Assisted Prehospital International Resuscitationan exception from consent trial terminated early?

Norman A. Paradis MD a, Gregory Young MS b, Stanley Lemeshow PhD b,

James E. Brewer MS a,b,c, Henry R. Halperin MD, MA c,?

aDepartment of Emergency Medicine, University of Southern California and ZOLL Circulation, Sunnyvale, CA, USA

bCenter For Biostatistics, College of Public Heath, Ohio State University; Columbus, OH, USA

cDepartments of Medicine, Radiology and Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA

Received 24 December 2009; revised 26 January 2010; accepted 10 February 2010

Abstract

Background: The ASPIRE trial (AutoPulse Assisted Prehospital International Resuscitation) was multicenter exception from consent clinical trial that compared mechanical cardiopulmonary resuscitation (CPR) with a device (AutoPulse-CPR) to traditional Manual CPR (manual-CPR) in out- of-hospital cardiac arrest. Enrollment was suspended early due to Safety concerns. One site (site C) made a potentially important protocol change midtrial, and enrollment at that site was noted to be independently associated with outcome.

Methods: The study used a post hoc reanalysis of source data and documentation using standard statistical approaches evaluating for possible secular, temporal, and trial design, factors that may have related to the trial’s outcome.

Results: The protocol change at site C also appears to have resulted in a delay in application of AutoPulse- CPR. Before and after the protocol change survival in patients receiving AutoPulse-CPR decreased from 19.6% to 4% (P = .024). Logistic regression analysis showed site C was significantly different (P = .008) from the remaining sites with respect to survival. Unlike site C, the other sites actually showed an increase over time in the primary end point of 4-hour survival (P = .008) favorable to AutoPulse-CPR. There did not appear to be significant safety (P = .42) nor efficacy concerns (P = .17) at these sites.

Conclusions: The difference in survival that caused early suspension of ASPIRE appears to have been limited to one site after its protocols change. At the time the trial was suspended, the outcomes of patients at the other sites appear to have been trending in favor of the intervention.

(C) 2010

? Disclosure: Dr Norman Paradis is an employee of ZOLL Circulation. Dr Stanley Lemeshow and Mr Gregory Young are consultants of ZOLL Medical Corporation, and their financial interests are governed by policies of Ohio State University. Mr Brewer is a consultant of ZOLL Medical. Dr Henry Halperin is a consultant of ZOLL Medical Corporation, and his financial interests are governed by policies of Johns Hopkins University.

* Corresponding author. Johns Hopkins Hospital, Baltimore, MD 21205, USA.

0735-6757/$ – see front matter (C) 2010 doi:10.1016/j.ajem.2010.02.002

Introduction

On March 31, 2005, the ASPIRE trial (AutoPulse Assisted Prehospital International Resuscitation, Clinical- Trials.gov NCT00120965), a prospective cluster-random- ized comparison of standard Manual cardiopulmonary resuscitation (manual-CPR) to CPR performed by a mechanical device (AutoPulse-CPR, ZOLL Circulation Inc, Sunnyvale, Calif), was terminated early by the data safety and monitoring board (DSMB) [1]. Although the predeter- mined primary outcome, 4-hour survival, remained similar between the 2 groups, hospital discharge and survival with intact neurologic outcome appeared to be worse with AutoPulse-CPR. ASPIRE was conducted under exception from consent [2]. Only a small number of clinical trials have been conducted under exception from consent, and ASPIRE was among the few to be terminated early for safety.

The AutoPulse provides chest compression with an automated load distributing band [3]. It had been developed to automate and optimize chest compression, and improved perfusion pressures, blood flows, and survival rates had been observed in preclinical studies and in limited uncontrolled case series [4-6]. Because the AutoPulse distributes its force over a larger area of the anterior chest than standard manual-CPR, there was also the possibility that it might be less injurious.

ASPIRE was intended to more prospectively compare standard manual-CPR, as defined by the American Heart Association’s Guidelines 2000 [7], to AutoPulse-CPR in out- of-hospital cardiac arrest patients on an intention-to-treat (ITT) basis in 3 US and 2 Canadian sites. The primary efficacy comparison was 4-hour survival in the subgroup of patients later determined to have had a cardiac etiology for their arrest. A secondary safety outcome was survival to hospital discharge.

Because ASPIRE’s unexpected outcome was difficult to explain in the context of preliminary studies and current understanding of CPR hemodynamics, we wondered if it might be explained by the design and execution of the trial itself. We therefore undertook a post hoc reanalysis of the primary data, accompanied by an examination of the trial design and execution, looking specifically for just such effects.

Material and methods

The ASPIRE trial methods and results have previously been reported [1]. In our reanalysis, we used original data, source documents, and results of an independent audit of the study sites themselves. In evaluating the trial, we compared its design and execution to recognized Good Clinical Practice as described by the International Conference on Harmonization [8].

An particularly unusual aspect of the trial design was that each site was permitted to choose one of 3 variations of the

protocol: (1) evaluation of cardiac rhythm, 2 minutes of randomized manual-CPR or AutoPulse-CPR, electrical countershock if appropriate, then repeat randomized CPR and shock delivery; (2) 2 minutes of manual-CPR, shock delivery if appropriate, then one minute of randomized manual-CPR or AutoPulse-CPR, then repeat shock delivery and randomized CPR; or (3) immediate shock delivery if appropriate, then randomized to manual-CPR or AutoPulse- CPR, then repeat shock and randomized CPR. At the start of the trial, all sites chose the first protocol option. However, one site (site C) subsequently switched from the first protocol option to the second option approximately half way through the study, and the original logistic regression had found site C to be significantly associated with survival to hospital discharge. With this in mind, we also undertook analysis of the effects of study site and temporal sequence on outcomes. In multicenter trials, overall treatment effect is typically evaluated using summary measures and interpreted as an average treatment effect across trial sites. However, before making this assumption, it is necessary to confirm homogeneity of the observed treatment effects across sites, so-called poolability of the data [9]. A single summary measure is adequate in describing the trial results only when the data are shown to be reasonably consistent, that is,

homogenous across sites.

We used a multistep process to detect and evaluate possible associations relating to treatment site, temporal effects, and patient outcomes. In particular, we evaluated any heterogeneity and effect on the outcome related to site C and its protocol change.

Our analytic plan included the following:

    1. Determine if site C was homogenous with the other sites, or should be separated for subsequent analysis, as demonstrated by significant statistical interaction between treatment (manual-CPR or AutoPulse-CPR), study site, or time.
    2. If a significant statistical interaction was found between site C and treatment arm, we then determined if the remaining sites were statistically homogeneous, thereby, permitting the study data derived from these 4 other sites to be analyzed together.
    3. We then reanalyzed the outcome of the study at the other sites.
    4. If analyses 1 to 3 are positive, we further analyzed time-dependent effects at site C and the 4 other sites independently.

As with any new technology, the efficacy of mechanical CPR may be associated with a learning curve as emergency medical services personnel become familiar with the application of the device. With this in mind, we also analyzed possible associations relating to initiation of AutoPulse-CPR and temporal effects at each site.

Logistic regression analysis is considered standard in analyzing clinical data with a dichotomous outcome [10,11]

and was used to assess whether treatment effect differed among trial centers. Two and 3-factor logistic regression models, with all interaction terms, were developed to discern interactions indicating trial outcome heterogeneity between site C and the remaining sites.

To test the trial data for a possibly significant interaction between site C and the AutoPulse-CPR treatment arm, 2 logistic regression models were developed. A 2-factor logistic model was developed to evaluate the survival to hospital discharge outcomes over the course of trial. A 3- factor logistic regression model was then developed to test the hypothesis that site C had a significant interaction with treatment effect on the trial’s primary end point (4-hour survival) over the course of the trial. After the evaluation of site C’s interaction with treatment effect over time, standard logistic regression methods were applied to the outcome data for the 4 remaining sites to determine their efficacy and safety results.

After preliminary logistic regression analysis, regression model robustness was confirmed with (1) tests for linearity for the time regression variable (patient event order), (2) tests for the inclusion of appropriate covariates for model factor adjustment, (3) adjustment for site by site cluster randomi- zation methods, (4) tests for influence of specific subjects (outlier analysis), and (5) tests for overall model goodness-of- fit. All analyses were conducted using established statistical software: SAS version 9.1 (SAS Institute, Cary, NC), and STATA version 9.2 (Statacorp, College Station, Tex).

In addition, the CUSUM change-point method of Basseville was used to determine if an abrupt statistical change occurred in site C’s AutoPulse-CPR data. The method is based on the integrated log ratio of likelihoods (log-ratio) [12-14].

Results

A priori, the ASPIRE investigators chose as their primary comparison population the subset of patients whose cardiac arrest appeared to be of cardiac etiology. However, no clinical events committee or other procedures were in place to prevent bias in this adjudication, and it was reportedly performed by the study coordinator or investigator at each site. Measures to prevent enrollment bias are particularly

Table 1 Survival to hospital discharge for each treatment arm, site C, and remaining 4 sites (ITT, n = 1071)

important in trials conducted under cluster randomization [15], and the selection of a subgroup postevent, only, may amplify this risk. For this reason, our analysis was performed on the more complete ITT patient data set (n = 1071).

ASPIRE was undertaken at a small number of sites with relativley large baseline intersite differences, including differences in the cluster definition at each site [1]. This supported our decision to confirm homogeneity between sites and over time.

Initial contingency table analysis partitioning treatment effects between site C and the 4 remaining sites revealed a considerable difference in survival to hospital discharge (Table 1). We found that the relationship between site C, AutoPulse-CPR treatment, and survival was significant and negative. Site C had significantly fewer patients in the AutoPulse treatment arm survive to hospital discharge (11.5%) when compared to the manual-CPR arm (26.7%). The remaining 4 sites showed no differences in survival to discharge between AutoPulse and manual-CPR.

At site C, overall survival to hospital discharge was 19% before the change in protocol then decreased dramatically to 4% (Table 2; Fisher exact test, P = .02). The CUSUM change-point method of Basseville detected significant change-points only for site C, coinciding with the date the change was known to have occurred (P = .02). In patients requiring electrical countershock, accurate AutoPulse de- ployment times were available in only a small number of patients (n = 21). In these patients, AutoPulse was applied 111 +- 26 seconds before countershock before the protocol change vs 112 +- 51 seconds postshock after the change (P b

.0001), a greater than 3 1/2 minute delay in application of the intervention.

After demonstration of significant site C treatment differences for hospital discharge, a sequence of logistic regression models were developed to test the hypothesis that site C had a significant negative interaction on the trial’s primary end point of 4-hour survival over time, defined as the order in which patients were enrolled.

Logistic regression analysis was performed to determine whether temporal trends existed in the rates for survival to 4 hours. The regression model used an explicitly constructed, and standardized, time variable, ORDS, representing the order of patient event in the trial. This was defined by ordering the site by Treatment outcomes data by the date and

AutoPulse Manual-CPR P OR Lower 95% OR boundary Upper 95% OR boundary

Site A

5/161

9/153

.281

0.513

0.168

1.566

Site B

4/60

2/51

.685

1.750

0.307

9.972

Site C

11/96 (11.5%)

28/105

(26.7%)

.006

0.356

0.166

0.763

Site D

16/185

10/161

.421

1.430

0.630

3.246

Site E

4/52

5/47

.732

0.700

0.176

2.778

All sites

40/555 (7.2%)

54/516

(10.5%)

.060

0.665

0.433

1.019

Other 4 sites

29/459 (6.3%)

26/411

(6.3%)

1.0

1.0

0.578

1.726

Table 2 Survival to hospital discharge for AutoPulse patients, site C only

Survival to discharge Total

adjustment was required. Model diagnostics were run to determine influential patient data. A single patient was identified as a highly influential subject and on that basis was removed (n = 1070).

No

Yes

The scale of the ORDS variable was assessed using

Before protocol change

37 (80.4%)

9 (19.6%)

46

fractional polynomial methods in a model that only included

Subsequent to

48 (96.0%)

2 (4.0%)

50

main effects. The fractional polynomial model had a strong

protocol change

Total 85 11 96

Before and after its protocol change (Fisher P = .024).

linear coefficient (P = .001), and the model deviance was not significantly reduced transforming ORDS variable. Lowess

plots also showed no significant departure from linearity.

time of a patient’s emergency call. An important temporal trend was demonstrated. Toward the beginning of the trial, the odds ratio (OR) for survival was significant in favor of manual-CPR (Table 3; OR, 0.606; P = .028, at time of 100th patient enrolled). However, over time, the treatment advantage of manual-CPR steadily decreased and eventually reversed. By the time the trial was terminated, the AutoPulse treatment arm was showing a steadily improving 4-hour survival (OR, 1.610 in favor of AutoPulse; P = .068, at time of 1000th patient enrolled; Table 3).

Because of the significant difference between site C and the remaining 4 sites with respect to survival to 4 hours and hospital discharge over the course of the trial, logistic regression analysis was performed to test the hypothesis that site C had a significant interaction with treatment effect on the trial’s primary end point (4-hour survival) over the course of the trial. This analysis included first- and second- order interactions between site C, ITT outcomes, and patient event order.

The model robustness was tested using risk factor modeling of patient and site-specific covariates. The unadjusted coefficient for the “ITTA*ORDS*SITC” term (ITT, patient event order, and site C terms, respectively) was

-0.822675 (P = .025). The model coefficients were adjusted

using covariates cited in the ASPIRE report. Using risk factor modeling, these covariates did not significantly alter the unadjusted interaction coefficient, and so, no covariate

Table 3 Odds ratios for 4-hour survival, comparing AutoPulse-CPR to manual-CPR, over the course of the trial (all trial sites, ITT, n = 1071)

Because of the practical difficulties in randomizing individual patients in ASPIRE, a cluster randomization scheme was used with vehicle as the clustering factor. As the assumption of independence may be violated by observa- tions within the same cluster, analyses must account for the possible correlation to properly estimate treatment effects and their significance. A total of 177 clusters varying in size from 1 to 30 (mean, 6.1) were identified in this study. A general estimation equations framework was used to compute the model, assuming an exchangeable correlation structure within the clusters. After adjustment for clustering and outliers, the P value of the interaction term modeling site C’s protocol change on the primary outcome increased from

0.025 to 0.045 (0.074 including the outlier mentioned previously). Overall goodness of fit was tested using the Hosmer-Lemeshow test with deciles of risk. There was no indication of a significant lack of fit (P = .81). The regression model, computed using robust methods and adjusting for significant covariates, cluster randomization, and outliers is included in Table 4.

Fig. 1 (Table 5) compares the relative treatment effects between site C vs the other 4 sites and displays the change over time in the OR of survival to 4 hours based on the model estimates. During what became the first half of the trial, there was no difference in the effect of treatment between site C and the other sites (patient event order 100-500; Table 5). However, for patients subsequent to that point, corresponding to the patients after the site C protocol change (patient event order 541), the difference in treatment effect was significant

Patient event order

OR (AutoPulse/ manual-CPR)

Lower 95% confidence interval boundary

Upper 95% confidence interval boundary

P

100

0.606

0.388

0.946

.028

200

0.675

0.462

0.987

.043

300

0.753

0.545

1.040

.085

400

0.839

0.632

1.114

.226

500

0.935

0.715

1.223

.625

600

1.043

0.787

1.380

.771

700

1.162

0.846

1.597

.354

800

1.295

0.893

1.880

.173

900

1.444

0.932

2.238

.100

1000

1.610

0.966

2.682

.068

Survival to 4 h

Model coefficient

Standard error

Wald

z

P N

|z|

Lower 95% confidence interval boundary

Upper 95% confidence interval boundary

ITTA

0.0943

0.1458

0.65

.518

-0.1915

0.3802

ORDS

-0.3846

0.1272

-3.02

.002

-0.6340

-0.1353

SITC

0.9227

0.2394

3.85

.000

0.4534

1.3920

ITTA*ORDS

0.3668

0.1656

2.22

.027

0.0422

0.6913

ITTA*SITC

-0.5634

0.3616

-1.56

.119

-1.2721

0.1453

ORDS*SITC

0.1784

0.2676

0.67

.505

-0.3461

0.7029

ITTA*ORDS*SITC

-0.9054

0.4493

-2.02

.044

-1.7860

-0.0249

Constant

-1.0870

0.1210

-8.99

.000

-1.3240

-0.8499

ITTA indicates intention to treat (AutoPulse treatment = 1, manual-CPR treatment = 0); ORDS, (standardized) patient event order (range, 1-1070); SITC, site C (site C = 1, not site C = 0); general estimation equations population-averaged model (Stata xtgee module; logit link, binomial family, exchangeable correlation, standard error adjusted for clustering).

(patient event order 600-1000; Table 5). This can also be seen in Figs. 2 and 3, showing the Z statistic [13], providing a cumulative measure of the relative benefit of AutoPulse-CPR over manual-CPR for the 4 sites and site C, respectively. At the time the trial was suspended, the OR for 4-hour survival in the remaining 4 homogeneous sites was greater than 6 times the OR for site C and strongly in favor of survival due to AutoPulse treatment (P b .01; Table 5). The logistic

Table 4 Three-factor logistic regression model for patient survival to 4 hours, evaluated for significant covariate adjustment, adjusted for cluster randomization, adjusted for outliers (all sites, ITT, n = 1070, Wald ?2 of 33.55; P b .0001)

Table 5 Odds ratio for 4-hour survival AutoPulse-CPR to manual-CPR for site C compared to 4 remaining sites, over the course of the trial (all sites, ITT, n = 1070)

Fig. 1 Odds for 4-hour survival in the AutoPulse-CPR treatment arm compared to the manual-CPR arm, over the course of the trial (as represented by the order of patient events); blue line represents site C, and the red line represents the remaining 4 sites. The AutoPulse-CPR steadily increased the likelihood for survival over manual-CPR at the remaining 4 sites, whereas it steadily decreased the likelihood for survival at site C.

regression analysis was repeated using a dichotomous patient order term centered at the known date of change. Again, significant treatment effects were found (P = .026).

For the remaining 4 sites (870 patients), standard regression analysis, with each of the sites as a factor in the analysis along with interaction terms, was used to evaluate survival to hospital discharge (Table 6) and 4-hour survival (Table 7). The analysis showed no significant effects (P = .42). The analysis was verified by determining whether there were significant effects without interactions. Neither this second analysis nor its difference with the interaction analysis were significant (P = .39, P = .41, respectively). With respect to survival to hospital discharge, there does not seem to have been safety concerns at the remaining 4 sites. Although the OR of AutoPulse to manual-CPR increased over time at the 4 other sites, it remained nonsignificant.

Similarly, logistic regression analysis showed that there was no overall (beginning to end) treatment effect for the remaining 4 sites (P = .39) and confirmed that there was no

Patient event order

OR of AutoPulse vs manual-CPR for site C

OR of AutoPulse vs manual-CPR for remaining

4 sites

P

100

1.337

0.655

.342

200

1.123

0.738

.503

300

0.944

0.831

.805

400

0.793

0.935

.697

500

0.666

1.053

.215

600

0.560

1.186

.041

700

0.470

1.335

.013

800

0.395

1.503

.009

900

0.332

1.692

.009

1000

0.279

1.905

.010

Fig. 2 The test statistic Z for the 4 homogenous sites. The test statistic Z, as defined by Whitehead, provides a cumulative measure of the advantage of the experimental therapy, along with the test statistic V that provides a measure of the information contained in Z about the treatment difference, and where the traditional ?2 statistic can be shown to be Z2/V. After an initial period favoring manual- CPR, the trend changes, and by the time the trial was terminated, the Z statistic showed a sizeable advantage for AutoPulse-CPR.

significant treatment effects with or without interactions (P = .11, P = .62, respectively). However, again, the OR for 4-hour survival of AutoPulse-CPR compared to manual- CPR steadily increased over time. At the time the trial was suspended, the odds of 4-hour survival with AutoPulse in the remaining 4 sites was 2.2 times better than the odds for manual-CPR (P = .008; 95% confidence interval, 1.2-3.9).

Review of source documents revealed a number of potential variances from Good Clinical Practice. Among those that may have contributed to the trial’s problematic outcome were as follows:

    1. Failure to include an adequate number of sites homogenous in their baseline clinical characteristics.

Fig. 3 Same as in Fig. 2 but for site C. Although the Z statistic remained relatively constant until the time of the protocol change, subsequently, there was a sizeable decrease in favor of manual-CPR.

    1. No single protocol, with protocol changes in one site not reflected in other sites.

Table 6 Odds ratio of AutoPulse-CPR to manual-CPR, over the course of the trial, for survival to hospital discharge (remaining 4 sites, ITT, n = 870)

Patient event order

OR

Lower 95% confidence interval boundary

Upper 95% confidence interval boundary

P

100

0.812

0.337

1.960

.644

200

0.861

0.418

1.774

.686

300

0.913

0.500

1.669

.768

400

0.968

0.559

1.676

.908

500

1.026

0.576

1.830

.930

600

1.088

0.551

2.150

.808

700

1.154

0.503

2.646

.736

800

1.223

0.448

3.341

.695

    1. Variability in cluster randomization, possibly with inadequate protections to prevent bias.
    2. No clinical events committee or standardized protocol to adjudicate the primary subgroup for analysis. Possible inadequate blinding of individual site adjudicators.
    3. Possible inadequacies in clinical trial and DSMB analytic plans in light of heterogeniety of the study sites [16].

Discussion

Our reanalysis of the ASPIRE trial produced a number of potentially important results. In particular, our analysis indicates that the apparent adverse safety results that caused the DSMB to terminate the trial early were localized to one site and that these adverse results occurred after the change in trial protocol at that site.

Table 7 Odds ratio of AutoPulse-CPR to manual-CPR for patient survival to 4 hours, over the course of the trial (as represented by patient event order) at the 4 homogeneous sites (ITT, n = 870)

Patient event order

OR

Lower 95% confidence interval boundary

Upper 95% confidence interval boundary

P

100

0.630

0.383

1.036

.069

200

0.752

0.501

1.130

.171

300

0.898

0.639

1.262

.536

400

1.072

0.786

1.464

.659

500

1.280

0.920

1.782

.143

600

1.529

1.034

2.260

.034

700

1.825

1.132

2.942

.014

800

2.179

1.222

3.885

.008

The other 4 sites appeared to be more homogenous. When their data were pooled and reanalyzed, it showed a trend that might be expected when a new device is deployed into a complex clinical environment. During the first portion of the trial, the relationship between treatment and survival favored the established therapy manual-CPR. However, over time, there was a consistent linear improvement in the outcome of patients randomized to AutoPulse-CPR compared to manual- CPR. Remarkably, at the time the trial was prematurely terminated for safety, patients receiving AutoPulse-CPR at the 4 homogenous sites were more than twice as likely to be alive at 4 hours, the study’s primary end point.

It is widely acknowledged that care must be taken in design and implementation of clinical trials. The protocol should be clear and similar at all centers [11]. In a multicenter trial, the different sites must be homogenous enough in their baseline clinical characteristics, as well as their execution of the protocol, that the resulting data may be analyzed in aggregate. If heterogeneity is found, the results must be interpreted with care, and vigorous attempts should be made to find causes of the heterogeneity in the design or execution of the trial. Failure to account for such heterogeneity may reduce the value of a multicenter trial to such a degree that it cannot be regarded as giving convincing evidence for reported results.

Not only should the statistical plan for clinical trials include evaluations of homogeneity, but, as indicated by our analysis of ASPIRE, also the statistical plan for the DSMB [16]. In addition, it is widely recognized that the DSMB analytic plan should also include evaluation for temporal effects [16]. An early trend toward harm may simply reflect a learning curve incumbent in roll out of new technology.

It is not unreasonable to use cluster randomization when evaluating resource intensive medical devices whose application to individual patients cannot be blinded. However, the risk of bias in these types of trials has been recognized [15]. Care must be taken that the clustering mechanism is similar at different sites and the trial design is as robust as possible. In particular, the investigators must endeavor to confirm that enrollment bias has not occurred. In a prehospital study such as ASPIRE, care should be taken to confirm that all eligible patients within each site have been accounted for within the clusters.

The ASPIRE’s investigators had prospectively decided that their analysis would be limited to the subset of patients without exclusions and whose cardiac arrest was deemed to be of cardiac etiology. Although this is not unreasonable, review of source documentation by an independent auditor indicated that no formal effort had been made to blind the investigator at each site when they adjudicated cases for inclusion based on etiology. The decision to terminate the trial early because of safety appears to have been based only on an analysis of the primary subset. Our reanalysis indicates that this may not have been a complete picture of the safety profile at the time the trial was suspended. Statistical tests for interaction that assess whether a treatment effect differs

between trial sites are considered standard in analyzing clinical data [9] and should be reported in any publication of the trial results.

Of necessity, application of a mechanical CPR device will include some period of interruption. By definition, ASPIRE’s 3 protocol options meant that the potential benefit of the device, along with any potential harm relating to interruption in the CPR during the device’s placement, would occur at different time points in the resuscitation sequence. Although the data were limited, our analysis found that the protocol change at site C may have resulted in 3.7- minute delay in application of the device. As we have shown, this change was associated with a profound worsening in the outcome of study patients. Although it was not intended, site C may have essentially performed a single site before-after trial evaluating the sequencing of mechanical CPR. The result indicates that it may not be safe to delay application of such therapies until after a period of failed manual-CPR and failed defibrillation. The difficulties associated with excep- tion from consent clinical trials may result in these data remaining permanently unique.

In both the single heterogeneous site and the 4 homogeneous sites, the odds in favor of survival to discharge decreased significantly as the AutoPulse deployment was delayed. A phased, observational, cohort analysis of 783 patients published cotemporaneously with ASPIRE showed a survival improvement during its later phase [4]. Although this improvement in outcome may have been a secular trend unrelated to the AutoPulse [17], the authors noted that early application of the device was a key element of their protocol and all benefit occurred when the response interval was less than 8 minutes.

The ORs for survival of AutoPulse-CPR compared to manual-CPR in the 4 homogeneous sites steadily increased over time (Fig. 2). Methodologically, the OR in favor of AutoPulse-CPR at the time of termination of the trial cannot be said to be predictive of the future course of the trial had it continued enrollment to completion. However, the results of our reevaluation suggests that further trials of the device would be reasonable and that it may not be appropriate to conclude that the AutoPulse is unsafe based on ASPIRE.

Our analysis is consistent with the assumption that resuscitation clinical trials of complex therapies may have a secular trend as a result of learning. Although ASPIRE included a run-in period to allow for training, our analysis indicates that such phases may need to be more robust in trials in resuscitation trials and may need to include the full research protocol. A number currently enrolling trials incorporate methodologies to address this issue.

The principal limitation in our reanalysis, clearly, is that it is retrospective by definition and it is a second examination of the data. At best, it should be considered hypothesis generating with respect to the status of the AutoPulse device and limited as an explanation for the outcomes in ASPIRE. However, it should be noted that our approach, although retrospective, would have been considered by many to be a

standard prospective analytic plan and that we applied it to the largest possible intention to treat set of patients. It should also be emphasized that a number of us have potentially biasing commercial relationships to the technology in question (see disclosures).

The ASPIRE investigators engaged a difficult clinical problem, and the DSMB likely terminated the trial in good faith. Our reanalysis should not be taken as a critique of these efforts. Rather, it again emphasizes the importance of good clinical practice and the rigorous application of robust analysis plans to the real-world execution of challenging research [18].

Conclusion

In the ASPIRE trial, the treatment effect in one site differed significantly before and after a change in the protocol and was also different from the remaining 4 sites. The difference in survival to hospital discharge that caused the DSMB to suspend the trial early appears to have been limited to this one site after its protocol change. The 4 other sites demonstrated homogeneous treatment effects. At the time the trial was suspended, the outcomes of patients at these sites appear to have been trending in favor of the intervention. Clinical trial statistical plans often and appropriately include the valuations of homogeneity and temporal effects.

References

  1. Hallstrom A, Rea TD, Sayre MR, et al. Manual chest compression vs use of an automated chest compression device during resuscitation following out-of-hospital cardiac arrest: a randomized trial. JAMA 2006;295(22):2620-8.
  2. Halperin H, Paradis N, Mosesso Jr V, et al. Recommendations for implementation of community consultation and public disclosure under the Food and Drug Administration’s “Exception from informed consent requirements for emergency research“: a special report from the American Heart Association Emergency Cardiovascular Care Com-

mittee and Council on Cardiopulmonary, Perioperative and Critical Care: endorsed by the American College of Emergency Physicians and the Society for Academic Emergency Medicine. Circulation 2007;116 (16):1855-63.

  1. Halperin HR, Paradis N, Ornato JP, et al. Cardiopulmonary resuscitation with a novel chest compression device in a porcine model of cardiac arrest: improved hemodynamics and mechanisms. J Am Coll Cardiol 2004;44(11):2214-20.
  2. Ong ME, Ornato JP, Edwards DP, et al. Use of an automated, load- distributing band chest compression device for out-of-hospital cardiac arrest resuscitation. JAMA 2006;295(22):2629-37.
  3. Casner M, Andersen D, Isaacs SM. The impact of a new CPR assist device on rate of return of spontaneous circulation in out-of-hospital cardiac arrest. Prehosp Emerg Care 2005;9(1):61-7.
  4. Timerman S, Cardoso LF, Ramires JA, Halperin H. Improved hemodynamic performance with a novel chest compression device during treatment of in-hospital cardiac arrest. Resuscitation 2004;61 (3):273-80.
  5. Guidelines 2000 for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Part 3: adult basic life support. The American Heart Association in collaboration with the International Liaison Committee on Resuscitation. Circulation 2000;102(8 Suppl):I22-59.
  6. Dixon Jr JR. The International Conference on Harmonization Good Clinical Practice guideline. Qual Assur 1998;6(2):65-74.
  7. Localio AR, Berlin JA, Ten Have TR, Kimmel SE. Adjustments for center in multicenter studies: an overview. Ann Intern Med 2001;135 (2):112-23.
  8. Hosmer DW, Lemeshow S. Applied Logistic Regression. 2nd ed. New York: John Wiley and Sons; 2000.
  9. Lewis JA. Statistical principles for clinical trials (ICH E9): an introductory note on an international guideline. Stat Med 1999;18(15): 1903-42.
  10. Brostrom G. A martingale approach to the change point problem. J Am

Stat Assoc 1997;439:1177-83.

  1. Pettitt AN. A simple cumulative sum type statistic for the change point problem with zero-one observations. Biometrika 1980;67:79-84.
  2. Basseville M, Nikiforov IV. Detection of abrupt changes: theory and application. Engelwood (NJ): Prentice-Hall; 1993.
  3. Puffer S, Torgerson D, Watson J. Evidence for risk of bias in cluster randomised trials: review of recent trials published in three general medical journals. BMJ 2003;327(7418):785-9.
  4. Ellenberg S, Fleming TR, Demets DL. Data Monitoring Committees in Clinical Trials: a practical perspective. John Wiley and Sons; 2002.
  5. Lewis RJ, Niemann JT. Manual vs device-assisted CPR: reconciling apparently contradictory results. JAMA 2006;295(22):2661-4.
  6. Nissen SE. ADAPT: the wrong way to stop a clinical trial. PLoS Clin Trials 2006;1(7):e35.

Leave a Reply

Your email address will not be published. Required fields are marked *