Original Contribution

Assessment of the pulmonary embolism rule-out criteria rule for evaluation of suspected pulmonary embolism in the emergency department

Stephen J. Wolf MD^a,e,?, Tracy R. McCubbin MD^b,e, Kristen E. Nordenholz MD^c,e,

N. Ward Naviaux MD^c,e, Jason S. Haukoos MD, MS^a,c,d,e

^aDepartment of Emergency Medicine, Denver Health Medical Center, Denver, CO 80204, USA

^bDepartment of Emergency Medicine, Kaiser Permanente/Exempla St Joseph Hospital, Denver, CO 80218, USA

^cDivision of Emergency Medicine, University of Colorado Health Sciences Center, Denver, CO 80262, USA

^dDepartment of Preventive Medicine and Biometrics, University of Colorado Health Sciences Center, Denver, CO 80262, USA

^eThe Colorado Clot Consortium, Denver, CO, USA

Received 5 March 2007; revised 16 April 2007; accepted 20 April 2007

Abstract

Background: Overuse of resources when evaluating pulmonary embolism (PE) is a concern if the D-dimer assay is improperly used in the evaluation of emergency department patients with suspected PE. The Pulmonary Embolism Rule-out Criteria rule was derived to prevent unnecessary diagnostic testing in this patient population. The objective of this study was to assess the PERC rule’s performance in an external population.

Methods: This was a secondary analysis of a prospectively collected database comparing PERC rule variables to diagnosis of PE in consecutive patients with suspicion for PE. Bivariate analysis on individual variables and the overall accuracy of the PERC rule were performed.

Results: Patients on 120 randomly assignED shifts were enrolled with a PE prevalence of 12%. The sensitivity, specificity, positive predictive, and negative predictive values of the PERC rule were 100% (95% confidence interval [CI], 79%-100%), 16% (95% CI, 10%-24%), 14% (95% CI, 8%-14%), and

100% (95% CI, 80%-100%), respectively, for the total patient population, and 100% (95% CI,

25%-100%), 33% (95% CI, 12%-35%), 2% (95% CI, 0%-11%), and 100% (95% CI, 75%-100%),

respectively, for the Low pretest probability population. Bivariate analysis showed unilateral Leg swelling, recent surgery, and a history of venous thromboembolic event to be predictive of the diagnosis of PE. Conclusions: The PERC rule may identify a cohort of patients with suspected PE for whom diagnostic testing beyond history and physical examination is not indicated.

(C) 2008

* Corresponding author. Department of Emergency Medicine, Denver Health Medical Center, Mail Code 0108, Denver, CO 80204, USA. Tel.: +1 303 436 8842; fax: +1 303 436 7541.

E-mail address: [email protected] (S.J. Wolf).

Introduction

Incorporation of the D-dimer into the evaluation of patients with suspected pulmonary embolism (PE) has changed the diagnostic algorithm considerably [1]. The

0735-6757/$ - see front matter (C) 2008 doi:10.1016/j.ajem.2007.04.026

D-dimer is now used to exclude PE in patients with low pretest probability determined either implicitly or explicitly [2,3]. When used in the proper settings, the D-dimer has the benefit of potentially decreasing resource use and increasing the efficiency of a patient’s evaluation for PE by averting radiographic evaluation without sacrificing overall diagnos- tic accuracy. However, when the D-dimer is used in patient populations with either too high or too low pretest probabilities, this new arm to the diagnostic algorithm has limitations. Specifically, given that this test is relatively easy and less costly to obtain, compared to radiographic studies, clinicians may lower their thresholds for initiating a medical evaluation for PE, and thus overuse the test. This, in addition to the poor specificity of the D-dimer, may result in an overall increase in radiographic evaluations performed in the very low pretest probability group that needs it least [4,5].

In 2004, Kline and colleagues [6] published a retro- spectively derived and prospectively internally validated clinical decision rule designed to identify patients with such a low pretest probability for PE that a D-dimer would not be necessary in their medical evaluations. The clinical decision rule was named the Pulmonary Embolism Rule-out Criteria rule and identified 8 clinical criteria using logistic regression modeling to exclude PE without further diag- nostic evaluation, including the use of a D-dimer. According to the rule, if all criteria are met, further evaluation is not indicated. The purpose of our study was to independently and externally validate the PERC rule.

Methods

Study design

This was a secondary analysis of a prospectively collected database for the validation of Wells criteria, a previously published clinical decision rule for deriving pretest probability in patients with suspected PE [7]. The Colorado Multiple Institutional Review Board approved this secondary analysis.

Study population

The methodology for data collection was described previously [7]. In brief, original data collection occurred from August 2001 through June 2002 at a residency- affiliated, community-based emergency department (ED), which serves a predominantly managed-care patient popula- tion (Kaiser Permanente). All variables were collected before, and without any knowledge of, the publication or presentation of the PERC rule.

Consecutive patients on 120 randomly generated, 8-hour shifts with clinical suspicion for PE after a history, physical examination, chest radiograph, and electrocardiogram were eligible for inclusion. Eligible patients were excluded if they:

(1) did not speak English; (2) were recently (b6 months) or

currently pregnant; (3) were morbidly obese (defined as a weight of more than 350 lb [160 kg] because of the weight limitations of the computed tomographic [CT] scanner used);

(4) were diagnosed previously with a know thrombophilia other than cancer; (5) were younger than 18 years or older than 85 years; (6) were critically ill or unable to consent; or

(7) were known to have a recently elevated or normal D-dimer assay.

All included patients had their pretest probability for PE calculated according to Wells’ criteria [8], an automated immunoturbidimetric D-dimer assay (Liatest D-di; Diagnos- tica Stago, Parsippany, NJ) performed, and underwent radiographic evaluation specific for PE (eg, CT angiogram of the chest or ventilation/perfusion [V/Q] scan) regardless of the D-dimer result. The sensitivity of this assay has been reported as 95% with a negative predictive value of 95%, and it has been shown as acceptable for the clinical evaluation of venous thromboembolism [9]. The CT scanner used was a General Electric single detector helical scanner (General Electric, CTI, General Electric, Fairfield, Conn). In addition, data collection occurred before the clinicians routinely used the D-dimer assay as a standard diagnostic tool in the evaluation of patients with suspected PE. As such, no enrolled patients were screened with a D-dimer assay to determine whether they required radiographic evaluation. A trained research assistant, blinded to the results of all D-dimer assays and the radiographic evaluation, performed informed consent and data collection.

Measurements

Demographic data collected included age, sex, and race/ ethnicity. Additional variables included age, initial heart rate, oxygen saturation as measured by pulse oximetry, evidence of asymmetric leg swelling, history of hemoptysis, recent surgery (defined as within 4 weeks from the time of enrollment), history of venous thromboembolism , and use of oral hormones at the time of enrollment. Each variable was subsequently dichotomized and considered negative according to the PERC rule if: (1) age was younger than 50 years; (2) initial heart rate was less than 100 beats per minute; (3) initial oxygen saturation was 90% or higher on room air; (4) no asymmetric lower leg swelling was present;

(5) no hemoptysis was reported; (6) no recent surgery was reported; (7) no history of VTE was reported; or (8) no oral hormone use was reported. Although the original PERC rule used an oxygen saturation of 95%, in our study, an oxygen saturation of 90% was used to adjust for the altitude (5, 280 ft above sea level) in which patients were enrolled.

Outcomes

End point variables included diagnosis of a PE after initial ED evaluation, diagnosis of any VTE during 3-month follow-up, or death due to VTE.

The diagnosis of PE was made when any of the following criteria were met: (1) high probability V/Q scan using modified Prospective Investigation of Pulmonary Embolism Diagnosis (PIOPED) criteria; (2) contrast-enhanced CT scan of the chest diagnostic of PE; (3) intermediate probability V/ Q scan with a high pretest clinical suspicion as determined by the treating provider; (4) pulmonary angiogram diagnostic of PE; (5) 3-month telephone interview follow-up in which the patient reported having been diagnosed with any VTE; or (6) medical record review documenting diagnosis of VTE within the 3-month follow-up period. All other patients were considered to have not been diagnosed with PE.

Medical record review was performed after 3 failed attempts at 3-month telephone interview for patients who were not initially diagnosed with PE. This follow-up included a review of their medical records and the hospital’s anticoagulation clinic enrollment for evidence of VTE. The study hospital (a Kaiser Permanente facility) maintains a comprehensive electronic medical record system that includes any clinic, emergency, or hospital visit. Patient demographic and contact information are continually updated. Any clinic, emergency, or hospital visit to any other local area hospital is logged into the electronic medical record system during authorization of the visit. If there was no entry after the Initial ED encounter, the database was queried to confirm that Kaiser Permanente insurance was still active for that patient.

Data management and statistical analyses

All data were entered into an electronic spreadsheet (Microsoft Excel, Microsoft Corporation, Redmond, Wash) and transferred into native SAS format using translational software (dfPower DBMS/Copy, DataFlux Corporation, Cary, NC). All statistical analyses were performed using SAS Version 9.1 (SAS Institute, Inc, Cary, NC).

Descriptive statistics were calculated for all variables. Median values with interquartile ranges were calculated for continuous variables and percentages with 95% confidence intervals (CIs) were calculated for categoric variables. The ?2 or Fisher exact test was performed to compare individual categoric PERC rule variables with respect to each end point. Each patient was categorized according to the PERC rule.

Table 1 Descriptive statistics for Pulmonary Embolism Rule-out Criteria rule performance based on pretest probability

Patients with any positive criterion were considered to not meet the PERC rule, meaning further evaluation for PE would be needed. All other patients were considered to meet the PERC rule, and as such be considered ruled-out for the diagnosis of PE. The sensitivity, specificity, positive predictive value, and negative predictive value, and their respective 95% CIs were calculated. No a priori sample size calculation or adjustment for multiple comparisons was performed. A P value of .05 or less was considered statistically significant.

Results

A total of 176 patients were eligible for enrollment during the 11-month study period. Of these, 23 (13%) patients were excluded, 7 for being younger than 18 years or older than 85 years, 6 for having had a recent known D-dimer value, 4 for being recently or currently pregnant, 3 for being morbidly obese, 3 for being critically ill or unable to consent, and 2 for being non-English speaking. Two patients met more than 1 exclusion criteria. Sixteen (9%) patients declined to participate in the study and 3 (2%) patients were missed because of patient volume in the ED. After consent was obtained, data were collected on the remaining 134 (76%) patients. Of these, 61 (46%) were men and the median age was 58 years (interquartile range, 43-72 years). Ninety-nine (74%) were white, 19 (14%) Hispanic, 11 (8%) African American, 2 (2%) Asian or Pacific Islander, and 3 (2%) of another racial or ethnic origin.

Of the 134 patients, 14 were diagnosed with PE on their initial visit and 2 were identified as having VTE (1 deep vein thrombosis and 1 PE) on follow-up, yielding a cumulative prevalence of PE of 12% (16/134). Seventeen (12%) patients received no initial radiographic study other that the initial chest radiograph in the ED despite the study protocol [7]; none of these patients had VTE at the time of follow-up. Eight (6%) patients could not be reached for telephone interview follow-up. All 8 of these patients had an initial radiographic study (CT angiogram or V/Q) that yielded negative results and all but 1 had negative Medical records review for VTE with confirmed active Kaiser Permanente insurance and participation. Thus, only 1 (1%) person was lost to follow-up.

Pretest probability ?	n	(%)	No. of PE (%)	PERC rule met (%)		Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)
Low	60	(45)	1 (2)	13	(22)	100 (25-100)	22 (12-35)	2 (0-11)	100 (75-100)
Moderate	60	(45)	9 (15)	6	(10)	100 (66-100)	12 (4-24)	17 (8-29)	100 (54-100)
High	14	(10)	6 (43)	0	(0)	100 (54-100)	0 (0-37)	43 (18-37)	+
Total	134	(100)	16 (12)	19	(14)	100 (79-100)	16 (10-24)	14 (8-24)	100 (80-100)
PE, pulmonary embolism; CI, confidence interval; PPV, positive predictive value; NPV, negative predictive value. * As defined by Wells’ criteria [8]. + Variable undefined because no denominator data were present.

Age, N50 y	87/134	65	(56-73)	13/16	81	74/118	63	.1
Heart rate, N100 beats/min	40/134	30	(23-39)	6/16	38	34/118	29	.6
Oxygen saturation, b90% ?	24/126	19	(13-27)	6/16	38	18/110	16	.08
Unilateral leg swelling	22/134	16	(10-23)	6/16	38	16/118	14	.03
Hemoptysis	6/134	4	(2-9)	1/16	6	5/118	4	.6
Recent surgery	18/134	13	(9-21)	5/16	31	13/118	11	.04
History of VTE	24/134	18	(12-26)	8/16	50	16/118	14	.002
Oral hormone use	23/129	18	(12-26)	3/16	19	20/113	18	1.0
PE, pulmonary embolism; CI, confidence interval; VTE, venous thromboembolism. * Oxygen saturation cut off adjusted for altitude (5,280 feet above sea level).

Table 1 shows the results of risk stratification of the patient cohort, the prevalence of PE, and the performance of the PERC rule for each pretest probability group (ie, low, moderate, and high pretest probability). Of note, 14% of the total cohort and 22% of the low pretest probability cohort met the PERC rule criteria, and none of them were diagnosed with PE. Given that each pretest probability cohort had a sensitivity of 100%, each respective negative likelihood ratio was zero.

Table 2 Bivatriate analysis of individual pulmonary embolism rule-out criteria (PERC) variables

Variables n % (95% PE

CI) positive

n %

PE

negative

n

P

%

Table 2 shows the individual PERC rule variables and their bivariate analyses. Thirteen total values were missing from the original data set. None of these missing variables impacted the PERC rule stratification, meaning all patients with a missing variable were already categorized as not meeting the PERC rule based on another positive variable.

Discussion

In 2004, Kline and colleagues [6] derived the PERC rule with the intention of defining a group of patients that would not require evaluation for PE beyond history and physical examination alone. In our analysis using a prospectively collected database of patients who were evaluated for PE, we found the PERC rule to be highly sensitive with an excellent negative predictive value both in a cohort of patients at low risk for PE and in a cohort of patients at all risks levels for PE. The specificity of the PERC rule appears to increase as risk of PE in the population decreases.

The PERC rule is the first attempt at defining a lower threshold for patients that require initiation of an evaluation for PE, beyond history and physical examination. In their derivation and concomitant internal validation, Kline et al showed that 25% of their low-risk patients met the PERC rule, and the rule demonstrated a sensitivity and specificity of 96% and 27%, respectively. In their Very low risk group, 15% of patients met the PERC rule, resulting in a sensitivity and specificity of 100% and 15%, respectively [6]. Our findings are consistent with these numbers for the low-risk population, thus suggesting external validation of the PERC

rule. It should be noted that both our study and the original PERC rule study appear to show worsening specificity with increasing probability of disease. Although spectrum bias in this situation cannot be excluded, this suggests that the PERC rule’s negative likelihood ratio decreases as the perceived risk of disease decreases, thereby making it most useful in the very low to low-risk populations [10].

Two publications to date have addressed this issue [11,12]. First, Righini et al [11] authored a Letter to the editor in which they report on 762 patients, and found a False negative rate for the PERC rule of 7%. Their patient population was undifferentiated with respect to the patient’s risk stratification and carried a PE prevalence of 26%, with 12% being PERC rule negative. They argue that this false negative rate significantly diminishes the rule’s clinical use. Conversely, Courtney et al [12] published an abstract showing the PERC rule to have a false negative rate of 2% in a patient cohort with 4% prevalence of PE, stating that the rule may be useful in identifying a subgroup very unlikely to have a PE. Our study differs from these 2 in several ways. First, we consider the performance of the PERC rule with respect to the patient’s risk stratification, allowing for a better understanding of its performance within each group. Second, our study demonstrated a higher sensitivity in the low-risk population and all patients combined, with no cases of PE being missed in patients who met the PERC rule. Finally, our study presents bivariate analysis on individual PERC variables, providing evidence as to which variables con- tribute most to this form of risk stratification for PE.

With respect to each PERC criterion, the greatest predictors of PE were recent surgery, history of VTE, and unilateral leg swelling. This is consistent with the literature that suggests recent surgery and history of VTE are major risk factors for VTE, carrying a relative risk between 5 and 20 [13]. Unilateral leg swelling has also been reported to be highly predictive of PE in patients suspected of the diagnosis with an odds ratio reported to be 3.5 to 5.8 [8,14].

Our study continues to explore which patients do not require medical evaluation beyond a history and physical examination for the diagnosis of PE. The increased

availability of the D-dimer has led, in many instances, to an ease in initiating a diagnostic workup for PE by practitioners. This ease has allowed physicians to lower their threshold, and subsequently increased the total numbers of workups, initiated for PE. At some point, this increased rate of evaluation of patients for PE will be offset by the false positive rate of available Diagnostic tests. Once this happens, we increase our overall use of Health care resources with questionable returns on quality of care.

In the future, if this PERC rule holds up to a large prospective external validation study, in addition to interrater reliability scrutiny, great strides could be made toward more effective and efficient resource use in the evaluation of PE.

There are several limitations in this study. First, this was a secondary analysis of a previously prospectively collected data set. The data set lent itself nicely to performing this external assessment because of the incorporation of all PERC rule variables in the original study design. The total number of patients enrolled in this study, however, was only 134, and the prevalence of PE in the cohort was 12%. Although the point estimates for the sensitivities and negative likelihood ratios of the PERC rule are reassuring, the relatively small sample size contributed to relatively wide CIs, limiting the conclusions that can be drawn from the data. A larger and prospective study should be performed to refine the accuracy of the estimated of the PERC rule. Selection and mis- classification bias may also have occurred in this study, although these forms of bias were minimized because of prospective enrollment of patients, the use of specific inclusion and exclusion criteria, the use of a highly sensitive D-dimer assay [9], Close monitoring of data collection by a designated study coordinator, and close oversight by other study personnel.

Conclusion

The PERC rule may identify a patient population for whom further diagnostic testing is not indicated. Larger prospective validation with interrater reliability testing is needed before widespread implementation of the PERC rule is made.

References

1. Roy PM, Colombet I, Durieux P, et al. Systematic review and meta- analysis of strategies for the diagnosis of suspected pulmonary embolism. BMJ 2005;331:259-68.
2. Wells PS, Anderson DR, Rodger M, et al. Excluding pulmonary embolism at the bedside without diagnostic imaging: management of patients with suspected pulmonary embolism presenting to the emergency department by using a simple clinical model and D- dimer. Ann Intern Med 2001;135:98-107.
3. Runyon MS, Webb WB, Jones AE, et al. Comparison of the unstructured clinician estimate of pretest probability for pulmonary embolism to the Canadian Score and the Charlotte rule: a prospective observational study. Acad Emerg Med 2005;12:587-93.
4. Kabrhel C, Matts C, McNamara M, et al. A highly sensitive ELISA D- dimer increases testing but not diagnosis of pulmonary embolism. Acad Emerg Med 2006;13:519-24.
5. Goldstein NM, Kollef MH, Ward S, et al. The impact of the introduction of a rapid D-dimer assay on diagnostic evaluation of suspected pulmonary embolism. Arch Intern Med 2001;161:567-71.
6. Kline JA, Mitchell AM, Kabrhel C, et al. Clinical criteria to prevent unnecessary diagnostic testing in emergency department patients with suspected pulmonary embolism. J Thromb Haemost 2004;2:1247-55.
7. Wolf SJ, McCubbin TR, Feldhaus KM, et al. Prospective validation of Wells criteria in the evaluation of patients with suspected pulmonary embolism. Ann Emerg Med 2004;44:503-10.
8. Wells PS, Anderson DR, Rodger M, et al. Derivation of a simple clinical model to categorize patient’s probability of pulmonary embolism: increasing the models utility with the SimpliRED D-dimer. Thromb Haemost 2000;83:416-20.
9. Heit JA, Meyers BJ, Plumhoff EA, et al. Operating characteristics of automated latex immunoassay fibrin D-dimer tests in the diagnosis of angiographically-defined acute pulmonary embolism. Thromb Hae- most 2000;83:970.
10. Mulherin SA, Miller WC. Spectrum bias or spectrum effect? Subgroup variation in diagnostic test evaluation. Ann Intern Med 2002; 137:598-602.
11. Righini M, Le Gal G, Perrier A, et al. More on: clinical criteria to prevent unnecessary diagnostic testing in emergency department patients with suspected pulmonary embolism. J Thromb Haemost 2005;3:188-9.
12. Courtney Dm, Pribaz JR, Senh AC, et al. Prospective evaluation of the Pulmonary Embolism Rule-out Criteria (PERC) rule: an 8-variable block rule to identify subjects at very low risk of pulmonary embolism. Acad Emerg Med 2006;13:S157-8 [Abstract].
13. British Thoracic Society Pulmonary Embolism Guideline Develop- ment Group. British Thoracic Society guidelines for the management of suspected acute pulmonary embolism. Thorax 2003;58:463.
14. Kline JA, Nelson RD, Jackson RE, et al. Criteria for the safe use of D-dimer testing in emergency department patients with suspected pulmonary embolism: a multicenter US study. Ann Emerg Med 2002; 39:144-52.