a b s t r a c t

Background: The Pulmonary Embolism Rule-out Criteria is an 8-variable clinical decision rule that identifies patients at low risk for pulmonary embolism (PE) to prevent unnecessary diagnostic testing in the evaluation of suspected PE in the emergency department (ED). The objective of this study was to determine PERC’s safety and diagnostic use in our institution’s ED population.

Methods: We performed a retrospective analysis on consecutive adult patients evaluated with Computed tomographic angiography for suspicion of PE at our ED during the dates January 1, 2011, to December 31, 2011. Patients negative for all 8 PERC criteria (ie, “PERC [-]”) were considered to be at low risk for PE. All data were analyzed using SPSS-20 (SPSS Inc, Chicago, IL) to calculate the variables of interest and their respective 95% confidence intervals (95% CIs).

Results: During the 12-month study period, 729 subjects were evaluated with CTA for suspicion of PE. Ten subjects were excluded because of nondiagnostic imaging studies. After exclusion, 719 subjects were available for analysis. Prevalence of PE was 4.5%. PERC (-) had a sensitivity of 96.9% (95% CI, 84.3%-99.4%), a negative predictive value of 98.8% (95% CI, 93.5%-99.8%), and a negative likelihood ratio of 0.26 (95% CI, 0.04-1.82) when used as an independent diagnostic test to exclude PE. Conclusions: Use of PERC could have safely avoided 11.5% of CTAs, reducing potential Patient harm, health care costs, and unnecessary diagnostic testing. Consistent with prior studies, PERC can be safely used to identify low-risk patients for whom further testing can be deferred.

(C) 2013

Background

Nearly 10 million patients visit the emergency department (ED) annually with complaints of dyspnea, chest pain, or both [1]. Pulmonary embolism (PE) is an important consideration as a

Abbreviations: PERC, pulmonary embolism rule-out criteria; DVT, deep venous thrombosis; PE, pulmonary embolism; CT, computed tomography; CTA, computed tomographic angiography; ED, emergency department; LR, likelihood ratio; NPV, negative predictive value; PPV, positive predictive value; CI, confidence interval.

? Authors’ contributions: A. Bozarth, N. Bajaj, and M. Wessling designed the study,

collected data, participated in the statistical analysis of data, and drafted and revised the manuscript. D. Keffer and S. Jallu collected study data, participated in the design of the study, and critically reviewed and revised the manuscript. G. Salzman provided input to the study design, was the study’s principal investigator, critically reviewed the statistical analysis of data, and critically reviewed and revised the manuscript before submission.

?? Competing interests: The authors have no conflicts of interest or competing

interests to disclose. The authors have no funding source to disclose. The authors have no commercial, financial, or other relationships to disclose related to the subject of the article.

* Corresponding author. School of Medicine, University of Missouri-Kansas City, 2411 Holmes St, Kansas City, MO 64108, USA. Tel.: +1 217 242 1008.

E-mail address: [email protected] (A.L. Bozarth).

life-threatening cause of these complaints that is associated with a mortality rate of up to 30% in untreated cases [2,3]. Accurate diagnosis of PE can be diagnostically challenging, and workup of patients with clinically suspected PE frequently leads to overtesting. Diagnostic evaluation of PE frequently involves use of D-dimer and computed tomographic angiography (CTA). In outpatients, D-dimer testing has been evaluated extensively for the exclusion of PE and can safely be used to exclude PE in patients with a Low pretest probability for the disease [4-6]. Unfortunately, the false-positive rate of D-dimer in this setting can be as high as 40%, leading to a large burden of unnecessary diagnostic testing to exclude PE, most commonly CTA with its associated risks [4].

In practice, CTA has become the standard of care to confirm or exclude the presence of PE; with increasing use, awareness of its disadvantages is being established [7]. Use of CTA in this setting is costly, is time consuming, and carries potential for patient harm such as simple and anaphylactic contrast allergies, contrast-induced acute renal failure, extravasation of contrast into Soft tissues, identification of lesions that require further workup, and radiation exposure. which could potentially increase the risk of malignancy [8-10]. Furthermore, there is emerging evidence that suggests that emergency physicians in the United States have a lower threshold for ordering radiologic

http://dx.doi.org/10.1016/j.ajem.2013.10.010

0735-6757/(C) 2013

studies to exclude PE, simply because of fear of lawsuits resulting in the practice of “defensive medicine” [11,12].

In 2004, Kline et al [13] developed an 8-variable clinical decision rule known as the Pulmonary Embolism Rule-out Criteria in an effort to identify patients at low risk for PE (Box 1), thus supporting the decision to forgo further diagnostic testing (eg, with D-dimer or CTA) in the workup of PE.

Since its development, PERC has been validated in several clinical settings as a highly sensitive Prediction tool for safely ruling out PE in low-risk patients [13-21]. However, PERC’s usefulness has been ques- tioned by 2 studies, both of which cite a PE prevalence of more than 20%. This figure is much higher than that reported in other PERC validation studies, particularly studies performed in the United States [22,23].

A recently published retrospective study suggests that PERC’s sensitivity is clinically adequate in patient populations with a low prevalence of PE, and if PERC had been used in this setting, approximately 25% of CTAs ordered could have been safely avoided [16]. Furthermore, a meta-analysis and systematic review evaluating the diagnostic accuracy of PERC found it to have a pooled sensitivity of 97% (95% confidence interval [CI], 96%-98%) and negative likelihood ratio of 0.17 (95% CI, 0.13-0.23) for ruling out PE in low-risk patients [20]. The meta-analysis found PERC to have an overall rate of missed PEs of less than 1% [20]. A limitation of this meta-analysis was that 6 of the 11 studies analyzed were authored by at least 1 researcher involved in the original derivation of PERC, representing nearly 80% of subjects included in the analysis. Thus, further collection of data in other populations could be clinically useful in determining the validity of PERC.

In an attempt to address concerns regarding the reliability of PERC’s performance, we performed a retrospective cohort study evaluating its use in our institution’s ED population. We sought to determine the prevalence of PE in our population and whether or not PERC can be safely used with a sensitivity, negative predictive value (NPV), and negative LR comparable with that previously reported [20].

Methods

Design

We performed a retrospective cohort study by individual medical record review of all consecutive patients who underwent CTA to investigate for suspicion of PE in our ED from January 1, 2011, to December 31, 2011. All CTAs were performed using a 128-slice multidetector helical CT scanner (SOMATOM Definition AS; Siemens

Box 1

Eight Questions of the PERC [13]

Medical Solutions USA, Inc., Malvern, PA). The study was approved by our institutional review board before its initiation.

Setting and population

The study institution, Truman Medical Center (Hospital Hill location), is a general medical and surgical hospital with 272 inpatient beds providing care to a diverse urban patient population in Kansas City, MI. Survey data show that approximately 68 000 patients visit the hospital’s ED annually. It also serves as a teaching hospital for the University of Missouri-Kansas City School of Medicine and its associated residency programs. Subjects included in the study were consecutive adult outpatients 18 years or older evaluated and treated in our ED with clinical suspicion of PE.

Study protocol

We obtained a consecutive list of all CTAs performed in our ED during the 12-month study period from our institution’s Quality Improvement Department. Consecutive sampling was used to limit interpreter bias. During the 12-month study period, 843 CTAs were ordered: 729 for suspicion of PE and 114 for evaluation of trauma and/ or aortic dissection (Table 1). Our institution uses separate CTA protocols in the evaluation of PE and aortic dissection. Computed tomographic angiographies ordered under the vascular dissection protocol were excluded from statistical analysis. If the investigators determined that the CTA had been ordered for purposes other than the exclusion of PE (eg, aortic dissection or trauma evaluation), it was excluded from our analysis. In addition, to meet the inclusion in the study, the CTA must have been ordered by or under direct supervision of a board-certified emergency medicine physician. After removal of the 114 CTA studies that were not performed for suspicion of PE, 729 studies were available for review.

Using a standardized questionnaire, the investigators (A.B., N.B., M.W., D.K., S.J.) simultaneously reviewed each electronic medical record to obtain the data of interest required to answer all 8 questions of the PERC rule (Box 1). To further limit experimental bias, one of the investigators (A.B.) performed a secondary medical record review to confirm the accuracy of the data collection. Subjects were classified as “PERC-negative (-)” and considered at Very low risk for PE if “No” was answered to all 8 questions in the PERC rule (Table 1). Subjects were classified “PERC positive (+)” if “Yes” was answered to 1 or more of the 8 questions in the PERC rule.

Measurements

Baseline demographic data including the age, sex, and race/ ethnicity of each subject were collected. If multiple sets of vital signs were obtained in the ED, all values were reviewed. If the patient’s age was at least 50 years (at the time of the encounter), the case was considered PERC (+). The case was also considered PERC (+) if any pulse rate recorded was more than 99 beats/min or if any pulse

Table 1

Age N 49 years old?

Pulse N 99 beats/min?

Pulse oximetry b 95% while breathing room air?

Any history of DVT or PE (requiring anticoagulation)?

Any surgery or trauma in the past 4 weeks (requiring general anesthesia or hospitalization, respectively)?

Any unilateral Leg swelling (obtained by documentation from the examiner)?

Any hemoptysis (patient report in the past week)?

Any hormone use (oral contraceptives, estrogen hormone use, other hormone use in male or female)?

Reproduced with permission from Kline et al [13].

Subjects excluded and reasons for exclusion

Reason for exclusion	No. of subjects (%)
Dissection studya	99 (11)
Ordered for trauma evaluation	15 (1.8)
Nondiagnostic CT studyb	10 (1.2)
Total excluded	124

a Ordered for the evaluation of aortic dissection. All of these studies were ordered as

a vascular dissection protocol.

^b Nondiagnostic CT study: as interpreted by the radiologist due to inability to visualize the pulmonary arteries, offering no diagnostic capability for detection of PE (malfunction of CT scanner or inappropriate timing of contrast bolus). All 10 of these studies were ordered as a PE protocol.

oximetry reading was reported more than 95% (while breathing ambient air). Each case was reviewed for documentation of the following variables: hemoptysis, prior diagnosis of venous thrombo- embolism (deep venous thrombosis or PE) requiring therapeutic anticoagulation, presence of unilateral leg swelling on examination, history of recent surgery or trauma in the past 4 weeks (requiring general anesthesia or hospitalization), or exposure to exogenous estrogen. If any of the mentioned variables of interest were present, the case was considered PERC (+).

A subgroup analysis was performed on the PERC (-) cohort to determine if D-dimer testing had been performed before CTA and if the D-dimer result was reported as “positive” or “negative.” Baseline demographic data were also collected in this cohort as described above. This subgroup was also followed up by medical record review for a minimum duration of 6 months after the CTA was performed to determine if any subjects in the PERC (-) cohort had been subsequently diagnosed as having PE after the Initial ED encounter.

Once the data of interest were obtained, the result of each encounter’s respective CTA (ie, presence or absence of PE) was determined. The results of all CTAs were interpreted independently by a radiologist certified by the American Board of Radiology. Each case was reviewed to determine the primary reason for obtaining the CTA. If the CTA was ordered for reasons other than the purpose of excluding PE (eg, aortic dissection and/or trauma evaluation), it was excluded from the data analysis. In addition, if the CTA was determined to be nondiagnostic by the interpreting radiologist (defined as insufficient contrast administration for visualization of the pulmonary vasculature to exclude PE and/or inappropriate timing of contrast administration), it was excluded from the data analysis.

Data management and statistical analysis

All data were entered electronically into a secured spreadsheet electronically (Microsoft Excel; Microsoft Corp, Redmond, WA), accessible only to the investigators, and all data were de-identified before analysis. None of the 8 PERC variables were missing in the final data set. All data were analyzed using SPSS-20 (SPSS Inc, Chicago, IL) to calculate the PERC rule’s sensitivity, specificity, positive predictive values, NPVs, and LRs for excluding PE and their respective 95% CIs. All values were interpreted independent of physician gestalt. An a priori sample size calculation was not performed before data collection.

Results

During the 12-month study period, 729 subjects underwent CTA in our ED for suspicion of PE. Among the 729 CTA studies available for analysis, 10 were excluded based on nondiagnostic CTAs (see definition above), leaving 719 subjects eligible for statistical analysis. The mean (SD) subject age was 47 (13) years (range, 18-87 years), and women comprised 61% of the study population. Sixty-two percent were African American, 32% white, 4% Hispanic, and 2% other (self-identified).

Pulmonary embolism was identified in 4.5% (32/719) of subjects on CTA. Among the 11.5% (83/719) of subjects who were classified as PERC (-), the prevalence of PE was 1.2% (1/83). Treating PERC (-) as an independent diagnostic test for excluding the presence of PE resulted in a sensitivity of 96.9% (95% CI, 84.3%-99.4%), a specificity of 11.9% (95% CI, 9.7%-14.6%), an NPV of 98.8% (95% CI, 93.5%-99.8%), and

Table 3

Diagnostic performance of the PERC

Statistic	Result	95% CI
Sensitivity	96.9%	84.3%-99.4%
Specificity	11.9%	9.7%-14.6%
Positive predictive value	4.9%	4.2%-6.8%
NPV	98.8%	93.5%-99.8%
Positive LR	1.10	1.028-1.178
Negative LR	0.262	0.038-1.821

a negative LR of 0.262 (95% CI, 0.04-1.82). The results of the study are outlined in Tables 2 and 3.

The PERC (-) cohort represented 11.5% (83/719) of subjects and was further analyzed to determine demographic variables (eg, age, sex, race) and D-dimer testing results (if performed). The subgroup was also reviewed to determine if any patients had been diagnosed as having PE at 6 months by medical record review. The mean (SD) age of the PERC (-) cohort was 40 (8) years (range, 19-49 years), and women comprised 67% of the cohort. Eighty-two percent were African American, 16% white, and 2% Hispanic (self-identified). One patient in the PERC (-) cohort was diagnosed as having PE during the initial ED encounter. Of the remaining 82 subjects classified PERC (-) in whom PE was excluded by CTA, 92.8% (76/83) had documented follow-up by medical record review for 6 months. No evidence of PE was documented in any of these subjects’ medical records available for review. Six subjects (7.2%) were lost to follow-up. D-Dimer testing was performed in 55.4% (46/83) of cases in the PERC (-) cohort. D-Dimer testing was reported positive in 78.3% of the 46 cases where it was obtained. D-Dimer testing was not obtained in 44.6% (37/83) of subjects in the low-risk PERC (-) cohort. The results of the PERC (-) cohort analysis are described in Table 4.

Limitations

Inherent limitations of this study include its retrospective design and lack of blinding to the study hypothesis. We chose to study PERC retrospectively to determine its real-time feasibility because some studies have questioned the safety of its use [22,23]. As noted by previously conducted retrospective studies attempting to validate the PERC rule, in addition to the 8 objective variables, PERC requires clinician “gestalt” to determine that a patient is at low risk for PE. We considered all patients in our study classified PERC (-) to be at “low risk” for PE, independent of physician gestalt. When PERC was originally validated, low risk by clinician gestalt was defined as less than 15% chance of PE being present. Similar to the study performed by Dachs et al [16], it was impossible for us to extract this subjective indicator because of our retrospective study design.

Our investigators were not completely blinded to the outcomes of interest; therefore, the possibility of observer bias cannot be excluded. In an effort to limit observer bias, we used a standardized questionnaire during the data collection process. Furthermore, each CTA was interpreted by a radiologist with no knowledge of the PERC results. An independent review of medical records was not feasible in our circumstances because of a lack of the funding. In an effort to limit interpreter bias, we used consecutive sampling with 12 months of data. We did not perform an interrater reliability assessment to evaluate the consistency between medical record reviewers because

Table 2

Two-by-two contingency table with observed results

	CT (+) for PE (n = 32)	CT (-) for PE (n = 687)
PERC (+) (n = 636)	31	605	PPV 4.9% (95% CI, 4.2%-6.8%)
PERC (-) (n = 83)	1 Sensitivity 96.9% (95% CI, 84.3%-99.4%)	82 Specificity 11.9% (95% CI, 9.7%-14.6%)	NPV 98.8% (95% CI, 93.5%-99.8%) Total n = 719

Table 4

PERC-negative cohort analysis

Subjects (n =83), n (%)

6-mo follow-up
Follow-up available?	77 (92.8)
Lost to follow-up	6 (7.2)
D-Dimer testing D-Dimer testing performed	46 (55.4)
Positive D-dimer	36 (78.3)
negative D-dimer	10 (21.7)
No D-dimer testing performed	37 (44.6)

who had D-dimer testing performed, 78% had a falsely positive result. This false-positive rate for D-dimer is much higher than that reported in the literature [4]. We hypothesize that this may be the result of lower test threshold cutoff points at our institution in comparison with historical controls. This further reemphasizes that D-dimer has its limitations when used to rule out PE, even in low-risk patients such as our PERC (-) subgroup.

As mentioned, 1 patient classified as PERC (-) was diagnosed as having a PE during the initial encounter. The patient was a 35-year-old morbidly obese man (body mass index, 45.6 kg/m²) with no medical history. He presented with complaints of pleuritic chest pain located over the right side of his chest. Vital signs were all

differences in the variables of interest are small; however, the possibility of experimental bias cannot be excluded. To further limit experimental bias, one of the reviewers (A.B.) performed a secondary medical record review to confirm the accuracy of the data collection.

Discussion

We hypothesized that PERC could be safely used at our institution, missing less than 2% of PEs, which we found to be true. In our retrospective analysis, we found only 1 patient (1.2%) classified PERC (-) to have a PE. We chose a PE miss rate cutoff of less than 2% based on the false-negative rate of previously conducted prospective studies evaluating patients at high risk for PE with a negative CTA, subsequently found to have a PE on 3- to 6-month follow-up [24- 26]. In addition, pooled results from a meta-analysis [27] involving 15 studies and 3500 patients with suspected PE found that the overall NPV of a negative CTA is 99.1%. This is similar to the NPV of pulmonary angiography, which ranges from 98.4% to 100% [28]. If most ED physicians choose to accept a negative CTA in the workup of PE in high-risk patients with a false-negative rate of approximately 2%, it would be acceptable to use PERC in low-risk patients if its false-negative rate is less than 2%.

We were unsure if our population differed in terms of PE prevalence or baseline demographic characteristics in comparison with prior studies. The prevalence of PE in our cohort was lower (4.5%) in comparison with a recent meta-analysis (median, 10%; range, 1.74%-25.7%), and our baseline demographics differed only in terms of race/ethnicity [20]. Although we cannot exclude the possibility that PERC’s high NPV of 98.8% in our cohort could be partially attributed to its low PE prevalence, it does fall within the range reported by studies included in the meta-analysis [20]. Moreover, the meta-analysis [20] found no significant association between PE prevalence and PERC’s diagnostic performance or on relative Diagnostic odds ratios. Based on posttest probability calcula- tions, we found that PERC could be safely implemented in populations with a pretest probability for PE of less than 7%. In our cohort, application of PERC at the bedside could have safely avoided 11.5% of CTA studies. We must note that the safe use of PERC has only been validated in populations with a low prevalence of PE. The findings of our study provide supporting evidence for PERC’s use in a population with a similar, low prevalence of PE. Therefore, we cannot comment on whether or not PERC can be safely used in populations with a higher prevalence of disease, and this remains an important area for future research.

We performed a subgroup analysis on the PERC (-) cohort.

Medical record review was performed on this cohort for 6 months to determine if any of the subjects had subsequently been diagnosed as having PE. The details of the results are outlined in Table 4. Of the 83 patients who were classified as PERC (-), 1 patient was diagnosed as having PE during the initial ED encounter. None of the remaining 76 PERC (-) patients (76/83) available for follow-up medical record review had been diagnosed as having PE at 6 months from the initial ED encounter. Surprisingly, we found that D-dimer testing was not performed in 44.6% of the PERC (-) cohort before CTA. In patients

within normal limits. He had no family history of venous thrombo- embolism. A D-dimer was not performed. Computed tomographic angiography demonstrated acute PE within the basilar segmental and subsegmental pulmonary arteries of the right lower lobe and a wedge-shaped peripheral airspace opacity within the distribution of the PE in the right lower lobe anterior basilar segment, suspicious for pulmonary infarction. Venous duplex Compression ultrasound of both lower extremities was unremarkable. He was anticoagulated with Intravenous heparin protocol and started on coumadin. A workup for inherited causes of thrombophilia was unremarkable.

We chose not to exclude CTAs that were nondiagnostic for subsegmental PE for a number of reasons. First, a major limitation of CTA for diagnosis of PE is its ability to detect these smaller subsegmental PEs, and even pulmonary angiography has limited Interobserver agreement for subsegmental PE, with reported ranges of 45% to 66% [29,30]. In addition, the clinical importance of subsegmental emboli has not been completely established, and some experts [31] have suggested that the diagnosis of subsegmental PE on CTA in patients “without physiologic compromise” may not be beneficial; they are almost nearly always nonfatal [32], and the potential harms of anticoagulation may outweigh the benefit of anticoagulation in this setting. We hypothesize that most PERC (-) patients with a PE are likely to fall into this low-risk category, and excluding this specific cohort (subsegmental PE with no benefit from therapeutic anticoagulation) will furthermore improve the overall NPV of this easy to use clinical prediction rule.

Conclusion

Consistent with previously described literature, we found that the use of the PERC maintains its high sensitivity and NPV to rule out PE in patients at low risk for PE. Our findings are consistent with a recently published meta-analysis evaluating the use of PERC. Application of the PERC in our ED could have safely resulted in 11.5% fewer CTAs, minimizing potential patient harm, unnecessary health care costs, and additional laboratory testing. Our data suggest that PERC can be safely used to identify low-risk patients for whom further testing can be deferred.

Acknowledgments

We would like to thank Emily Wessling, PharmD, for her assistance with the editorial process and John Foxworth, PharmD, for his assistance with statistical analysis.

References

1. McCaig LF, Burt CW (2003) National hospital ambulatory medical care survey: 2001 emergency department summary. US Department of Health and Human Services, Centers for Disease Control and Prevention, National Cancer for Health Statistics
2. Giuntini C, Di Ricco G, Marini C, et al. Pulmonary embolism: epidemiology. Chest 1995;107:3S-9S.
3. Manganelli D, Palla A, Donnamaria V, Giuntini C. Clinical features of pulmonary embolism. Doubts and certainties. Chest 1995;107:25S-32S.
4. Stein PD, Hull RD, Patel KC, et al. D-Dimer for the exclusion of acute venous thrombosis and pulmonary embolism: a systematic review. Ann Intern Med 2004;140:589-602.
5. Dunn KL, Wolf JP, Dorfman DM, et al. Normal D-dimer levels in emergency department patients suspected of acute pulmonary embolism. J Am Coll Cardiol 2002;40:1475-8.
6. Brown MD, Rowe BH, Reeves MJ, et al. The accuracy of the enzyme-linked immunosorbent assay D-dimer test in the diagnosis of pulmonary embolism: a meta-analysis. Ann Emerg Med 2002;40:133-44.
7. Bettmann MA, Baginski SG, White RD, et al. ACR Appropriateness Criteria(R) acute chest pain-suspected pulmonary embolism. J Thorac Imaging 2012;27:W28-31. http://dx.doi.org/10.1097/RTI.0b013e31823efeb6.
8. Brenner DJ, Hall EJ. Computed tomography-an increasing source of radiation exposure. N Engl J Med 2007;357:2277-84. http://dx.doi.org/10.1056/NEJMra072149.
9. Smith-Bindman RLJ. radiation dose associated with common computed tomogra- phy examinations and the associated lifetime attributable risk of cancer. Arch Intern Med 2009;169:2078-86. http://dx.doi.org/10.1001/archinternmed.2009.427.
10. Mitchell AM, Kline JA. Contrast nephropathy following computed tomography angiography of the chest for pulmonary embolism in the emergency department. J Thromb Haemost 2007;5:50-4. http://dx.doi.org/10.1111/j.1538-7836.2006.02251.x.
11. Katz DA, Williams GC, Brown RL, et al. Emergency physicians’ fear of malpractice in evaluating patients with possible Acute cardiac ischemia. Ann Emerg Med 2005;46:525-33. http://dx.doi.org/10.1016/j.annemergmed.2005.04.016.
12. Studdert DM, Mello MM, Sage WM, et al. Defensive medicine among high-risk specialist physicians in a volatile malpractice environment. JAMA 2005;293: 2609-17. http://dx.doi.org/10.1001/jama.293.21.2609.
13. Kline JA, Mitchell AM, Kabrhel C, et al. Clinical criteria to prevent unnecessary diagnostic testing in emergency department patients with suspected pulmonary embolism. J Thromb Haemost 2004;2:1247-55.
14. Beam D, Brewer K, Kline JA. EMF-5: application of the pulmonary embolism rule-out criteria in a rural population. Ann Emerg Med 2007;50:S132. http:

    //dx.doi.org/10.1016/j.annemergmed.2007.06.323.

    Courtney D, Pribaz J, Senh A. Prospective evaluation of the pulmonary embolism rule-out criteria (PERC) rule: an 8-variable block rule to identify subjects at very low risk of pulmonary embolism. Acad Emerg Med 2006;13:S157-8.

15. Dachs RJ, Kulkarni D, Higgins GL. The pulmonary embolism rule-out criteria rule in a community hospital ED: a retrospective study of its potential utility. Am J Emerg Med 2011;29:1023-7.
16. Hogg K, Dawson D, Kline J. Application of pulmonary embolism rule-out criteria to the UK Manchester Investigation of Pulmonary Embolism Diagnosis (MIOPED) study cohort. J Thromb Haemost 2005;3:592-3.
17. Kline JA, Courtney DM, Kabrhel C, et al. Prospective multicenter evaluation of the pulmonary embolism rule-out criteria. J Thromb Haemost 2008;6:772-80. http://dx.doi.org/10.1111/j.1538-7836.2008.02944.x.
18. Kline JA, Peterson CE, Steuerwald MT. Prospective evaluation of real-time use of the pulmonary embolism rule-out criteria in an academic emergency department. Acad Emerg Med 2010;17:1016-9. http://dx.doi.org/10.1111/j.1553-2712.2010.00826.x.
19. Singh B, Parsaik AK, Agarwal D, et al. Diagnostic accuracy of pulmonary embolism rule-out criteria: a systematic review and meta-analysis. Ann Emerg Med 59:517- 520.e1-4. http://dx.doi.org/10.1016/j.annemergmed.2011.10.022.
20. Wolf SJ, McCubbin TR, Nordenholz KE, et al. Assessment of the pulmonary embolism rule-out criteria rule for evaluation of suspected pulmonary embolism in the emergency department. Am J Emerg Med 2008;26:181-5.
21. Righini M, Le Gal G, Perrier A, Bounameaux H. More on: clinical criteria to prevent unnecessary diagnostic testing in emergency department patients with suspected pulmonary embolism. J Thromb Haemost 2005;3:188-9.
22. Hugli O, Righini M, Le Gal G, et al. The pulmonary embolism rule-out criteria (PERC) rule does not safely exclude pulmonary embolism. J Thromb Haemost 2011;9:300-4.
23. Van Strijen MJL, De Monye W, Kieft GJ, et al. Accuracy of single-detector spiral CT in the diagnosis of pulmonary embolism: a prospective Multicenter cohort study of consecutive patients with abnormal perfusion scintigraphy. J Thromb Haemost 2005;3:17-25. http://dx.doi.org/10.1111/j.1538-7836.2004.01064.x.
24. Musset D, Parent F, Meyer G, et al. Diagnostic strategy for patients with suspected pulmonary embolism: a prospective multicentre outcome study. Lancet 2002;360:1914-20. http://dx.doi.org/10.1016/S0140-6736(02)11914-3.
25. Van Belle A, Buller H, Huisman M, et al. Effectiveness of managing suspected pulmonary embolism using an algorithm combining clinical probability, D-dimer testing, and computed tomography. JAMA 2006;295:172-9. http://dx.doi.org/ 10.1001/jama.295.2.172.
26. Quiroz R, Kucher N, Zou KH, et al. clinical validity of a negative computed tomography scan in patients with suspected pulmonary embolism: a systematic review. JAMA 2005;293:2012-7. http://dx.doi.org/10.1001/jama.293.16.2012.
27. Van Beek EJ, Reekers JA, Batchelor DA, et al. Feasibility, safety and clinical utility of angiography in patients with suspected pulmonary embolism. Eur Radiol 1996;6: 415-9.
28. Stein PD, Henry JW, Gottschalk A. Reassessment of pulmonary angiography for the diagnosis of pulmonary embolism: relation of interpreter agreement to the order of the involved pulmonary arterial branch. Radiology 1999;210: 689-91.
29. Diffin DC, Leyendecker JR, Johnson SP, et al. Effect of anatomic distribution of pulmonary emboli on interobserver agreement in the interpretation of pulmonary angiography. AJR Am J Roentgenol 1998;171:1085-9.
30. Newman DH, Schriger DL. Rethinking testing for pulmonary embolism: less is more. Ann Emerg Med 2011;57:622.
31. Donze J, Le Gal G, Fine MJ, et al. Prospective validation of the Pulmonary Embolism Severity Index. A clinical prognostic model for pulmonary embolism. J Thromb Haemost 2008;100:943-8.