a b s t r a c t

Objectives: To determine the impact of a non-restrictive clinical decision rule on CT utilization for Emergency De- partment patients suspected of having an acute aortic syndrome .

Methods: We prospectively assessed the performance of a previously described, collaboratively designed, non-re- strictive clinical decision rule for AAS. Emergency Department patients with suspected AAS were stratified into low and high-risk groups based on decision rule results, from July 2013-August 2014. Patients with acute trauma, prior AAS or aortic surgery were excluded. CT dose reduction protocols were concurrently implemented as a quality improvement measure. Bivariate analysis was performed to compare the prospective cohort with the his- torical derivation cohort for CT utilization rates, results of CT, AAS incidence and radiation exposure. The perfor- mance of the clinical decision rule was evaluated.

Results: Compared with the historic cohort, the study cohort demonstrated a lower CT utilization rate [0.344% (427/124,093) versus 0.477% (1465/306,961), (p b 0.001)], a trend toward higher CT diagnostic yield [4.4% (19/427) versus 2.7% (40/1465), (p = 0.08)]. AAS incidence was similar [0.015% (19/124,093) versus 0.013% (40/306,961), (p = 0.57)]. The mean effective radiation dose was markedly lower [12 +- 5.5 mSv versus 43

+- 20 mSv, (p b 0.0001)]. The clinical decision rule correctly stratified only 56% (10/18) of patients with AAS as high-risk. Conclusions: A non-restrictive, collaboratively designed, clinical decision rule for Emergency Department patients with suspected AAS performed poorly in risk-stratifying patients for AAS. However, its implementation was associated with a significant and safe decrease in CT utilization.

(C) 2017

Introduction

Acute aortic dissection is a devastating event [1]. The more general term acute aortic syndrome (AAS) refers to a group of diagnoses with similar clinical presentations including aortic dissection, intramural hematoma, ruptured aneurysm and penetrating atherosclerotic ulcer. Because of its high associated morbidity and mortality, it is important to rapidly identify patients with AAS in order to expedite their treat- ment. Computed tomography (CT) is the primary diagnostic imaging modality for suspected AAS because of its availability, accuracy and speed [2,3]. Reliance on CT, however, has prompted broad concern

* Corresponding author at: Department of Radiology, Montefiore Medical Center, 111 East 210 Street, Bronx, NY, United States.

E-mail address: lharamati@gmail.com (L.B. Haramati).

¹ Current address: Department of Radiology, Westchester Medical Center, 100 Woods Road, Valhalla, NY 10595.

² Current address: Department of Emergency Medicine, Long Island Jewish Medical Center, New Hyde Park NY 11040.

related to increases in radiation exposure to the population from medi- cal imaging [4-6]. This concern is compounded by the low positivity rate of CT scans performed for suspected AAS.

Recent publications have shown promise in incorporating clinical and laboratory parameters into Order entry interfaces as an effective way to improve CT utilization and yield. These efforts have largely relied upon clinical decision support (CDS) to present validated and evidence- based information to guide testing. Implementation of CDS to practice is expected to expand, in line with the Health information technology and Economic Health (HITECH) Act of 2009, which aims to encourage inte- gration of health information technology into clinical decision-making. CDS implementation has been associated with improved appropriate- ness and reduction in imaging use for several applications [7-12].

AAS remains a significant clinical diagnostic challenge upon initial presentation. The minority (15-43%) of cases are suspected on initial evaluation and up to 30% remain undiagnosed ante-mortem [3,13-15]. Conversely, multiple diagnoses including acute coronary syndromes, pleurisy, anxiety, musculoskeletal and acute gastrointestinal disorders

http://dx.doi.org/10.1016/j.ajem.2017.04.014

0735-6757/(C) 2017

have clinical presentations that overlap with AAS. Many patients under- go CT with negative results. Validated structured clinical criteria and de- cision rules have been developed to guide imaging for patients with headache, trauma and suspected pulmonary embolism while safely obviating the need for imaging low-risk patients [16-19]. Unfortunately, a validated clinical decision rule for AAS remains elusive. Our group retrospectively studied clinical and radiographic findings for a cohort of Emergency Department patients with suspected AAS and developed a preliminary clinical decision rule that required prospective testing (Fig. 1) [20].

The purpose of the present study is to determine the impact of non- restrictively implementing this preliminary clinical decision rule on the CT utilization rate and CT yield in ED patients suspected of having an AAS and to prospectively test its performance.

Methods

Study population

This HIPAA compliant study was approved by the institutional review board with a waiver of informed consent. The study population included all adults in our urban multihospital academic healthcare system from July 26, 2013 through August 30, 2014 who underwent CT for suspected AAS at the two Emergency Department sites that were included in the historical clinical decision rule derivation cohort. Exclusion criteria were age b 18 years, history of acute Thoracic trauma, prior AAS and prior aortic surgery, the same as in the decision rule derivation cohort, to facilitate comparison of the study and historical cohorts.

Decision rule implementation

The Radiology and Emergency Departments collaboratively devel- oped and implemented a new diagnostic CT protocol with a unique study name (CT Aortic Dissection), which permitted easier physician or- dering, exam protocolling and tracking for quality metrics. Audits were performed to assure clinical practice compliance with utilization of this study type for suspected AAS by searching our picture archiving and communication system (PACS) and our radiology information system (RIS). The computer provider order entry (CPOE) for this new examina- tion was specifically designed to prospectively test the clinical decision rule non-restrictively. It required answers to a brief questionnaire, joint- ly designed by the Radiology and Emergency Department staff, prior to submission of the CT order while clearly stating that the answers to the questions would not permit or restrict access to imaging. The question- naire (Fig. 2) asked whether the patient experienced chest pain, if the chest pain was still present or had resolved at the time of order entry, and whether the pain was abrupt or gradual onset.

CT dose reduction

We previously documented high radiation exposure for patients un- dergoing dissection protocol CT scans [21] and as a quality improvement

metric, we implemented several CT dose reduction strategies concurrent with introduction of the clinical decision rule. This included limiting the scan length of the pre-contrast phase to the chest, use of 100 kVp tube voltage, and decreases in the maximum allowed tube current in concert with application of adaptive statistical iterative reconstruction.

Key outcome measures

All Emergency Department patient visits for the study time periods were identified using Looking Glass(TM) Clinical Analytics (Streamline Health, Atlanta, Georgia) (LG-CA), a software application developed to evaluate health care quality, effectiveness and efficiency using clinical and administrative data sets [22]. We recorded the age, gender and race for the study and historical cohorts. Cases were identified by utiliz- ing various electronic databases, some specifically designed for this audit.

Chest radiograph reports were manually reviewed for the presence or absence of abnormal aortic or mediastinal contours. CT reports were manually reviewed for the presence or absence of AAS.

Radiation dose for each CT exam was extracted from the archived dose report, available in PACS for review on every examination. The dose length product (DLP) was converted to effective dose using a con-

version factor of 18 uSv/mGy* cm [23]. These values were compared to historical data from the derivation cohort [21].

The prospective clinical data points acquired through the question- naire were reviewed and patients were stratified as high or low risk for AAS based on our previously reported model [20]. The model strati- fied patients as low risk if their chest pain had resolved or if their chest radiograph was normal and their pain was not described as abrupt onset in nature.

Data analysis

Categorical proportional analysis was performed using ?² or Fisher exact test to compare the historical and study cohorts and Student’s t-test. Performance of the clinical decision rule was evaluated for test parameters using the CT findings as the diagnostic gold standard. Statistical analysis was performed in R using the HMISC library soft- ware. Binomial confidence intervals were calculated with R using binconf function.

Results

During the study period 124,093 unique patients visited our ED and 427 patients underwent a CT scan for suspected AAS. The study and his- torical cohorts were majority women and reflect the multiethnic popu- lation served by our inner-city academic medical center (Table 1).

CT utilization and results

The CT utilization rate was 0.344% (427/124,093), significantly lower than the CT utilization rate of 0.477% (1465/306,961) for the historical

Fig. 1. Schematic flow chart demonstrating stratification of high and low risk patients based on clinical and radiographic parameters.

Fig. 2. Screenshot of computer provider order entry (CPOE) with questionnaire form used to prospectively accrue clinical stratification data at time of order entry. Note the clearly stated non-restrictive nature of data accrual.

cohort (p b 0.0001). This represents a relative reduction of CT usage of 28% (95%CI 20%-35%) and an absolute reduction of 0.133% for the entire ED population. The number of ED visits per one omitted CT is 751 (Table 2).

The incidence of AAS in the study cohort was 0.015% (19/124,093), as compared to 0.013% (40/306,961) in the historical cohort (p = 0.57). The AAS specific diagnostic yield of CT was 4.4% (19/427) for the study cohort versus 2.7% (40/1465) for the historical cohort (p = 0.08).

Performance of the preliminary decision rule

Complete prospective data from the preliminary decision rule ques- tionnaire were available for 84% (359/427) of patients who underwent CT for AAS. Forty-six percent (165/359) of these patients were stratified as low-risk for AAS and 54% (194/359) as high-risk for AAS. The prelim- inary decision rule only stratified 56% (10/18) of patients with AAS as high risk; 44% (8/18) of those with AAS were stratified as low risk. The preliminary decision rule performance characteristics were a sensitivity of 0.55 (95%CI 0.31-0.78), specificity of 0.45 (95%CI 0.41-0.51), positive

Table 1

Demographic characteristics of the study and historical cohorts.

ED visits Study 124,093 Historical 306,961

predictive value 0.05 (95%CI 0.025-0.93) and negative predictive value

0.95 (95%CI 0.91-0.98). Of note, among the 8 patients erroneously strat- ified as low-risk (false negatives), 6 patients reported no chest pain at all and 2 reported chest pain that resolved at the time of imaging. Among patients with incomplete decision rule data, the CTs were negative for AAS in 98.5% (67/68).

Radiation dose

The effective dose of CT scans performed during the study period was 12 +- 5.5 mSv, significantly lower than the dose of 43 +- 20 mSv in the historical derivation cohort (p b 0.0001). This corresponds to a mean relative reduction of 72% and a mean absolute reduction of 31 mSv. Given that 427 patients were scanned for AAS during the study period, CT conferred approximately 5124 mSv (12 x 427) on the total Emergency Department population of 124,093 patients (0.041 mSv/patient). In contrast, the 1465 patients that were scanned

Table 2

CT data for study and historical cohorts.

Study Historical p value

ED visits 124, 093 306,961

CT for AAS Number of exams 427 1465

Exam rate 0.0034 0.0048 b 0.0001?

Age (years +- SD) Female (%)	47 +- 11 77,710 (62.6)	45 +- 10 188,298 (61.3)	+ AAS on CT	Cases positive rate	19 0.044	40 0.027	0.08?
Race (%)			AAS incidence		0.0013	0.0015	0.57?
White	16,337 (13.2)	50,550 (16.4)	CT dose (mSv)	Per CT	12 +- 5.5	43 +- 20	b 0.0001??
Black	39,030 (31.5)	103,204 (33.6)		Per ED patient	0.041	0.21

Hispanic 59,914 (48.3) 132,615 (43.2)

Note. Numbers in parenthesis indicate percentages.

* p values calculated using Fisher’s exact test.

?? p values calculated using Student’s t-test.

for AAS during the historical period received approximately 62,995 mSv (43 x 1465) for the total Emergency Department population of 306,961 patients (0.21 mSv/patient). Therefore, accounting for both the protocol revisions and the reduced CT utilization in the Emergency Department population, there was a mean relative reduction of radiation exposure of 80% and an absolute reduction of 0.17 mSv/patient. This corresponds to avoidance of a 10 mSv radiation exposure in 1 of 59 Emergency De- partment visits.

Discussion

The present prospective study of Emergency Department patients undergoing CT for suspected AAS demonstrated poor performance of a non-restrictive preliminary clinical decision rule derived from a histor- ical cohort at our institution [20], stratifying only 56% of patients who had an AAS as high risk. The incidence of AAS remained stable between the two study periods, which confirms the similarity of the historical derivation and prospective study cohorts to allow for confident compar- ison. We also demonstrated a 28% decrease in CT utilization and a trend toward a higher diagnostic yield of CT compared with the historical study period, although overall yield remained low.

Since the decision rule was non-restrictive and access to CT was not dependent on the answers to its questions, the best explanation for de- crease in CT utilization is a change in clinical practice. However, the mechanism for the change in practice is unclear. Collaborative discus- sions and our prior research likely resulted in heightened awareness of appropriate CT use, as did the thoughtfulness required to answer the simple questionnaire. The stable incidence of AAS supports the safe- ty of this change in practice; however, a true miss rate remains unknowable.

Better targeted utilization and lower dose protocols together result- ed in a 72% decrease in radiation exposure for our Emergency Depart- ment patients with an absolute reduction of 0.17 mSv per patient for the entire Emergency Department population (the equivalent of almost two frontal and lateral chest radiographs for every patient). The goal of optimizing CT protocols for AAS must balance retaining the necessary very high sensitivity for this life-threating diagnosis against the reality of exposing a large population without AAS to the risks and costs of ra- diation and contrast without concomitant benefit. In the present study, the large majority of radiation reduction was accomplished by attention to technical improvements including routine use of lower tube voltage, use of new iterative reconstruction software as well as limiting the scope of multiphasic imaging, which has recently become increasingly scrutinized both by physicians and laypeople [24]. Ongoing radiation re- duction strategies for CT remain important in caring for this difficult population. We estimate that about 10% of the population-wide reduc- tion in radiation exposure is attributable to the ultimate radiation re- duction modification, not undergoing imaging at all.

The still low positivity rate of CT imaging for AAS suggests that fur- ther practice improvement is achievable by improved patient selection. Clinical decision support and more intelligent computerized order entry are hailed as transformative methods for improvement of health care delivery and outcomes, including better imaging utilization [7-9, 25-27]. Unfortunately, AAS seems less amenable than other diseases like pulmonary embolism, for risk stratification, and there remains no adequate method of triaging patients with suspected AAS for imaging [27].

While the need for a validated decision rule remains high, the pre- liminary rule we designed and prospectively tested demonstrated unac- ceptably poor sensitivity and must be rejected. In retrospect, the derivation method for this rule suffered from two deficiencies that are more important than we initially realized. The first was the use of retro- spective chart review, with its inherent insensitivity to patients without classical clinical symptomatology, in this case the presence of chest pain. The largest group of false negatives was patients without any chest pain at all. A significant proportion of AAS patients present with back pain,

abdominal pain or findings of end-organ ischemia (e.g., stroke or cold extremity). Second, none of the 19 patients in our original study with complete resolution of their pain had AAS. Since a variable with perfect discrimination defeats logistic models, these patients were removed prior to building our original risk stratification model and were classi- fied as low risk. Subsequent modeling revealed that a finding of zero of 19 could be expected to be seen 2.5% of the time even if the true risk was 13.4% for AAS, an unacceptably high attack rate with a conven- tional notion of not rare enough to be improbable [28]. Two of 38 pa- tients in the prospective study cohort whose chest pain resolved completely had AAS on CT. Although this is statistically consistent our prior results, we have demonstrated that resolved pain is an unreliable indicator of low risk for AAS. This highlights the importance of prospec- tive testing and validation of clinical decision rules prior to clinical implementation.

Limitations of the present study include the fact that, although data

were entered prospectively, reducing patient chest pain descriptors to simple check-boxes limits the nuanced way these clinical parameters are observed. Despite explicitly stating that the questions did not re- strict access to CT, the introduction of this data collection at the point of care order reminds clinicians of the signs and symptoms of AAS and most likely impacted Ordering patterns. Additionally, complete prospec- tive data were only available in 85% of the study population; this limita- tion is mitigated by fact that 98.5% of the CTs with incomplete data were negative. The relatively short study duration of slightly more than one year is another limitation. Perhaps the observed effect will decrease with time as physicians become more accustomed to the order entry in- cluding the decision rule. The majority ethnic minority population of our medical center limits the generalizability of these results but focuses on an understudied population that merits explicit attention.

Conclusion

A collaboratively designed, clinical decision rule for ED patients with suspected AAS performed poorly in risk-stratifying patients for AAS. However, its implementation was associated with a significant and safe decrease in CT utilization.

Acknowledgements

Funding: No extramural funding.

Meeting: Presented, in part, at the Radiological Society of North America annual meeting 2014.

Conflicts of interest: Linda B Haramati’s spouse is a board member of Kryon. None of the other authors have conflicts of interest to report.

References

1. Olsson C, Thelin S, Stahle E, Ekbom A, Granath F. Thoracic aortic aneurysm and dis- section: increasing prevalence and improved outcomes reported in a nationwide population-based study of more than 14,000 cases from 1987 to 2000. Circulation 2006;114(24):2611-8.
2. Hayter RG, Rhea JT, Small A, Tafazoli FS, Novelline RA. Suspected aortic dissection and other aortic disorders: multi-detector row CT in 373 cases in the emergency set- ting. Radiology 2006;238(3):841-52.
3. Sullivan PR, Wolfson AB, Lecky RD, Burke JL. Diagnosis of acute thoracic aortic dissec- tion in the emergency setting. Am J Emerg Med 2000;18(1):46-50.
4. Fazel R, Krumholtz HM, Wang Y, Ross JS, Chen J, Ting HH, et al. Exposure to low-dose ionizing radiation from medical imaging procedures. N Engl J Med 2009;361(9): 849-57.
5. Hurwitz LM, Reiman RE, Yoshizumi TT, Goodman PC, Toncheva G, Nguyen G, et al. Radiation dose from contemporary cardiothoracic multidetector CT protocols with an anthropomorphic female phantom: implications for cancer induction. Radiology 2007;245(3):742-50.
6. Smith-Bindman R, Lipson J, Marcus R, Kim KP, Mahesh M, Gould R, et al. Radiation dose associated with common computed tomography examinations and the associ- ated lifetime attributable risk of cancer. Arch Intern Med 2009;169(22):2078-86.
7. Vartanians VM, Sistrom CL, Weilburg JB, Rosenthal DI, Thrall JH. Increasing the ap- propriateness of outpatient imaging: effects of a barrier to ordering low-yield exam- inations. Radiology 2010;255(3):842-9.
8. Blackmore CC, Mecklenburg RS, Kaplan GS. Effectiveness of clinical decision support in controlling inappropriate imaging. J Am Coll Radiol 2011;8(1):19-25.
9. Raja AS, Ip IK, Prevedello LM, Sodickson AD, Farkas C, Zane RD, et al. Effect of com- puterized clinical decision support on the use and yield of CT pulmonary angiogra- phy in the emergency department. Radiology 2012;262(2):468-74.
10. Wasser EJ, Prevedello LM, Sodickson A, Mar W, Khorasani R. Impact of a real-time computerized duplicate alert system on the utilization of computed tomography. JAMA Intern Med 2013;173(11):1024-6.
11. Ip IK, Schneider L, Seltzer S, Smith A, Dudley J, Menard A, et al. Impact of provider- led, technology-enabled radiology management program on imaging. Am J Med 2013;126(8):687-92.
12. Shinagare AB, Ip IK, Abbett SK, Hanson R, Seltzer SE, Khorasani R. Inpatient imaging utilization: trends of the last decade. AJR Am J Roentgenol 2014;202(3):W277-83.
13. Bansal RC, Chandrasekaran K, Ayala K, Smith DC. Frequency and explanation of false negative diagnosis of aortic dissection by aortography and transesophageal echocar- diography. J Am Coll Cardiol 1995;25(6):1393-401.
14. Klompas M. Does this patient have an acute thoracic aortic dissection? JAMA 2002; 287(17):2262-72.
15. Von Kodolitsch Y, Schwartz AG, Neinber CA. Clinical prediction of acute aortic dissec- tion. Arch Intern Med 2000;160(19):2977-82.
16. Drescher FS, Chandrika S, Weir ID, Weintraub JT, Berman L, Lee R, et al. Effectiveness and acceptability of a computerized decision support system using modified Wells criteria for evaluation of suspected pulmonary embolism. Ann Emerg Med 2011; 57(6):613-21.
17. Wells PS, Ginsberg JS, Anderson DR, Kearon C, Gent M, Turpie AG, et al. Use of a clin- ical model for safe management of patients with suspected pulmonary embolism. Ann Intern Med 1998;129(12):997-1005.
18. Reinus WR, Erickson KK, Wippold 2nd FJ. Unenhanced emergency cranial CT: opti- mizing patient selection with univariate and multivariate analyses. Radiology 1993;186(3):763-8.
19. Blackmore CC. Clinical prediction rules in trauma imaging: who, how, and why? Ra-

    diology 2005;235(2):371-4.

    Lovy AJ, Bellin E, Levsky JM, Esses D, Haramati LB. Preliminary development of a clin- ical decision rule for Acute aortic syndromes. Am J Emerg Med 2013;31:1546-50.

20. Lovy AJ, Rosenblum JK, Levsky JM, Godelman A, Zalta B, Jain VR, et al. Acute aortic syndromes: a second look at dual phase CT. AJR Am J Roentgenol 2013;200:805-11.
21. Bellin E, Fletcher DD, Geberer N, Islam S, Srivastava N. Democratizing information creation from health care data for quality improvement, research and education — the Montefiore Medical Center Experience. Acad Med 2010;85(8):1362-8.
22. Huda W, Ogden KM, Khorasani MR. Converting dose-length product to effective

    dose at CT. Radiology 2008;248:995-1003.

    Rawat U, Cohen SL, Levsky JM, Haramati LB. ACR white paper based comprehensive dose reduction initiative is associated with reversal of upward trend in radiation dose for chest CT. J Am Coll Radiol 2015;12:1251-6.

23. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical Decision support systems on Medication Safety: a systematic review. Arch In- tern Med 2003 Jun 23;163(12):1409-16.
24. Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, Sam J, et al. Effects of computerized clinical decision support systems on practitioner perfor- mance and patient outcomes: a systematic review. JAMA 2005 Mar 9;293(10): 1223-38.
25. American College of Emergency Physicians Clinical Policies Subcommittee (Writing Committee) on Thoracic Aortic Dissection, Diercks DB, Promes SB, Schuur JD, Shah K, Valente JH, et al. Clinical policy: critical issues in the evaluation and management of adult patients with suspected acute nontraumatic thoracic aortic dissection. Ann Emerg Med 2015 Jan;6(1):32-42.e12.
26. Bellin E. Riddles in accountable healthcare: a primer to develop analytic intuition for medical homes and population health. South Carolina: Create Space; 2015.