a b s t r a c t

Background: It is believed that patients who return to the Emergency Department (ED) and require admission are thought to represent failures in diagnosis, treatment or discharge planning. Screening readmission rates or patients who return within 72 h have been used in ED Quality Assurance efforts. These metrics require significant effort in chart review and only rarely identify care deviations.

Objective: This study was conducted to evaluate the yield of reviewing ED return visits that resulted in an ICU admission. This study was conducted to evaluate the yield of reviewing ED return visits that resulted in an ICU admission. We planned to assess if the return visits with ICU admission were associated with deviations in care, and secondarily, to understand the common causes of error in this group.

Methods: Retrospective review of patients presenting to a university affiliated ED between January 1, 2005 and December 31, 2015 and returned within 14 days requiring ICU admission.

Results: From 1,106,606 ED visits, 511 patients returned within 14 days and were admitted to an ICU. 223 patients returned for a reason related to the index visit (43.6%). Of these related returns, 31 (13.9%) had a deviation in care on the index visit. When a standard diagnostic process of care framework was applied to these 31 cases, 47.3% represented failures in the initial diagnostic pathway.

Conclusion: Reviewing 14-day returns leading to ICU admission, while an uncommon event, has a higher yield in the understanding of quality issues involving diagnostic as well as systems errors.

(C) 2019

Introduction

Insight into the quality and efficiency of hospital care has become an expectation, with government and private payers increasingly tying reimbursement to performance. Despite this, it has been challenging to identify high yield parameters of quality in Emergency Department (ED) care; hence the existence of only a small subset of publicly reported measures reflecting the Quality of ED care.

Readmissions, a proxy measure thought to represent opportunities

for Improved care on the index visit, has been widely adopted in other subspecialties [1]. As such, a similar metric was derived in emergency care - looking at the volume of unscheduled returns to the ED that result in a hospital admission. Despite the initial broad adoption of this metric [2,3], it has increasingly come under criticism, with several recent studies suggesting that patients that return to the ED with admission often do not represent deviations in care on their index

* Corresponding author at: Zero Emerson Place Suite 3B, Boston, MA 02114, United States of America.

E-mail address: [email protected] (P. Borczuk).

visit [4-8] and instead these patients are often admitted simply because they returned.

Since looking at all unscheduled returns with admission has a relatively low yield to identify deviations in care, we hypothesized that looking specifically at patients with an unscheduled ED return visit resulting in an ICU admission may represent an event with a higher incidence of low quality care on the index visit. Only a small number of studies have looked at this patient population; for example, risk factors for return visits and ICU admission were described in a Taiwanese ED [9]. To our knowledge, no study to date has investigated the yield of this measure, or the types of errors most commonly found in this patient population.

Our primary objective was to determine the rate of return visits with ICU admission that that had deviations in care, and secondarily, to understand the common causes of error in this group.

Materials and methods

This was a retrospective cohort study performed at a single, urban, tertiary care center with an annual ED volume of over 110,000 visits. A query of the Emergency Department Information

https://doi.org/10.1016/j.ajem.2019.158430 0735-6757/(C) 2019

E. Aaronson et al. / American Journal of Emergency Medicine 38 (2020) 1584-1587 1585

System was performed for all patients who presented to the ED between January 1, 2005 to December 31, 2015 where an initial visit resulted in discharge and a second ED visit within 14 days resulted in an ICU admission. A month at the start and the end of the dataset were set aside so no patients prior to January 1, 2005 and after November 2015 would have a missed return. Our initial screen was based on a 30 day return, but as 80% of returns and readmits to the ICU occurred within 14 days, this time interval was chosen to be best ensure a relationship between the initial ED visit and the ICU admission.

Visits were classified as ‘related’ or ‘unrelated’ based on the discharge diagnosis by two separate Attending emergency physicians based on chief complaint, discharge diagnosis and ICD 9,10 coding. For example, if an index visit was for cellulitis, and a return was for pneumonia, these were classified as ‘unrelated.’ However, if an index visit was for ‘cellulitis’ and the return visit was for ‘diarrhea’ these were classified as related, given concern for causation related to Clostridium Difficile. Similarly, if the index visit was for chest pain, and the return visit was for pulmonary embolus, these were classified as ‘related.’ The group of ‘related’ visits served as the cohort of patients that underwent full chart

review. Kappa values were measured initially on a subset of patients prior to expanding classification of the initial 511 patients. The	Infectious Cardiac Abdominal	7 4 3	22.6% 12.9% 9.7%
group of ‘related’ visits served as the cohort of patients that	Withdrawal/intoxication	1	3.2%
underwent full chart review.	Total	24	77.4%
During full case review, each case was independently (blinded) reviewed by two resident Emergency Physicians (PGY 2,3) who	No deviation n = 223
traced the care, as documented, through a defined care process	Infection	41	21.4%

Table 1

Demographic and administrative data on patients with deviations in care vs. those without deviations in care.

Variable	Patients with significant deviation	Patients with no significant deviation	P value
	N= 31	N = 223
Age (mean years)	46.7	47.9	0.78
Sex (male)	19	115	0.52
English language	27	170	0.50
LOS visit one (mean hours)	5.3	7.2	0.09
Race (Caucasian)	22	141	0.46
Time difference	95.7	129.9	0.04

between visits (h)

Table 2

Deviations based on etiologies (top 5).

Deviation n = 31

Neurologic 9 29%

adapted from the IOM and our malpractice carrier, CRICO Strategies [10,11]. The elements in this process form Table 3 in the manuscript. Based on this table, the residents each determined if there was a deviation of care and, if so, where in the care process the deviation took place. Each case in the final deviation group was reviewed by an Emergency attending physician who work in Quality and Patient Safety. These two physicians also adjudicated any difference in classification by the two resident reviewers.

Kappa scores were calculated between the resident reviewers and all statistics performed in IBM SPSS 21 (Armonk, NY).

Results

There were 1,106,606 patient visits during the 10-year study period, during which 511 patients returned within 14 days and were admitted to an ICU. Of the patients that returned, 223 of them returned for a reason related to the index visit (43.6%). Of these related returns, 31 (13.7%) were deemed to have had a deviation in care on the index visit. Table 1 reveals that there was no differences between the care deviation group and the care appropriate group with respect to examined administrative data, except that the time between visits in the deviation group was shorter.

The most common organ systems represented in the group with deviations in care were neurologic, infectious, cardiac, abdominal and venous thromboembolism. Asthma and intoxication were common reasons to return with ICU admission without identified deviations in care. Table 2 compares the top 5 organ system classification in each group.

Fig. 1 summarizes how deviations were coded and demonstrates the overlap in the 3 major framework domains of initial diagnostic progress (IDP), testing and results processing (TRP) and follow up planning (Table 3). The largest sources of deviation were the result of errors in the initial diagnostic process. There is a clear relationship between not considering a diagnosis and not ordering a test, which results in overlaps in coding between IDP and TRP.

Asthma/COPD	26	13.5%
Neurologic Withdrawal/intoxication	24 19	12.5% 9.9%

Abdominal 18 9.4%

Total 128 66.7%

Fig. 1. 31 cases and deviation of care per framework coding. (Each case may have more than one code per Venn circle. For example, a case may have an IDP5 as well as an IDP6 code. However, for this chart such a scenario is coded as 1 IDP unit.)

The initial screen to assess whether visit one and visit two were related had a kappa of 1.0 based on a 50-patient sample (two ED attendings). The interrater kappa between the first and second resident reviewer in assessing the classification of deviation of care over the entire sample was 0.88.

1586 E. Aaronson et al. / American Journal of Emergency Medicine 38 (2020) 1584-1587

Table 3

Framework utilized to code clinical deviations.^a

Category Description N (%) Total = 63 deviations

Initial diagnostic pathway (IDP)

35 (55.6)

IDP1: Problem noted; care sought Access and scheduling issues 2 (3.2)

IDP2: History and physical conducted Missing or inadequate 2 (3.2) IDP3: Patient assessed, and symptoms evaluated Patient’s complaints are not thoroughly addressed 3 (4.8) IDP4: Ongoing monitoring of clinical status Re-evaluate patient 2 (3.2)

IDP5: Differential diagnosis established Differential too narrow, too reliant on prior diagnosis or no

differential done

11 (17.5)

Testing and results processing (TPR)

IDP6: Diagnostic tests ordered Testing is impeded/inadequate due to incomplete assessments 14 (22.2) 13(20.6)

TRP1: Tests performed Ordered tests are not done, or done incorrectly 8 (12.7)

TRP2: Tests interpreted Reports of tests are incomplete or inaccurate 6 (9.5)

Follow up and coordination

TRP3: Tests results transmitted/received by ordering physician

Problems in receiving tests or timely review of tests 0

15 (23.8)

(FU)

FU1: Physician follows up with the patient Lack of follow up testing, or communication of findings with the patient

7 (11.1)

FU2: Referrals/Consults are appropriately ordered Failure to make or manage specialist referrals 4 (6.3)

FU3: Patient information that is communicated among care team

Failure by provider to communicate information that affects the work up

4 (6.3)

FU4: Patient and providers establish a follow up plan Lack of adherence to treatment and follow up plan 0

^a Many cases had more than one code within each main coding group (i.e. within IDP) as well as code in several groups (i.e. IDP and TRP).

Discussion

In this retrospective study, we looked at 10 years of ED visits that resulted in an ICU admission and found that although rare, return visits resulting in ICU admission represent a high yield metric in the analysis of deviations of care. The types of errors that we found most commonly were related to the initial diagnostic process.

Although ED returns with hospital admission were initially thought to represent a group with higher morbidity and mortality [12,13], it turns out that a review of these cases has in fact very low yield for identifying error [4-7,14,15]. Most recently, a retrospective analysis of over 9 million ED patients concluded that hospital admissions associated with return visits is likely not a sound proxy for deficits in the quality of care at the index visit [16].

As a trigger tool, 72 hour returns with hospital admission has been demonstrated to be a high resource, low yield activity to identify deviations in care [4-7,14]. In a prior report of our 72 hour returns, representing a subpopulation of this papers cohort (3 year period from 2012 to 2015) [4] we had 50 deviation events in 2001 patient screened. If we compare this 72 hour strategy to the ICU readmit strategy, there is a 242% increase in yield for deviation (31 deviation events in 511 patients screened). This is substantially higher than other trigger tools which have been reported in the literature [2,5,17]. If this metric was chosen as a routine quality metric for a monthly Quality or M&M meeting, it would require an initial screening of 7 cases per month to determine if the two visits were related, and then a deeper dive of 3 cases per month to assess for deviation in care.

The cases that were identified in this cohort as having deviations were most frequently related to the diagnostic process, most often related to establishing an appropriate differential diagnosis. Of note, nearly a quarter of all failures were related to testing and results processing. In looking more closely at these cases however, it does not appear that these were failures to order the appropriate test for a diagnosis being considered, but rather a failure to consider the diagnosis and as a result, the test was not ordered.

Leading national patient safety organizations have recently increased their focus on the area of diagnostic error, recognizing that this is likely a failure mode that is under-reported through proactive safety reporting systems [11]. As such, it is often difficult for Healthcare organizations to identify incidences where Diagnostic errors have occurred. Given that the most common type of error identified through our review was related to the initial diagnostic process, looking at ICU

admissions 14 days after ED discharge as part of an electronic health record reporting task for a safety meeting or morbidity and mortality conference can uncover diagnostic errors (versus errors in testing or follow-up).

Although infectious processes, cardiac disease and neurological disease were common etiologies in the cases in which a deviation in care was noted, asthma and intoxication were noted to be etiologies in which there was often no deviation noted. This may represent an opportunity to narrow the scope of the trigger for case review to exclude cases of Intoxicated patients or those with asthma, given that these were often deemed to represent recurrence of their disease or progression of their illness in a way that could not have been anticipated on the index visit.

Limitations

Our study had several limitations. As a single center study, the findings may not be generalizable, especially to smaller non-academic centers. Related to this, the lack of any standardized guidelines for which patients warrant ICU admission limit the generalizability to other institutions. Given the long length of stay at our institution, it is also possible that there were patients who were admitted to the ICU, stayed in the ED long enough awaiting an unavailable ICU bed who stabilized and then were down-graded in care. These patients would not have been captured in our study sample. We also did not capture patients in the sample who were admitted to a general medical unit and subsequently were transferred to an ICU. We were only able to capture patients who returned to our institution, and as such, we would have missed patients who returned to another hospital were admitted to their ICU. Although the protocol used to match visit one and visit two was liberal, there is a chance, without full case review, that some cases could have been erroneously coded as ‘unrelated’ when in fact they were related.

Conclusion

We are unlikely to find an ideal measure to assess quality of care in the ED given the heterogeneity of the population we serve. However, the development of a panel of measurements should include metrics that can be easily screened as offer the best yield to find a remediable deviation. Such a measure should include an assessment of patients

E. Aaronson et al. / American Journal of Emergency Medicine 38 (2020) 1584-1587 1587

who are discharged from the ED who return within 14 days who require ICU level of care.

Declaration of competing interest

None.

References

1. US Centers for Medicare & Medicaid Services. Readmissions reduct progr https:// www.cms.gov/medicaremedicare-fee-for-service-payment/acuteinpatientpps/ readmissions-reduction-program.html Accessed January. 2016.
2. Lindsay P, Schull M, Bronskill S, Anderson G. The development of indicators to measure the quality of clinical care in emergency departments following a modified-Delphi approach. Acad Emerg Med. 9(11):1131-1139.
3. American College of Emergency Physicians. Appendix B: ACEP CEDR QCDR measure information. https://www.acep.org/globalassets/sites/acep/media/cedr/cedr-qcdr- measure-specifications2019.pdf. [Accessed 30 August 2019].
4. Aaronson E, Borczuk P, Benzer T, Mort E, Temin E. 72h returns: a trigger tool for diagnostic error. Am J Emerg Med. 2017. https://doi.org/10.1016/j.ajem.2017.08. 019.
5. Rising KL, Wiebe DJ, Hollander JE, Carr BG. Return visits to the emergency department: the time-to-return curve. Acad Emerg Med Aug. . 2015;21:864-71 (8 SRC-BaiduScholar FG-1).
6. Rising KL, Padrez K a., O’Brien M, Hollander JE, Carr BG, Shea J a. Return visits to the emergency department: the patient perspective. Ann Emerg Med 2014;65(4): 377-386.e3. doi:https://doi.org/10.1016/j.annemergmed.2014.07.015.
7. Pham JC, Kirsch TD, Hill PM, Deruggerio K, Hoffmann B. Seventy-two-hour returns may not be a good indicator of safety in the emergency department: a national study. Acad Emerg Med. 2011;18(4):390-7. https://doi.org/10.1111/j.1553-2712. 2011.01042.x.
8. Nunez S, Hexdall A, Aguirre-Jaime A. Unscheduled returns to the emergency department: an outcome of Medical errors? Qual Saf Health Care. 15(2): 102-108.
9. Tsai I, Sun CK, Chang CS, Lee KH, Liang CY. Characteristics and outcomes of patients with emergency department revisits within 72 hours and subsequent admission to the intensive care unit. Tzu Chi Med J. 2016;28(4):151-6.
10. CRICO Strategies. 2014 annual benchmarking report, malpractice risks in the diagnostic process Boston, MA ; 2014.
11. Balogh E, Miller B, Ball J, National Academies of Sciences, Engineering, and Medicine, Institute of Medicine, Board on health care services, Committee on Diagnostic Error in Health Care, editors. Improving diagnosis in health care. Washington DC: The National Academic PRess; 2015. https://doi.org/10.17226/21794.
12. Pierce JM, Kellerman AL, Oster C. “Bounces”: an analysis of short-term return visits to a public hospital emergency department. Ann Emerg Med. 19(7):752-757.
13. Hu SC. Analysis of patient revisits to the emergency department. Am J Emerg Med. 10(4):366-370.
14. Easter JS, Bachur R. Physicians’ assessment of pediatric returns to the Emergency Department. J Emerg Med. 44(3):682-688.
15. Abualenain J, Frohna WJ, Smith M, et al. The prevalence of quality issues and adverse outcomes among 72-hour return admissions in the emergency department. J Emerg Med. 2013;45(2):281-8. https://doi.org/10.1016/j.jemermed.2012.11.012.
16. Sabbatini AK, Kocher KE, Basu A, Hsia RY. In-hospital outcomes and costs among patients hospitalized during a return visit to the emergency department. Jama. 2016;315(7):663-71. https://doi.org/10.1001/jama.2016.0649.
17. Calder L, Pozgay ARS, et al. Adverse events in patients with return emergency department visits. BMJ Qual Saf. 2015;24:142-8.