a b s t r a c t

Background: The incidence of errors and adverse events in emergency medicine is poorly characterized. Objective: The objective was to systematically determine the rates and types of errors and adverse events in an academic, tertiary care emergency department (ED).

Methods: Prospective data were collected on all patients presenting to a tertiary-care academic medical center ED with an annual census of 55,000 patients between January 2009 and November 2012. Cases of patients meeting predetermined criteria were systematically identified by an electronic medical record system. Criteria for review included patients who (1) returned to the ED within 72 hours and were admitted on their second visit, (2) were admitted from the ED to the floor and then transferred to the intensive care unit (ICU) within 24 hours, (3) ex- pired within 24 hours of ED arrival, (4) required airway management, or (5) were referred to the QA committee as the result of complaints. Cases were randomly assigned to individual physicians not involved with the care. All cases were reviewed using a structured electronic tool that assessed the occurrence of error and adverse events. Institutional review board jurisdiction was waived by the Beth Israel Deaconess Medical Center IRB.

Results: During the study period, 152,214 cases were screened and 2131 cases (1.4%) met prespecified criteria for review. The incidence of error in these cases was 9.5% (95% confidence interval [CI], 8.3%-10.8%), representing an overall incidence of 0.13% among all ED patients. In cases that involved error, 50.5% occurred among patients who returned to the ED within 72 hours; 17.3% occurred among floor-to-ICU transfers; 5.4% occurred among mortality cases; 2.0% occurred among airway cases; and 24.8% occurred among cases referred as the result of complaints. The incidence of adverse events in the reviewed cohort was 8.3% (CI, 7.2%-9.6%), representing an overall incidence of 0.11% among all ED patients. In cases that involved adverse events, 48.6% occurred among patients who returned to the ED within 72 hours; 16.4% occurred among floor-to-ICU transfers; 9.0% occurred among mortality cases; 1.1% occurred among airway cases; and 24.9% occurred among cases referred as the result of complaints.

Conclusion: Although the overall incidence of error and adverse events in EDs is low, the likelihood of such events is markedly increased among patients who return to the ED within 72 hours, among patients who require floor-to-ICU transfer within 24 hours, and among those whose cases come to attention as the result of complaints.

(C) 2015

Introduction

Errors and adverse events have been recognized as a major problem in medicine since the 1990s [1,2]. Despite the Institute of Medicine’s 2000 call to action with the publication of To Err Is Human and subse- quent works, the burden of Medical errors remains significant [3,4]. Re- cent data suggest that many adverse events still go unreported and the incidence of adverse events attributable to error may be increasing [5,6]. Robust data on the incidence of error and adverse events are needed to mitigate this problem, yet the evidence base remains poor, especially

? Funding: none.

?? The findings of this study were presented as an abstract at the 2013 Annual Meeting of the Society for Academic Emergency Medicine (SAEM)

* Corresponding author at: Department of Emergency Medicine, Beth Israel Deaconess Medical Center, West, Clinical Center 2, One Deaconess Rd, Boston, MA, 02215. Tel.: +1 303 900 3055.

E-mail address: [email protected] (R.S. Klasco).

¹ These authors contributed equally to this work.

in the field of emergency medicine. A 2013 systematic review by Stang et al attests to the dearth of high-quality data on the incidence of errors and adverse events in the emergency department (ED) [7]. In a search of 11,624 citations, they were only able to identify10 relevant articles, 8 of which were observational in design, all of which were of low to moder- ate methodological quality.

The objective of the present study was to systematically determine the rates of errors and adverse events among patients presenting to an academic, tertiary care ED.

Methods

Study design, goals, and oversight

This was a prospective cohort study of all patients presenting to a tertiary care academic ED (annual census of 55,000) between January 2009 and November 2012.

http://dx.doi.org/10.1016/j.ajem.2015.08.042 0735-6757/(C) 2015

R.S. Klasco et al. / American Journal of Emergency Medicine 33 (2015) 1786-1789 1787

The study’s goals were to determine the overall incidence of error and to determine the incidence of error within the following

Table 1

Characteristics of cases reviewed

tation was adequate; (4) resource utilization was appropriate; (5) pro-

prespecified categories of patients: (1) 72-hour returns to the ED, ICD-9 code description		ICD-9-CM	Cases	% of total
(2) floor-to-ICU transfers within 24 hours of hospital admission, Abdominal pain unspecified site		789.00	356	13.3%
(3) deaths within 24 hours of hospital admission, and (4) patients iden- Shortness of breath		786.05	128	4.8%
tified by complaints. Oversight was provided by the ED quality assur- Nausea and vomiting		787.0	106	4.0%
ance (QA) committee, which is integrated into the hospital’s overall Unspecified chest pain		786.5	104	3.9%
QA operations through formal processes and procedures as illustrated Unspecified accidental fall		E888.9	103	3.9%
in the Figure. Fever, unspecified		780.6	101	3.8%
Unspecified intracranial hemorrhage		432.9	96	3.6%
Pain in limb		785.1	80	3.0%
2.2. Selection of participants Cellulitis and abscess		528.3	79	3.0%
Altered mental status		780.97	78	2.9%
All patients presenting to the ED within the study period were eligi- Lumbago		742.2	78	2.9%
Cardiac arrest		427.5	76	2.8%
ble for inclusion. With the exception of cases identified by patient or Other malaise and fatigue		719.4	73	2.7%
complaints, all cases were identified systematically according to the Headache		784.0	68	2.5%
prespecified criteria noted above by an electronic QA dashboard that Hemorrhage of gastrointestinal tract unspecified		578.9	51	1.9%
interfaced with a commercially available health information system Coma		780.01	48	1.8%
(HIS) system [8]. For the cases that originated by a complaint, senior Acute alcoholic intoxication in alcoholism		303.00	46	1.7%
unspecified drinking…
leadership made a subjective decision about whether or not to forward Epilepsy, unspecified		345.9	40	1.5%
it for a formal QA review. Dizziness and giddiness		788.1	39	1.5%
Acute respiratory failure		518.81	35	1.3%
2.3. Data collection and processing Swelling of limb		729.81	35	1.3%
Acute but ill-defined cerebrovascular disease		436	31	1.2%
Acute pharyngitis		462	27	1.0%
Two physician-reviewers who were not involved in the care of the Laboratory examination		V72.6	25	0.9%
study patients reviewed each case independently. Each case was scored Hematuria, unspecified		599.70	23	0.9%
according to an 8-point Likert scale to determine whether: (1) errors Syncope and collapse		780.2	20	0.7%
were made by the ED team; (2) adverse events occurred; (3) documen- suicidal ideation		V62.84	18	0.7%
Accidental poisoning by drugs, medicinal		E850-E858	17	0.6%
cedures were performed competently; (6) medical judgment of the ED	Cervicalgia	723.1	17	0.6%
team was adequate; and (7) care was coordinated appropriately. The types of errors identified were confined to the above categories. Provi-	Motor vehicle traffic accident involving collision with other vehicle Cough	E814.0 786.2	17 15	0.6% 0.6%

substances, …

sion was made for free-text comments by the reviewers. A QA commit- tee consisting of physicians, nurses, hospital QA representation, and ancillary staff adjudicated each case in a manner consistent with our previous work [9]. Confidence intervals (CIs) were generated using the CONFIDENCE function of Microsoft Excel 2010.

Aortic aneurysm and dissection	441	14	0.5%
Diarrhea	787.91	14	0.5%
Allergy, unspecified	995.3	13	0.5%
Epistaxis	784.7	13	0.5%
Other abnormal glucose	790.29	13	0.5%
Pain in joint	719.4	13	0.5%
Hemiplegia and hemiparesis	342	12	0.4%
Hypotension unspecified	458.9	12	0.4%

Results

Characteristics of cases

To facilitate analysis, cases were normalized to International Classifi- cation of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes

[10,11]. The characteristics of the most common types of cases (ie, the 39 ICD-9 code descriptions that accounted for 80% of all cases) appear in Table 1. Among these, abdominal pain was the most common

Figure. Key: adverse events (AE), Beth Israel Deaconess Medical Center (BIDMC), Board of Directors (BOD), Centers for Medicare & Medicaid Services (CMS), Controlled Risk Insurance Company (CRICO), ED (ED), emergency medicine (EM), Health care quality (HCQ), Patient Care Assessment and Quality Committee (PCAC), Quality Improvement (QI), Risk Management Foundation (RMF).

1788 R.S. Klasco et al. / American Journal of Emergency Medicine 33 (2015) 1786-1789

condition. A short list of 12 conditions (ie, abdominal pain, shortness of breath, nausea and vomiting, chest pain, fall, fever, intracranial hemor- rhage, arm or leg pain, skin infections, altered mental status, back pain, and cardiac arrest) accounted for half of all cases.

Incidence of errors and adverse events

During the study period, 152,214 patients were screened, of which 2131 (1.4%) met predefined criteria for review (Table 1). Of these 2131 patients, 202 (9.5%; 95% CI, 8.3%-10.8%) (0.13%) were identified as having been exposed to errors in their care, and 177 (8.3%; 95% CI, 7.2%-9.6%) (0.11%) suffered adverse events. Thirty-two cases were iden- tified as having coincident errors and adverse events. This number is likely to be artificially low, however, as limitations in data acquisition precluded a comprehensive assessment (Table 2).

72-Hour returns

Among 1003 patients who returned to the ED within 72 hours and were admitted on the second visit, 102 (10.2%; 95% CI, 8.5%-12.2%) were identified as having been exposed to errors in their care. This con- stituted 50.5% of all errors. Eighty-six of these patients (8.6%; 95% CI, 7.0%-10.5%) sustained adverse events, which accounted for 48.6% of all adverse events.

Floor-to-ICU transfers

Among 342 patients who required transfer to the ICU within 24 hours of hospital admission, 35 (10.2%; 95% CI, 7.5%-13.9%) were identified as having been exposed to errors in their care. This constitut- ed 17.3% of all errors. Twenty-nine of these patients (8.5%; 95% CI, 6.0%- 12.0%) sustained adverse events, which accounted for 16.4% of all adverse events.

Death within 24 hours of admission

Among 443 patients who died within 24 hours of hospital admis- sion, 11 (2.5%; 95% CI, 1.4%-4.4%) were identified as having been ex- posed to errors in their care. This accounted for 5.4% of all errors. Sixteen of these patients (3.6%; 95% CI, 2.2%-5.8%) sustained adverse events, which accounted for 9.0% of all adverse events.

Airway cases

Data were available on 122 patients who required airway manage- ment in the ED, of which 4 (3.3%; 95% CI, 1.3%-8.1%) were identified as having been exposed to errors in their care. This accounted for 2.0% of all errors. Two patients (1.6%; 95% CI, 0.5%-5.8%) sustained adverse events, which accounted for 1.1% of all adverse events.

Complaints

Among 221 patients referred to the QA committee as a result of com- plaints, 50 (22.6%; 95% CI, 17.6%-28.6%) were identified as having been exposed to errors in their care. This accounted for 24.8% of all errors.

Forty-four of these patients (19.9%; 95% CI, 15.2%-25.7%) sustained ad- verse events, which accounted for 24.9% of all adverse events.

Coincident errors and adverse events

In 32 cases, there were coincident errors and adverse events. The phrase coincident errors and adverse events refers to errors and adverse events that occurred in the same case. The actual number of such cases, however, is certain to be larger, as technical limitations in data ac- quisition for this subset resulted in incomplete reporting. Data on cases with coincident error and adverse events were only available for the 22- month period from December 2010 through November 2012. Although this limitation makes it impossible to comment comprehensively on the nature of such cases, available data are classified in Table 3. Not all errors were associated with adverse events; hence, there are differences be- tween the rates of errors and adverse events. The authors sought to as- sociate errors with adverse events during the medical record Review process under the confidentiality requirements of the QA process.

Discussion

The overall rates of error (0.13%) and adverse events (0.11%) identi- fied in our study were exceedingly low. It should be assumed that these results were a function of our highly specific screening criteria rather than a reflection of an unusually error-free environment. Nevertheless, similar rates have been reported in a methodologic study of Medication errors across 36 hospitals and skilled-nursing facilities in which errors were identified by medical record reviewers and incident report reviews [11].

Although there are few studies specific to emergency medicine to serve as comparators, our findings of an incidence of errors of 9.5% and an incidence of adverse events of 8.3% within the study cohort com- port with the body of evidence in general medicine. A systematic review of In-hospital adverse events found an incidence of 9.2% [12]. The 1991 Harvard Medical Practice Study demonstrated an incidence of adverse events of 3.7% among hospitalized patients [1]. Using a retrospective medical record review methodology, Baker et al [13] found an overall Adverse event rate of 7.5% among a small group of community hospitals; Rothschild et al [14] found an overall rate of adverse events of 20.2% among more acutely ill patients in the ICU using a prospective medical record review methodology; and a prospect of cohort study found ad- verse events related to drugs to be common in primary care [15].

It is difficult to draw broad inferences from these studies, however, in view of significant interstudy differences in definitions and disparate methodologies. We used an 8-point Likert scale in the assessment of er- rors and adverse events, which differs from the methodology used by others. Interstudy comparisons of subgroups of patients are further hampered by small numbers that increase the likelihood of type I and type II errors.

Various solutions have been proposed to remedy this problem. Weingart et al [16] have studied a voluntary physician-based reporting system, although voluntary systems have predictable, inherent weak- nesses. Pollack et al [17] called for increased collaboration between emergency physicians and hospitalists, but other investigators note a lack of enthusiasm on the part of physicians that would tend to vitiate

Table 2

Rates of error and adverse events by category

	Cases	Errors	Error rate	95% CI for error rate	Percentage of total errors	Adverse events	Adverse event rate	95% CI for adverse event rate	Percentage of total adverse events
Total	2131	202	9.5%	0.083-0.108	100.0%	177	8.3%	0.072-0.096	100.0%
72-h returns	1003	102	10.2%	0.085-0.122	50.5%	86	8.6%	0.070-0.105	48.6%
Floor-to-ICU transfers	342	35	10.2%	0.075-0.139	17.3%	29	8.5%	0.060-0.119	16.4%
Mortality cases	443	11	2.5%	0.014-0.044	5.4%	16	3.6%	0.022-0.058	9.0%
Airway cases	122	4	3.3%	0.013-0.081	2.0%	2	1.6%	0.005-0.058	1.1%
Complaints	221	50	22.6%	0.176-0.286	24.8%	44	19.9%	0.152-0.257	24.9%

R.S. Klasco et al. / American Journal of Emergency Medicine 33 (2015) 1786-1789 1789

Table 3

Errors associated with adverse events

Error type Conditions

Care coordination Abdominal aortic aneurysm, anaphylaxis, tachyarrhythmia

diagnostic errors Pancreatitis, cellulitis, pelvic fracture, carbon dioxide narcosis, epidural hematoma, Acetaminophen toxicity, pneumonia, transient ischemic attack, drug toxicity, cerebral venous sinus thrombosis, Pericardial tamponade, Bowel obstruction, pancreatitis

Management errors Urosepsis, depression, ureterolithiasis with

pyelonephritis, ventricular tachycardia, postexposure prophylaxis, airway management

Table 3 describes the 3 types of errors encountered (ie, care coordination, diagnostic er- rors, and management errors) and the 22 conditions in which these errors occurred. Al- though inferences may be drawn about each of these adverse events, case-specific details were excluded to conform to the confidentiality requirements of the QA process.

such efforts [18]. Still, others posit that the incidence of Medical errors has been artificially inflated and question whether current definitions and methodologies can provide reliable estimates for comparisons [19,20].

Our approach, therefore, has several strengths. The first lies in its use of objective definitions of potential error (eg, floor-to-ICU transfers, 72- hour returns). Minimizing the inherent subjectivity in the definition of errors can allow for more valid comparisons and inferences to be drawn. A second strength is its use of electronic medical records and programmatic technologies to identify potential cases. Similar electron- ic health record (EHR)-facilitated surveillance methodology has shown promise in identifying errors in the Primary care setting [21]. Manual medical record review may miss some cases of error or adverse event and, even if perfect, is not scalable in a manner that could provide an op- erating mechanism for enterprise-wide quality assurance. A third bene- fit of our approach is its integration into the hospital’s QA system. The formalized involvement of various services, disciplines, and levels of leadership serves a powerful checks-and-balances function and pro- vides a mechanism for continuous quality improvement. The selection of a study cohort by prespecified criteria applied systematically, as has been done in the present study, has several advantages. This technique minimizes the subjectivity inherent in manual medical record reviews. It also allows for scalability across a large population of patients and, perhaps most importantly, facilitates validation by independent investi- gators and identification of trends by ongoing monitoring.

Limitations

Our study has several limitations. First, interrater or intrarater reli- ability was assessed informally. Although future efforts are planned to standardize this process, informal assessment and the iterative nature of our committee process suggest good agreement.

A second limitation was sample size. Although more than 150,000 records were screened programmatically, the absolute numbers of cases meeting our narrowly defined criteria were small. Expansion of our criteria should result in greater capture of errors and adverse events in the future.

The lack of complete data to identify the set of cases in which error and adverse events intersect was an important limitation. Improve- ments in data capture will eliminate this problem going forward. It is also anticipated that an integrated feedback loop can be developed that will incorporate the gleanings of this process into real-time clinical decision support. Such a mechanism could interdict errors and adverse events before they have the potential to reach the patient.

It is also possible that our study overstated error rates by sampling from subpopulations that were perceived to be at increased likelihood for errors and adverse events, such as 72-hour returns and complaints. Al- though 4 high-likelihood populations were surveilled, it is likely that other

high-likelihood groups have yet to be identified. Furthermore, as a single- institution study, this investigation may have limited generalizability.

Finally, the normalization of cases to a Standardized taxonomy (ie, ICD-9-CM) required reductive logic, as patients frequently presented with multisystem problems and active comorbidities. The normalization process also entailed subjectivity in the assignment of cases to various cat- egories. Nevertheless, the findings presented in Table 1 comport with those of others, lending support to the validity of the process [22,23].

Conclusions

Despite widespread recognition of the problem, medical errors re- main a significant concern and are often associated with adverse events. Our study provides much-needed baseline information on the rates and types of errors in an academic ED. It also demonstrates the feasibility of using information technology to enhance the reproducibility and scal- ability of the QA process.

References

1. Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 1991;324(6):370-6.
2. Leape LL, Brennan TA, Laird N, et al. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med 1991;324(6):377-84.
3. Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: building a safer health system [Internet]. The National Academies Press; 2000 [Available from: http://www.nap.edu/openbook.php?record_id=9728].
4. Crossing the quality chasm: a new health system for the 21st century [Internet]. The National Academies Press; 2001[Available from: http://www.nap.edu/openbook. php?record_id=10027].
5. Department of Health and Human Services, Office of Inspector General. Hospital in- cident reporting systems do not capture most Patient harm [Internet]. Available from: http://oig.hhs.gov/oei/reports/oei-06-09-00091.pdf; 2012.
6. Public Health Council presentation on serious reportable events in Massachusetts hospitals-August 2014. Serious reportable event (SREs) [Internet]. [cited 2014 Sep 9];Available from: http://www.mass.gov/eohhs/gov/departments/dph/programs/ hcq/serious-reportable-event-sres.html.
7. Stang AS, Wingert AS, Hartling L, Plint AC. Adverse events related to emergency de- partment care: a systematic review. PLoS One 2013;8(9):e74214.
8. Handel DA, Wears RL, Nathanson LA, Pines JM. Using information technology to im- prove the quality and safety of emergency care. Acad Emerg Med 2011;18(6):e45-51.
9. Handler JA, Gillam M, Sanders AB, Klasco R. Defining, identifying, and measuring error in emergency medicine. Acad Emerg Med 2000;7(11):1183-8.
10. Centers for Medicare & Medicaid Services. ICD-9 code lookup [Internet]. [cited 2014 Sep 29];Available from: http://www.cms.gov/medicare-coverage-database/ staticpages/icd-9-code-lookup.aspx.
11. Flynn EA, Barker KN, Pepper GA, Bates DW, Mikeal RL. Comparison of methods for detecting medication errors in 36 hospitals and skilled-nursing facilities. Am J Health Syst Pharm 2002;59(5):436-46.
12. De Vries EN, Ramrattan MA, Smorenburg SM, Gouma DJ, Boermeester MA. The inci- dence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care 2008;17(3):216-23.
13. Baker GR, Norton PG, Flintoft V, et al. The Canadian Adverse Events Study: the inci- dence of adverse events among hospital patients in Canada. CMAJ 2004;170(11): 1678-86.
14. Rothschild JM, Landrigan CP, Cronin JW, et al. The Critical Care Safety Study: the in- cidence and nature of adverse events and serious medical errors in intensive care*. Crit Care Med 2005;33(8):1694-700.
15. Gandhi TK, Weingart SN, Borus J, et al. adverse drug events in ambulatory care. N Engl J Med 2003;348(16):1556-64.
16. Weingart SN, Callanan LD, Ship AN, Aronson MD. A physician-based voluntary reporting system for adverse events and medical errors. J Gen Intern Med 2001; 16(12):809-14.
17. Pollack CV, Amin A, Talan DA. Emergency medicine and hospital medicine: a call for collaboration. Am J Med 2012;125(8):826.e1-6.
18. Blendon RJ, DesRoches CM, Brodie M, et al. Views of practicing physicians and the public on medical errors. N Engl J Med 2002;347(24):1933-40.
19. Brennan TA. The Institute of Medicine report on medical errors-could it do harm? N Engl J Med 2000;342(15):1123-5.
20. Hayward RA, Hofer TP. Estimating hospital deaths due to medical errors: prevent- ability is in the eye of the reviewer. JAMA 2001;286(4):415.
21. Singh H, Giardina TD, Forjuoh SN, et al. Electronic health record-based surveillance of diagnostic errors in primary care. BMJ Qual Saf 2012;21(2):93-100.
22. Goldman L, Kirtane AJ. Triage of patients with acute chest pain and possible cardiac ischemia: the elusive search for diagnostic perfection. Ann Intern Med 2003; 139(12):987-95.
23. Stanton VA, Hsieh Y-H, Camargo CAJ, et al. Overreliance on symptom quality in diag- nosing dizziness: results of a multicenter survey of emergency physicians. Mayo Clin Proc 2007;82(11):1319-28.