Risk factors and prognostic predictors of unexpected intensive care unit admission within 3 days after

ED discharge

Ju-Sing Fan MD^a, Wei-Fong Kao MD^a,b, David Hung-Tsang Yen MD, PhD^a,c,*,

Lee-Ming Wang MD^a, Chung-I Huang MD^a,b, Chen-Hsen Lee MD^a,c

^aDepartment of Emergency Medicine, Taipei Veterans General Hospital, Taipei 112, Taiwan, ROC

^bDepartment of Emergency Medicine, School of Medicine, National Yang Ming University, Taipei 112, Taiwan, ROC

^cInstitute of Emergency and Critical Care Medicine, School of Medicine, National Yang Ming University, Taipei 112, Taiwan, ROC

Received 11 January 2007; revised 2 March 2007; accepted 2 March 2007

Abstract

Objective: Our objective was to investigate the risk factors and prognostic predictors of unexpected intensive care unit admission within 3 days after emergency department (ED) discharge.

Methods: From January 1, 2001, through December 31, 2005, patients admitted to the ICU unexpectedly within 3 days after being discharged from the ED were enrolled. Medical records of these patients were retrospectively reviewed. We categorized each patient’s characteristics into dichotomous groups and used the v² test to identify risk factors for unexpected ICU admission within 3 days after ED discharge. A multiple logistic regression was applied to examine possible independent predictors of poor prognoses.

Results: During the study period, 365321 patients visited our ED; 241(0.07%) were unexpectedly admitted to the ICU within 3 days after being discharged from the ED. Mean patient age was

74.2 F 16.4 years. The rate of ICU admissions caused by Medical error was 0.019% F 0.004% of all visits and 29.0% F 5.7% of all unexpected ICU admissions. The overall mortality rate was 19.9% (48/241). Risk factors for unexpected ICU admission within 3 days after discharge from the ED were age of 65 years or older (odds ratio [OR], 5.4; 95% confidence interval [CI], 4.0-7.4), Ambulance transport (OR, 5.1; 95% CI, 3.9-6.5), no accompanying family (OR, 3.5; 95% CI,

2.7-4.5), nonambulatory status (OR, 4.2; 95% CI, 2.9-5.0), not living at home (OR, 2.5; 95% CI,

1.9-3.3), Medicaid insurance (OR, 3.6; 95% CI, 2.8-4.7), and emergency stay of more than 24 hours (OR, 4.4; 95% CI, 3.4-5.7). The independent predictors of mortality were age of 65 years or older (OR, 2.4; 95% CI, 1.7-3.6), multiple comorbidities (OR, 4.0; 95% CI, 1.8-8.5), medical error leading to ICU admission (OR, 3.9; 95% CI, 1.8-8.3), and Acute Physiology and Chronic Health Evaluation II score of 20 or higher (OR, 2.9; 95% CI, 1.1-7.8).

* Corresponding author. Department of Emergency Medicine, Taipei Veterans General Hospital, Taipei 112, Taiwan, ROC. Tel.: +886 2 28757377; fax: +886 2 28757842.

E-mail address: [email protected] (D.-H.T. Yen).

0735-6757/$ – see front matter D 2007 doi:10.1016/j.ajem.2007.03.005

Conclusions: In our study, the risk factors and prognostic predictors of unexpected ICU admission within 3 days after ED discharge were identified. Based on these risk and Prognostic prediction factors, further strategies for decreasing the incidence of serious adverse events of ED-discharged patients can be implemented.

D 2007

Introduction

In recent years, adverse events in medical care have become a great threat to patients’ safety in emergency departments (EDs) [1-5]. In the Harvard Medical Practice Study report, although only 1.5% to 3% of all adverse events developed in the ED, 25% of those ED adverse events led to death or permanent disability. Of these serious or fatal events, 70% were attributed to medical error [1]. To help improve the quality of medical care and patient safety in the ED, additional attention should be paid to discover the risk factors for those adverse events, especially factors resulting in serious consequences [1-5].

Large retrospective studies such as the Harvard Medical Practice Study and the Colorado-Utah study have given some indications of the nature of errors that have caused adverse events in the ED, but these reports only considered errors associated with patients admitted from the ED [1,3-5]. Most patients seen in the ED, those that were discharged, were not included [1,4,5]. Although the literature discussing ED revisits has mentioned unexpected consequences in ED- discharged patients [6-11], no report has focused on events in which patients rapidly deteriorated after being discharged and then unexpectedly died or were admitted to Intensive care units on return visits, which characterizes the most serious adverse events involving ED-discharged patients [1,6-11]. Indeed, to our knowledge, studies focus- ing on this type of serious adverse event in ED-discharged patients are rare [5].

The aim of this study was to elucidate the incidence, risk factors, and prognostic predictors for unexpected ICU admission within 3 days after ED discharge. We also classified causes of these unexpected ICU admissions, including medical-error and non-medical-error causes, and tried to construct a preventive strategy. The ultimate goal was to provide reference standards for improving patient safety in the ED.

Methods

Patients

This study was conducted in the ED of a tertiary referral medical center, where 73064 F 5123 (mean F SD) patients were treated each year in the past decade. The study period was from January 1, 2001, to December 31, 2005. Each month, we used the emergency quality-assurance computer program of our hospital to screen patients who were

admitted to the ICU within 3 days after being discharged from the ED. Candidates included patients with or without trauma and pediatric patients. Patients were enrolled if they had not been treated in a ward, outpatient department, or ED of another hospital during the time between their initial visit and their ICU admission. This study was approved by the institutional review committee of Taipei Veterans General Hospital, Taipei, Taiwan.

Data collection and analysis

We collected data from the patients’ medical records and included demographic data, physicians’ records, nurses’ records, and records of laboratory and imaging studies. To determine the cause of the unexpected ICU admission, we defined medical-error causes for ICU admission as inap- propriate care occurring during emergency care, which was the main reason for unexpected deterioration in the patient’s condition and for resultant admission to the ICU. Using this definition, 2 experienced emergency physicians independently determined whether the patient’s situation resulted from medical error by reviewing the medical records and by consulting the initial treating physician if available. The reviewers’ supervisor made the final decision in each case. Besides being an experienced emergency physician, the supervisor also had considerable experience in determining medical error. To further analyze the relationship between prognosis and the type of medical error, we clinically classified medical error as being either diagnosis-related or nondiagnosis-related.

Statistical analysis

To identify risk factors for unexpected ICU admission, we categorized each patient’s characteristics into dichotomous groups based on the risk factors for 72-hour ED return visit or readmission. These factors had been identified in previous studies [6-11] and were used to statistically analyze patients’ relative risk. Patient characteristics analyzed included age (z65 or b65 years), sex (male or female), means of transportation (ambulance or other), presence of accompa- nying family members (yes or no), ambulatory status (yes or no), residency (home or other place such as nursing home), insurance (Medicaid or other), ED triage grade (emergency or nonemergency), type of illness on First visit (trauma or other), and duration of ED stay (z24 or b24 hours). Because all the ED patients (including the study cases) were treated by physicians of the same grade (board-certificated emer- gency physician) in our ED, the factor of treating physician’s grade was not put into the statistic analysis.

We statistically analyzed the clinical factors possibly related to the prognoses to identify independent predictors of poor prognoses. The selected predictors for analysis had primarily been documented to be related to the prognoses of critically ill patients [12,13], including age of 65 years or older, male sex, multiple comorbidities, emergency triage grade on first visit to the ED, ED stay of 24 hours or longer, no inpatient specialist consultation on the first visit , surgical intervention on return visit, use of inotropic agents on return visit, intubation on return visit, return visit with ICU admission within 24 hours after ED discharge, and Acute Physiology and Chronic Health Evaluation II score of 20 points or higher. Poor prognoses were considered to be death (including deaths in the hospital and deaths of terminally ill patients who dis- charged themselves and died during follow-up), prolonged admission (hospital stay z4 weeks), being able to walk before admission but not after discharge, and readmission within 3 months after being discharged from wards. During the period of data collection, 1 patient was still hospitalized at last follow-up, and the final prognosis could not be determined. Walking ability was confirmed by telephone in all but 2 patients. These 3 patients with missing data were not included in calculations related to prognosis; however, we assumed that this small omission would not affect the final results.

Age (y) z65b	146487	189	5.4 (4.0-7.4)
b65	218834	52
Sex
Male	216776	151	1.2 (0.9-1.5)
Female	148545	90
Transportation
Ambulanceb	67547	129	5.1 (3.9-6.5)
Other	297774	112
Companion
Family	254429	96

In addition, poor prognoses were analyzed according to

different Diagnostic categories. Diagnoses were grouped into categories using the following Diagnostic codes from the International Classification of Diseases, Ninth Revision, Clinical Modification: 290-319, Mental disorders; 390-459, circulatory system disorders; 460-519, respiratory system disorders; 520-579, digestive system disorders; and all other codes for other diagnoses.

Statistical analysis was performed using software (SPSS,

initial visit and their ICU admission. Mean patient age was

74.2 F 16.4 years.

Seventy ICU admissions were due to medical error, as determined by the reviewers’ supervisor. Therefore, the rate of ICU admissions caused by medical error was 0.019% F 0.004%, or 29.0% F 5.7% of all unexpected ICU admissions. For this determination, j was 0.72, indicating consistency higher than that of previous studies on inpatients (j = 0.20-0.40) [1-4,14]. Diagnosis-related

medical error occurred in 24 of 70 patients; 14 (58.3%) of the 24 patients died. Nondiagnosis-related medical error affected 46 patients; 10 (21.7%) of these patients died. The difference between diagnosis- and nondiagnosis-related medical error was significant among patients who died (mortality rate, 58.3% F 19.6% and 21.7% F 11.8%, respectively; P = .003).

version 13.0; SPSS, Chicago, Ill). Cohen n values were used to examine the consistency of decisions regarding deterio-	None or otherb Ambulatory	110 892	145	3.5 (2.7-4.5)
ration in the patient’s condition caused by medical error. The	Yes	157219	37
cross-table method was used to calculate the Relative risks of	Nob	208012	204	4.2 (2.9-5.0)
unexpected ICU admission in patients with different	Residence

characteristics. A multiple logistic regression was used to examine possible independent factors that might have been related to the patient’s prognosis. P b .05 indicated a statistically significant difference.

Table 1 Characteristics of patients discharged from the

ED and their odds ratios for unexpected ICU admission within 3 days

Characteristic ED patients Unexpected ICU

(N = 365321) admissions

No. Odds ratio^a (N = 241)

Initial triage grade

Emergency Nonemergency Type of illness Trauma

Other

34283

331038

23

218

1.0 (0.7-1.6)

81633

283688

47

194

0.8 (0.6-1.2)

Emergency stay (h)

^a Data in parentheses are 95% confidence intervals.

^b Significantly associated with an increased risk of unexpected ICU admission within 3 days.

Home	292809	148
Other placeb	72512	93	2.5 (1.9-3.3)
Insurance Medicaidb	127230	159	3.6 (2.8-4.7)
Other	238091	82

Results

In this 5-year study, 365321 patients were treated in the ED of our hospital, and 246 were admitted to the ICU within 3 days after being discharged from the ED. All patients were admitted to the ICU by revisiting the ED, and 241 patients fulfilled the study criteria, resulting in a rate of 0.07% F 0.008%. Among the 5 excluded patients, 3 had been hospitalized, and the remainder had been treated in the EDs of other hospitals during the time between their

b24	304440	128
z24b	60881	113	4.4 (3.4-5.7)

Predictor	Odds ratioa
	Death after	Prolonged	Nonambulatory status	Readmission
	ICU admission	admission	after discharge	within 3 mo
Patient characteristics
Age z65 y	2.4 (1.7-3.6)*	4.9 (1.6-15.4)*	3.2 (1.2-8.1)*	2.6 (1.8-4.0)*
Male sex	0.7 (0.3-1.5)	1.5 (0.7-3.3)	1.1 (0.5-2.2)	0.5 (0.2-1.1)
Multiple comorbiditiesb	4.0 (1.8-8.5)*	7.2 (3.3-15.5)*	0.9 (0.4-1.8)	3.6 (2.2-5.6)*
On first visit
First triage grade	0.5 (0.1-2.5)	1.5 (0.1-2.3)	1.3 (0.3-4.7)	0.9 (0.2-4.2)
Emergency stay z24 h	1.9 (0.4-8.1)	1.3 (0.3-5.3)	0.7 (0.2-2.3)	0.8 (0.2-3.0)
No consultation	0.8 (0.3-1.7)	1.6 (0.7-3.4)	0.5 (0.3-1.2)	1.1 (0.5-2.5)
On return visit
Surgical intervention	0.9 (0.4-2.1)	0.6 (0.3-1.3)	1.2 (0.5-2.4)	0.8 (0.4-1.8)
Use of inotropics	0.9 (0.2-3.7)	0.6 (0.2-2.6)	0.4 (0.1-1.6)	0.5 (0.1-2.3)
Intubation	2.7 (0.5-15.0)	2.0 (0.4-11.2)	3.1 (2.7-5.5)*	2.1 (0.1-1.1)
Medical error	3.9 (1.8-8.3)*	0.6 (0.3-1.4)	1.1 (0.5-2.1)	0.4 (0.4-11.8)
Return within 24 h	0.9 (0.4-2.1)	0.9 (0.4-1.9)	0.8 (0.4-1.7)	1.1 (0.5-2.3)
APACHE II score z20	2.9 (1.1-7.8)*	0.8 (0.4-2.0)	3.0 (2.2-4.4)*	0.7 (0.3-1.6)

The risk of unexpected ICU admission within 3 days significantly increased for patients 65 years or older, those brought in by ambulance, those not accompanied by family, those nonambulatory, those not living at home, those with Medicaid insurance, and those who stayed in the ED for 24 hours or longer (Table 1).

Table 2 Predictors of poor prognoses in patients admitted to the ICU unexpectedly within 3 days after discharge from ED

^a Data in parentheses are 95% confidence intervals of the odds ratio.

^b Multiple comorbidities was defined as more than 3 of the following comorbidities: liver cirrhosis, hypertension, diabetes, old stroke, congestive heart failure, chronic obstructive pulmonary disease, Chronic renal insufficiency, coronary artery disease, and malignancy.

* P b .05, statistically significant.

With regard to prognoses, among 165 patients who had an APACHE II score of 20 or higher, 41 (24.8%) died; among 76 patients with an APACHE II score of less than 20, seven (9.2%) died, and 48 had prolonged admission. Among all patients in the study, 75 (including mortality cases) were unable to walk on discharge, and 23 were readmitted to the hospital within 3 months. Table 2 lists the factors possibly related to a poor prognosis. Independent predictors of death were age of 65 years or older, multiple comorbidities, ICU admission caused by medical error, and an APACHE II score of 20 points or higher. In survivors, independent predictors for prolonged admission were age of 65 years or older and multiple comorbidities. Predictors for

the inability of a previously ambulatory patient to walk on discharge were age of 65 years or older, intubation, and an APACHE II score of 20 points or higher. Predictors of readmission within 3 months after discharge were age of 65 years or older and multiple comorbidities.

Respiratory system disorders resulted in the most unexpected ICU admissions, followed by circulatory sys- tem, digestive, and mental disorders (Table 3). Rates of nonambulatory status after discharge were significantly different among diagnostic categories (higher in mental disorders, P b .001), but the rate of death ( P = .76), prolonged admission ( P = .07), and readmission within 3 months ( P = .19) were not.

Discussion

Unexpected early ICU admission of an ED-discharged patient is a serious adverse event of ED management.

Table 3 Comparisons of prognoses for different diagnostic categories diagnostic category (N = 241) No. of patientsa
	Death	Prolonged admission	Nonambulatory after dischargeb	Readmission within 3 mo
Respiratory (n = 96)	22 (22.9)	15 (15.6)	26 (27.1)	6 (6.3)
Circulatory (n = 55)	11 (20.0)	10 (18.2)	14 (25.5)	8 (14.5)
Digestive (n = 45)	6 (13.3)	8 (17.8)	8 (17.8)	2 (4.4)
Mental (n = 27)	5 (18.5)	11 (40.7)	21 (77.8)	4 (14.9)
Other (n = 18)	4 (22.2)	4 (28.6)	6 (33.3)	3 (16.7)
a Data in parentheses are percentages. b Significant difference for each diagnostic category.

Understanding why such serious adverse events happen to find ways to prevent them may be the most important topic in studies related to patient safety [1,2]. In this study, we investigated the risk factors, causes, and prognostic factors.of patients unexpectedly admitted to the ICU. Because patient characteristics and criteria for ICU admission differ in various EDs, the incidences of unexpected ICU admis- sions may also differ. To our knowledge, no previous study has focused on these incidences; hence, we were unable to judge the approximate incidence range. However, in an ED revisit study conducted in another medical center in Taiwan, the rate of unexpected ICU admission within 3 days was 0.04%. This rate was lower than ours (0.07%) partly because our admissions included a greater number of single elderly patients [8,9]. In terms of ICU care for these patients admitted to the ICU unexpectedly within 3 days after being discharged from our ED, 7 (9.2%) of 76 patients with an APACHE II score of 10 to 19 points died, and 41 (24.8%) of 165 patients with an APACHE II score of 20 points or higher died. These rates were lower than the respective rates of 12% to 22% (score of 10-19) and 40% (score z20) that Knaus et al [13] reported.

Risk factors for unexpected ICU admission were also the

risk factors for other kinds of adverse events. McCusker et al

[10] reported that living alone and additional functional problems were significantly associated with repeated visits to the ED. Martin-Gill and Reiser [11] described a significant difference in the risk of 72-hour return and admission in different types of insurance and initial diagnostic categories. Furthermore, multiple studies have mentioned that the elderly are predisposed to have several kinds of adverse events because of their poor functional reserve, atypical presentation, highly complex diseases, and poor compliance in therapy after they leave the hospital [6-11,15].

As previous studies have shown, the reliability of judging whether an adverse event was caused by medical error has always been unsatisfactory [1-4,14]. However, the consis- tency of judging in our study was statistically significant, with high j values. The reason for past difficulties was possibly because the reviewers (nurses, physicians, or surgeons) had to make judgments across all medical specialties. Because it is impossible for any reviewer to be an expert in all fields, previous study designs enhanced differences in how situations are interpreted and in their judgments of causality [1-4,14]. In our study, all reviewers were experienced emergency physicians with identical professional training, the topic to be judged was in their professional field, and the initial attending physician was consulted when available to clear up ambiguities. Therefore, we observed a corresponding increase in consistency.

Diagnoses in 223 (92.5%) of 241 patients fell into 4 diagnostic categories commonly noted among ED patients: respiratory system disorders, circulatory system disorders, digestive system disorders, and mental disorders. This distribution demonstrated that the patient’s condition could unexpectedly deteriorate even if the emergency physician

was well familiar with the disease. It also suggested that establishing a mechanism to prevent errors in managing patients of these disease categories is imperative to reduce the number of unexpected ICU admissions.

In terms of the predictors of the poor prognoses, we found that old age and multiple comorbidities were predictors of many unfavorable outcomes. These predictors are known from the time of the patient’s initial visit. If a patient has either advanced age or multiple comorbidities, appropriate care should be taken in his or her treatment. Extending observation and treatment times, requesting a supervisor to verify the suitability of management, and even lowering the threshold for admission may be necessary. If the decision is made to discharge the patient, a good discharge plan is also required. Researchers have suggested that, when these patients are discharged from emergency care, a specialized, multidisciplinary consulting team should be assembled to design a subsequent treatment and follow-up plan [15]. Chern et al [16] suggested implementing a telephone follow- up program to decrease the incidence of severe adverse events. If patients with unexpected ICU admission have any of the poor prognostic factors, we should make a greater effort in caring for these patients to improve their prognoses. Of note, if the situation evolves into a medical dispute, the decision to take legal action and the final verdict are both closely related to the patient’s outcome [17].

Factors other than medical error accounted for approx- imately two thirds of all unexpected ICU admissions. Patients in this situation had an unexpected deterioration in their condition after receiving appropriate care. Their readmission demonstrates the uncertainty and unpredictabil- ity of medical outcomes [18]. Room for improvement in this setting is relatively limited. In the converse, medical error represents avoidable human error, and, in this study, ICU admissions caused by medical error or inappropriate care were associated with a worsened prognosis. Previous authors have also mentioned that medical error was related not only to poor prognoses but also to medical disputes, indicating the urgency of improvements in this aspect [1-5,12-17,19]. Diagnosis- and nondiagnosis-related medi- cal error was examined because errors emerging from the diagnostic process usually cause more catastrophic results [5,20]. In our study, diagnostic error was significantly associated with an increased mortality rate. This finding once again indicated that the diagnosis is a pivotal part of the Clinical process in the ED.

Prevention

As a general measure, a protective mechanism to prevent unexpected ICU admission should be constructed and applied in the treatment of patients with risk factors. Besides assembling a multidisciplinary consulting team to care for elderly patients and telephone follow-up for high-risk patients, some authors advocate designing a computerized warning system for high-risk patients to prevent adverse

events because this method is more efficient and less labor- intensive than others [21].

As a specific measure, detailed discussion may be implemented in each unexpected ICU admission caused by medical error to determine the factors contributing to the error [1-5,22,23]. Contributing factors can be found using the framework provided by Vincent et al [22]. They divided these factors into 7 categories: institutional context, organi- zational and management factors, work environment, team factors, individual factors, task, and patient factors. A category-by-category analysis based on this framework can be done to accurately pinpoint the source of the problem, which may then be resolved. Most researchers recommend this systematic method, wherein the mistake is found by looking at the institution, the work environment, the protective mechanism, the physician, and the patient. Solely examining the physician’s mistakes is no longer recommended [1-5,14,19-24].

Limitations

This study had some limitations. First, the interactions between risk factors for unexpected ICU admissions could not be taken into account. The factors could only be related to an increase in the incidence and not identified as independent risk factors. Second, although the prognostic factors calculated in the regression model were independent predictors, the sample size might have slightly reduced the statistical power. Third, this study was retrospective, and incomplete data were difficult to avoid. However, the emergency and inpatient data we collected were the most basic information required to complete a case history. Therefore, the effect of incomplete data was reduced to be minimal. Fourth, we did not recruit the patients who were admitted to the ICU of other hospitals, those who were admitted after 3 days of ED discharge, and those who died before return visit, which might underestimate the overall incidence of serious adverse events of ED management.

Conclusions

Early unexpected ICU admission after ED discharge, especially admission related to medical error, is an important patient safety topic for which improved under- standing and prevention are urgently needed. Our study elucidated patient characteristics related to an increased incidence of unexpected ICU admissions within 3 days after ED discharge and yielded poor prognostic predictors. A protective mechanism should be constructed to help prevent errors in the care of patients with risk factors, and additional effort should be made to improve the clinical outcomes of patients with poor prognostic factors. In addition, the errors contributing to each unexpected ICU admission should be identified using a system analysis method.

References

1. Brennan TA, Leape LL, Laird NM, et al. incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 1991;324(7):370 – 6.
2. Nguyen TV, Hillman KM, Buist MD. Adverse events in British hospitals. Preventive strategies, not epidemiological studies, are needed. BMJ 2001;322:1425.
3. Leape LL, Brennan TA, Laird N, et al. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study

II. N Engl J Med 1991;324:377 – 84.

1. Thomas EJ, Studdert DM, Bustin HR, et al. Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care 2000;38:261 – 71.
2. Hobgood C, Croskerry P, Wears RL, et al. Patient safety in emergency medicine. In: Tintinalli JE, Kelen GD, Stapczynski JS, editors. Emergency medicine: A comprehensive study guide. 6th ed. New York7 McGraw-Hill; 2004. p. 1912 – 8.
3. Keith KD, Bocka JJ, Kobernick MS, et al. Emergency department revisits. Ann Emerg Med 1989;18:964 – 8.
4. Pierce JM, Kellerman AL, Oster C. dBouncesT: an analysis of short- term revisits to a public hospital emergency department. Ann Emerg Med 1990;19:752 – 7.
5. Hu CH. Analysis of patient revisits to emergency department. Am J Emerg Med 1992;10:366 – 70.
6. Liaw SJ, Bullard MJ, Hu PM, et al. Rates and causes of emergency department revisits within 72 hours. J Formos Med Assoc 1999;98: 422 – 5.
7. McCusker J, Healey E, Bellavance F, et al. Predictors of repeat emergency department visits by elders. Acad Emerg Med 1997;4: 581 – 8.
8. Martin-Gill C, Reiser RC. Risk factors for 72-hour admission to the ED. Am J Emerg Med 2004;22:448 – 53.
9. Olsson T, Terent A, Lind L. Rapid Emergency Medicine Score can predict long-term mortality in nonsurgical emergency department patients. Acad Emerg Med 2004;11:1008 – 13.
10. Knaus WA, Draper EA, Wagner DP, et al. APACHE II: a Severity of disease classification system. Crit Care Med 1985;13:818 – 29.
11. Thomas EJ, Lipsitz SR, Studdert DM, et al. The reliability of medical record review for estimating Adverse event rates. Ann Intern Med 2002;136:812 – 6.
12. Caplan GA, Williams AJ, Daly B, et al. A randomized, controlled trial of comprehensive geriatric assessment and multidisciplinary interven- tion after discharge of elderly from the emergency department: the DEED II study. J Am Geriatr Soc 2004;52:1417 – 23.
13. Chern CH, How CK, Wang LM, et al. Decreasing clinically significant adverse events using feedback to emergency physicians of telephone follow-up outcomes. Ann Emerg Med 2005;45:15 – 23.
14. Brennan TA, Sox CM, Burstin HR. Relation between negligent adverse events and the outcomes of medical-Malpractice litigation. N Engl J Med 1996;335:1963 – 7.
15. Scheidt S, Wenger N, Weber M. Uncertainty in medicine: still very much with us in 2004. Am J Geriatr Cardiol 2004;13:9 – 10.
16. Vincent CA. Research into medical accidents: a case of negligence? BMJ 1989;299:1150 – 3.
17. Kuhn GJ. Diagnostic errors. Acad Emerg Med 2002;9:740 – 50.
18. Kawamoto K, Houlihan CA, Balas EA, et al. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005; 330:765 – 8.
19. Vincent C, Taylor-Adams S, Stanhope N. Framework for analyzing risk and safety in clinical medicine. BMJ 1998;316:1154 – 7.
20. Adams JG, Bohan JS. System contributions to error. Acad Emerg Med

2000;7:1189 – 93.

1. Reason J. Human error: models and management. BMJ 2000; 320:768 – 70.