Article

The inaccuracy of determining overcrowding status by using the National ED Overcrowding Study Tool

a b s t r a c t

Background: Emergency department (ED) crowding has become more common, and perceptions of crowding vary among different health care providers. The National Emergency Department Overcrowding Study (NEDOCS) tool is the most commonly used tool to estimate ED crowding but still uncertain of its reliability in different ED settings.

Objective: The objectives of this study are to determine the accuracy of using the NEDOCS tool to evaluate overcrowding in an extremely high-volume ED and assess the reliability and consistency of different providers’ perceptions of ED crowding.

Material and methods: This was a 2-phase study. In phase 1, ED crowding was determined by the NEDOCS tool. The ED length of stay and number of patients who Left without being seen were analyzed. In phase 2, a survey of simulated ED census scenarios was completed by different providers. The interrater and intrarater agreements of ED crowding were tested.

Results: In phase 1, the subject ED was determined to be overcrowded more than 75% of the time in which nearly 50% was rated as severely overcrowded by the NEDOCS tool. No statistically significant difference was found in terms of the average length of stay and the number of left without being seen patients under different crowding categories. In phase 2, 88 surveys were completed. A moderate level of agreement between health care providers was reached (? = 0.5402, P b .0001). Test-retest reliability among providers was high (r = 0.8833, P = .0007). The strength of agreement between study groups and the NEDOCS was weak (? = 0.3695, P b .001).

Conclusion: Using the NEDOCS tool to determine ED crowding might be inaccurate in an extremely high-volume

ED setting.

(C) 2014

  1. Introduction

In recent years, with increased demands associated with the growth of annual emergency department (ED) visits in comparison with static or limited hospital and ED resources, ED crowding has become a more prevalent and worsening problem throughout the nation, especially in urban areas [1-3]. Despite an increase in the

? Source(s) of support in the form of equipment, drugs, or grants (including grant numbers): None.

?? Competing interests: N/A.

? Author contributions: HW and RDR conceived the study and developed the design

in consultation with all of the authors. KB, CAH, BB, KW, and NCW assembled the data set and collected the data. HW, RDR, RDJ, and NRZ conducted the statistical analyses and drafted the article, and all authors read and approved the final manuscript. HW takes responsibility for the manuscript as a whole.

* Corresponding author. Department of Emergency Medicine, John Peter Smith Health Network, 1500 S Main St, Fort Worth, TX 76104.

E-mail address: [email protected] (H. Wang).

literature concerning this subject, there is still no definitive consensus on terminology or an actual operational identification of ED crowding. Among all current models, the National Emergency Department Crowding Study (NEDOCS) outcome is currently considered the most commonly used and reliable ED crowding scoring tool developed to date in terms of its relatively higher consistency and better performance in determining the degree of ED crowding status [4-7]. It was developed from 8 academic EDs with moderate-to-high annual volumes (ranging from 40000 to 83000 with an average of 57000/ year). It has been validated well in the setting of EDs with similar annual volumes when compared with those of the study group but remains unproven when compared with EDs whose volumes are outside the range of the study group. This is especially notable when using NEDOCS in the setting of an extremely high-volume ED.

In addition, NEDOCS and other ED overcrowding tools have been developed based on varying health care providers’ perceptions of ED crowding, which may neither be objective nor accurate [4,8,5-7]. Furthermore, the intraobserver and Interobserver variability of

http://dx.doi.org/10.1016/j.ajem.2014.07.032

0735-6757/(C) 2014

perceptions of ED crowding in different health care providers was rarely tested and compared in previous studies. Therefore, the goals of this study are (1) to determine the accuracy of using the NEDOCS scoring tool to evaluate ED overcrowding in an extremely high ED volume setting, and (2) to assess the reliability and consistency of using different health care providers’ real time perceptions of ED crowding.

27 patients directly admitted to acute psychiatric unit 251 patients transferred from urgent care center

539 patients: no NEDOCS score calculated when registered

Transfer to other service 355 (5.18%)

AMA 60 (0.88%)

Eloped 38 (0.55%)

Expired 5 (0.07%)

6854 patients enrolled in this study

7671 patients registered during study period

  1. Materials and methods
    1. Study design

This was a prospective observational 2-phase study designed to externally validate the accuracy of the NEDOCS scoring tool in determining overcrowding status in an extremely high-volume ED. The institutional review board approved the study.

The first phase involved data analysis of the NEDOCS scale to determine ED overcrowding status. This study was performed at ED of John Peter Smith Health Network from June 1 to June 24, 2013. The study ED is a publicly funded urban tertiary care hospital ED with an annual volume of approximately 110000. This academic ED hosts an Emergency Medicine Residency Program and also is a level 1 trauma center. The NEDOCS score was calculated by using an online calculator (http://www.nedocs.org) every 2 hours at real-time points during the study period. Briefly, 2 constants (the number of ED and hospital- licensed beds) and 5 other variables were required to calculate the NEDOCS scores [4]. Considering that the number of ED-licensed bed was a fixed variable reported from the previous study, this constant was not changed when the hallway beds were added sometime at ED [4]. Emergency department crowding status was determined by the NEDOCS score and further divided into 3 different categories: not overcrowded, overcrowded, and severely overcrowded. Not over- crowded status was defined as NEDOCS score of less than 100, overcrowded was defined as NEDOCS score of less than 140 (including score of 100), and severely overcrowded was considered if the score was higher than 140 (including 140) [4]. All patients during the study period were assigned to have NEDOCS scores calculated at the time the patients were registered in the ED and then stratified into 3 different crowding categories. Patients who were directly admitted by other services and immediately moved out of the ED were excluded from this study. Patients who were transferred from our urgent care center due to a requirement for higher level of care and/or potential high-risk presentation were excluded from the study. Because of the study hospital policy, these patients who transferred from urgent care center will have the priority to be placed in an ED bed as earlier as possible no matter which Emergency Severity Index levels they triaged initially. Therefore, the length of ED stay of these patients might not well correlate with the ED crowding status. If NEDOCS score could not be calculated at certain time points due to incomplete data recorded, patients who registered at ED within those time periods were also excluded from this study (Fig. 1). To know whether ED overcrowding will potentially affect ED operations, length of ED stay and the number of patients who Left without being seen were used as the markers for ED efficiency measurements. The length of stay (LOS) in the ED of each patient was collected, and the average of LOS was analyzed and compared under the different ED crowding conditions. Further analysis was performed and compared in terms of the LOS under the different categories including patient Acuity levels and type of disposition (eg, discharge, admission, transfer, etc). In addition, the number of LWBS patients was also analyzed and compared in different ED crowding conditions determined by the NEDOCS scale.

The second phase was a survey questionnaire study to measure the

group agreement of perceived ED crowding by different health care providers, including physicians, nurses, residents, and senior medical students as compared with the NEDOCS score. This phase of the study

Discharged

4482 (65.39%)

Admission

1373 (20.03%)

LWBS

541 (7.89%)

Fig. 1. Shows the flow diagram of the number of patients included in the ED crowding study in June 2013.

was conducted in July and August of 2013 separately. Clinical variables reviewed when deriving the NEDOCS score included total patients in the ED, total admits in the ED, number of ventilators used in the ED, longest admit time in hours, and waiting room time (in hours) of the most recent patient placed in a bed in the ED. Two constants including total ED beds and total Hospital beds were also used in deriving the NEDOCS score. The NEDOCS model was designed based on the “input-throughput-output theory [9,10].” This model does not take into consideration the potential impact of the ESI score of patients in the ED at the time of scoring nor does it consider physician, resident, and nurse staffing levels as potential contributors to ED overcrowding scores. Our survey questionnaire study was designed based on an “input-throughput-output-overall staffing” model, which includes physicians, nurses, and residents on duty. Other clinical variables such as the number of patients with different ESI levels and the longest wait time of those patients in the waiting room at the time of scoring were also included. Ten different clinical scenarios consisting of 20 different Operational variables of different levels in each scenario were given to physicians, nurses, residents, and medical students to determine perceived ED overcrowding (Appendix). Considering the importance of recognizing an over- crowded or severely overcrowded status in the ED by health care providers, most of these scenarios (8/10) were designed assuming a severely overcrowded status in the ED based on the NEDOCS score relative to each scenario.

Scenario 1 was considered the basic setting with the NEDOCS score

of 150, indicating a severely overcrowded status in the ED. Each operational variable that was initially not included in NEDOCS was changed at different severity levels in scenarios 2, 3, 5, 6, 8, and 10 when compared with the basic setting (see detail in Fig. 2). These potential independent operational variables were the numbers of nurses, physicians, and residents on duty; different acuity levels of the test patients who had already been seen and evaluated by physicians/ residents in the ED and whose dispositions were pending; different acuity levels of test patients in the waiting room who had not yet been seen; and the longest time that a test patient had remained in the waiting room pending examination room placement. These opera- tional variables were considered to represent a potentially more profound effect on ED overcrowding yet are not included in the NEDOCS scoring system. In addition, the severity of NEDOCS variables

Visual Analog Scale of ED Crowing by Different Healthcare Providers

9

Nurse Physician Senior Resient

Junior Resident and Student

8

7

6

5

4

Base S2 S3 S5 S6 S8 S10

Fig. 2. Shows the average VAS of ED crowding by different health care providers participating in the survey study. Health care providers were divided into 4 groups including physicians, nurses, senior residents, and junior residents and medical students. Scenario 1 was established as the baseline setting with a NEDOCS score of 150, indicating an ED severely overcrowded status. Scenarios 2, 3, 5, 6, 8, and 10 were different scenarios that each changed the severity level of 1 operational variable. Scenario 2 increased the waiting time among patients in the waiting room; scenario 3 decreased the number of nurses on duty; scenario 5 increased the acuity levels of patients in the waiting room; scenario 6 increased the acuity levels of patients already in the ED and previously seen by physicians or residents but not yet dispositioned; scenario 8 decreased the number of physicians on duty; and scenario 10 decreased the number of residents on duty. The variables from NEDOCS included in these 7 scenarios were not changed. Therefore, each scenario has a NEDOCS score of 150 indicating a severely overcrowded status. The results of this study showed a moderate-to- good level of agreement between groups but weak agreement when compared with the NEDOCS.

were changed in scenarios 4, 9, and 7 as a reference to consider the different ED crowding statuses (severely overcrowded, overcrowded, and not overcrowded). In this manner, we intended to derive change

(s) in perception(s) of ED overcrowding status by health care providers as a function of changing operational variables.

The perceptions of ED overcrowding were rated on a 0 to 10 cm Visual analogue scale (VAS). To simulate a NEDOCS score, individual VAS scores recorded for each different scenario were automatically multiplied by 20 to allow for consistent comparison with the NEDOCS scale, where scores greater than or equal to 100 were considered overcrowded and scores greater than or equal to 140 represented severely overcrowded. Therefore, ED overcrowding was considered for a VAS greater than or equal to 5 and was considered severely overcrowded with a VAS greater than or equal to 7. Concurrently, the NEDOCS score was calculated for each scenario and compared with VAS results.

Considering potential differences in the perceptions recorded between the different test groups, survey results were analyzed by 4 different groups: physicians, nurses, senior residents (the second and third year emergency medicine residents), and junior residents/ medical students. Because the first survey was done in July when new interns started their first month in training, we combined the interns and medical students in 1 group, as this seems a more accurate grouping for data analysis. Some participants were tested twice at a minimum interval of at least 30 days apart to avoid recall bias [11,12]. The study was analyzed to determine whether different health care providers had similar perceptions of ED crowding and to determine individual consistency and reliability.

Data analysis and statistics

Considering the operational significance of determining relative ED overcrowding status, the study score was divided into 3 categories:

not overcrowded, overcrowded, and severely overcrowded. Patients were automatically assigned to these 3 groups based on ED overcrowding scores at the time when a patient registered for services in the ED. To compare the differences between LWBS and ED LOS relative to the different ED overcrowding groups, an analysis of variance with Bonferroni correction method was used.

In the phase 2 survey, Interrater agreement was measured by using the ? statistic (? N 0.4 was considered as moderate strength of agreement, and ? N 0.6 was considered as strong agreement). In addition, some participants who completed the initial survey were randomly chosen to test again at least 30 days later to determine the stability of test-retest (intrarater) agreement by using a correlation coefficiency test (r). A minimum 30 days interval was considered sufficient to avoid recall bias among those tested twice. A value of r N 0.5 was considered substantial and reliable.

All statistical analysis was performed using STATA 12 (STATA, College Station, TX), and a P b .05 was considered a statistically significant difference.

  1. Results

The Average ED LOS and the number of LWBS patients variables were found to have no significant difference when comparing patients registered under the overcrowded vs severely overcrowded conditions.

From June 1 to June 24, 2013, NEDOCS scores were calculated a total of 263 individual time points resulting in a 91.3% of completion rate (263/288). Among these 263 time points, 48.29% (127/263) were considered severely overcrowded, 28.14% (74/263) were overcrowd- ed, and 23.57% (62/263) were not overcrowded. During the same study period, there were a total of 7671 patients registered at ED seeking medical care. Among these 7671 patients, 27 patients were directly admitted to the acute psychiatric unit and were therefore excluded from this study. A total of 251 patients initially evaluated at our urgent care center and then transferred to ED for further evaluation were also excluded from this study. Another 539 patients were excluded due to no NEDOCS scores calculations during the period when those patients registered at ED. Therefore, 6854 patients were enrolled in this study for data analysis (Fig. 1). General patient demographics are shown in Table 1. The average LOS of ED patients with different acuity levels is shown in Table 2. The results indicate

Table 1

General information of patient information on ED crowding study

General information Phase 1 study

Total no. of patients (n) 7671

No. of patients for data analysis (n) 6854

Male (%) 3180 (48.62%)

Age (mean +- SD, 95% CI) (43.11 +- 15.60;

95% CI, 42.73-43.48)

Acuity level (n)

ESI 1 253

ESI 2 1500

ESI 3 3050

ESI 4 1650

ESI 5 331

Unclassified 60

Total no. of ambulance arrival (%) 2026 (29.5%)

Total no. of admission (%) 1373 (20.0%)

Total no. of patients LWBS (%) 541 (7.9%)

Abbreviation: CI, confidence interval.

In the study of June, a total of 6540 patients had age and sex information due to restricted information not able to be released from 314 patients.

Table 2

The comparison of the average LOS in patients with different acuity levels under different ED overcrowding status in phase 1 study

ED crowding status ESI 1 (h) ESI 2 (h) ESI 3 (h) ESI 4 (h) ESI 5 (h) Mean +- SD (n)

Patients who were discharged from ED

Not overcrowded

6.8 +- 4.9 (6)

4.4 +- 2.5 (161)

4.7 +- 2.1 (377)

3.3 +- 1.8 (289)

2.7 +- 1.3 (52)

Overcrowded

4.5 +- 3.7 (8) (a0.658)

5.1 +- 3.0 (216) (a0.121)

5.2 +- 2.6 (547) (a0.038)

4.0 +- 2.5 (352) (ab 0.001)

4.4 +- 7.9 (48) (a0.044)

Severely overcrowded

4.5 +- 2.5 (17) (b1.000)

4.8 +- 3.2 (379) (b0.636)

5.5 +- 3.3 (1096) (b0.148)

3.8 +- 2.0 (747) (b0.501)

3.1 +- 1.6 (172) (b0.066)

Patients who were admitted from ED

Not overcrowded

4.3 +- 2.9 (54)

9.0 +- 5.4 (113)

10.0 +- 6.1 (101)

7.7 +- 5.3 (10)

3.75 (1)

Overcrowded

6.6 +- 6.0 (49) (a0.043)

9.9 +- 7.1 (142) (a0.864)

10.7 +- 6.0 (154) (a1.000)

8.8 +- 6.5 (7) (a1.000)

12.9 +- 5.3 (2) (a1.000)

Severely overcrowded

6.1 +- 4.6 (85) (b1.000)

9.7 +- 7.0 (317) (b1.000)

11.4 +- 7.3 (317) (b0.803)

12.4 +- 12.4 (15) (b1.000)

5.4 (1) (b1.000)

Abbreviations: mean, the average of LOS in hours; n, the number of patients.

a P value of the comparison between 2 groups of patients under the different ED crowding conditions (not overcrowded vs overcrowded).

b P value of the comparison between 2 groups of patients under the different ED crowding conditions (overcrowded vs severely overcrowded).

that no statistically significant difference was reached regarding the average LOS of patients with the same level of acuity who registered at ED under the different crowding conditions. Similarly, there was no statistical difference in LOS regardless of disposition (admission vs discharge). There was a slightly shortened LOS in patients presenting to ED under severely overcrowded conditions than those presenting under overcrowded conditions but failed to reach statistical signifi- cance. When using LWBS as another patient outcome marker to determine whether it was affected by the different ED crowding statuses, our results found no statistically significant difference regarding the number of LWBS patients when registered initially under the different ED crowding conditions (Table 3). Taken together, given the status of the ED as overcrowded over 75% of the time and severely overcrowded nearly 50% of the time accompanied with no significant differences in terms of average ED LOS, and the number of LWBS patients registered under these different overcrowding condi- tions raises the question of the possibility of NEDOCS score inaccuracy when examining near real-time resource demands in the setting of an extreme high-volume ED.

Different operational variables were scored as more or less significant in terms of anticipated affect on ED crowding by the different individuals within the study groups of health care providers. However, the overall impression of ED overcrowding is highly consistent within a given group. Emergency department crowding status was overestimated by NEDOCS score when com- pared with the perceptions of the health care providers participating in the study.

A questionnaire survey including 10 different scenarios (see Appendix) was given to different health care providers in July

Table 3

The comparison of LWBS patients under different ED overcrowding status

ED crowding status determined by the NEDOCS scale The no. of LWBS patients

every 2 hours (n)

Not overcrowded 1.3 +- 1.6

Overcrowded 2.1 +- 2.5 (a0.132)

Severely overcrowded 2.4 +- 2.4 (b1.000)

a P value of the comparison between 2 groups of patients under the different ED crowding conditions (not overcrowded vs overcrowded).

b P value of the comparison between 2 groups of patients under the different ED crowding conditions (overcrowded vs severely overcrowded).

and August of 2013. A total of 88 surveys were completed by 76 health care providers within 2 months including physicians, nurses, residents, and fourth year medical students. The results of this study showed that 55.4% (383/704) of scenarios were considered severely overcrowded by different health care providers, whereas 100% of scenarios were scored as severely overcrowded using the NEDOCS tool. This indicates that NEDOCS score might overestimate the severely overcrowding status in those EDs similar to the study facility. In addition, we noted that there were different perceptions of crowding status as a function of the operational variables considered by the study survey as reported by the different health care providers participating in the study (Fig. 2). It further indicated that all the operational variables examined can potentially affect ED overcrowd- ing assessment to a certain degree. Increased patient wait time and decreased numbers of nurses and physicians on duty seemed to have the greatest impact on scoring, thereby resulting in a status of overcrowding or severely overcrowding.

In addition, our survey study showed a moderate-to-strong level of agreement between groups (? = 0.5402, P b .0001), indicating its interrater reliability among different health care providers’ perceptions in determining ED overcrowding status. Conversely, the strength of agreement between study groups and the NEDOCS score was weak (? = 0.3695, P b .001). Of note is the fact that 12 different health care providers completed this survey study twice at an interval of at least 30 days between the initial and second iterations. Results among this cohort demonstrated high test-retest reliability (r = 0.8833, P = .0007) indicating a substantial correlation.

Taken together, the average scores determining different ED crowding levels as scored by different health care providers at the same point in time are relatively reliable although subjective. We therefore conclude from data analysis that the perceptions of ED overcrowding by the NEDOCS tool might be overestimated when compared with the perceptions by the different health care providers.

  1. Discussion

In recent years, ED crowding and “overcrowding” have become more and more prevalent in regional and tertiary referral hospitals [13-15]. It has been shown that an overcrowded status will significantly affect ED operational efficiency and safety in a negative manner to include Ambulance diversion, increased ED return visits within 72 hours of initial visit, length of ED stay, LWBS rates, and decreased patient satisfaction scores [16-19]. Improved early

predictors of ED overcrowding status will assist ED and hospital administrators in implementing near real-time interventions de- signed to avoid reaching a Critical mass resulting in an unsafe environment due to insufficient resources. Currently, there is no criterion standard overcrowding severity tool available to accurately determine such status across all ED platforms. Because there is no criterion standard to determine ED overcrowding status, ED crowd- ing estimation tools previously derived considered using health care provider perceptions as the criterion standard. Their results showed reasonable agreement of ED overcrowding status among different providers [4-8]. The results of our survey study demonstrated similar findings indicating the reliability of provider’s perceptions of ED overcrowding.

Several ED overcrowding estimation tools were derived using different operational parameters. The NEDOCS is one tool that is used nationwide and has been found to determine ED overcrowd- ing status with relatively high consistency [4]. The NEDOCS score was derived from results of a questionnaire obtained from 8 academic EDs in the United States. Both our survey and pilot study results indicate that NEDOCS might overestimate the overcrowding status in the ED. This may be due to different perceptions of ED overcrowding by health care providers working at different ED locations. Studies showed that evaluators’ perceptions can be biased based on the habitual view of evaluator’s routine practice within a particular setting [20,21]. Crowding is considered a perception and can be changed based on the different evaluator’s experience. It could be quite similar by the different evaluators working at different EDs of similar settings. To the contrary, it could also be significantly different and might not be practical when ED settings change. The average annual volume of EDs participating in development of the NEDOCS tool was 57 000, which is only half of the annual volume of the ED in this study. Therefore, the health care provider’s perception of ED crowding may be magnified with the use of the NEDOCS tool to evaluate overcrowding status in an extremely high-volume ED setting. Again, it is reasonable to expect some variation among different ED overcrowding scoring tools status indicators when applied to different ED environments.

Apart from population selection bias, these ED overcrowding

estimation tools have limitations, and none of them included all of the potential operational parameters together in a single tool. The NEDOCS tool is limited to 5 variables using a mathematical formula, but it does not include physician, resident, and nurse staffing levels and patient illness severity level, which could potentially affect the accuracy of ED crowding estimation. In addition, the NEDOCS tool was measured every 4 hours and also skipped the 5 AM time point considering the relatively low volumes at that time, thereby generating incomplete time points over the standard 24-hour period for data analysis. Evaluation of ED crowding every 4 hours might not accurately reflect true ED crowding because the whole work up and disposition process is often finished within 4 hours of ED arrival in most patients. These potential ED crowding status changes within 4 hours may not be captured for data analysis.

Overall, our study outcomes indicate that the NEDOCS might not be a suitable tool to determine ED crowding in an academic ED setting affiliated with a publicly funded tertiary care hospital system providing extremely high-volume access to the needs of its patient population. Alternate ED overcrowding estimation tools developed in similar ED settings might provide more reliable and predictable Resource management strategies.

  1. Limitations

Results of this study are from a single Urban academic ED affiliated with a publicly funded hospital system, which has a very high annual

ED volume and high psychosocial risks in its patient population. Considering these data are from this single institution, we performed an internal validation gathering data from a different study time in the same institution. Validation exercises achieved similar results indi- cating reasonable consistency (data not shown). As previously mentioned, we realize that the study results may be skewed by virtue of population selection. Therefore, a larger multicenter study among similar ED environments is required to achieve external validation. At present, there remains no criterion standard tool capable of defining ED overcrowding. Sole reliance on perceptions of different health care providers may be overly subjective. This study tested the interrater and intrarater variability of health care providers’ perceptions, and results demonstrate a moderate-to-good agreement across study participants. Further analysis of this agreement requires a larger multicenter study to achieve validation in this area.

Using VAS (0-10) to estimate ED crowding is different as compared with the NEDOCS scoring tool. Multiplying VAS results by a standard factor of 20 to match the NEDOCS score may expand minor differences to a significant level. In this study, we were focused on determining 3 different levels of ED crowding (severely overcrowded, overcrowded, and not overcrowded) instead of the absolute number of a given score that may minimize bias. Test-retest comparison has some limitation as a measure of reliability. Perception of ED crowding status might change between 2 separate survey periods among individuals obtaining advance knowledge on ED overcrowding determination models at some point between completion of the initial and follow-up surveys. To ascertain this effect among study participants, a question was asked of each individual as part of the second survey inquiring whether they had learned anything new regarding ED overcrowding after the first survey. Only 1 health care provider answered yes, which did not affect the individual’s survey results significantly. Recall that using correlation to measure reliability does not measure absolute agreement in a strict sense. Only the relative agreement of a higher score measured at time 1 matched to a higher score measured at time 2 may be realized. The purpose of this study is to determine the reliability of using the VAS by different health care providers to determine the levels of ED crowding in 3 different categories rather than the determination of an absolute score and is therefore less affected when using correlation to determine test-retest reliability. In addition, other variable such as the occupancy of ED functional beds that could also correlate with ED crowding was not included in this study.

Our study showed that using the NEDOCS score to determine

ED crowding status had no strong correlation with increased LWBS and average ED LOS, which is inconsistent with the findings reported previously [22-24]. This could indicate the inaccuracy of using NEDOCS to determine ED crowding status. However, other factors could be acted as potential confounders such as the number of hall beds added at ED, the staffing hour difference between day and night shifts, and different number of patients with different severity levels registered at ED under different crowding status, etc. Furthermore, using ED LOS and number of LWBS patients as the markers to determine the outcome of ED crowding may not be accurate or reliable. Other negative patient care outcome markers might need to be analyzed such as ED 72-hour returns, mortality, patient satisfactions, etc. Therefore, a large multicenter study with more operational variable measure- ments is required for external validations.

Acknowledgment

The authors thank all the ED attending physicians, residents, medical students, nursing staff, and unit clerks participating in this study. Special thanks to Ms Daphne Celmer who helped in transferring the raw data to electronic files.

Appendix

Unlabelled image

Survey table: General information: hospital bed: 537, ED total bed: 56.

Scenario

1

2

3

4

5

6

7

8

9

10

1. Overall staff

Number of nurses on duty

30

30

15

30

30

30

30

30

30

30

Number of physicians on duty

4

4

4

4

4

4

4

2

4

4

Number of residents on duty

12

12

12

12

12

12

12

12

12

4

Total number of ED patients

90

90

90

90

90

90

72

90

90

90

2. Waiting room

Total number of patients in the waiting room

34

34

34

34

34

34

16

34

34

34

Longest time patient waiting at waiting room (h)

3

6

3

3

3

3

1

3

2

3

Number of patients in the waiting room with different

0, 0, 20,

0, 0, 20,

0, 0, 20,

0, 0, 20,

0, 0, 30,

0, 0, 20,

0, 0, 20,

0, 0, 20,

0, 0, 20,

0, 0, 20,

level of ESI (1, 2, 3, 4, 5)

10, 4

10, 4

10, 4

10, 4

3, 1

10, 4

10, 4

10, 4

10, 4

10, 4

3. ED

Number of patients occupied ED bed waiting to be dispositioned

41

41

41

41

41

41

54

41

48

41

Number of patients waiting to be dispositoined at ED bed with

2, 5, 18,

2, 5, 18,

2, 5, 18,

2, 5, 18,

2, 5, 18,

4, 8, 23,

2, 11, 25,

2, 5, 18,

1, 9, 22,

2, 5, 18,

different level of ESI (1, 2, 3, 4, 5)

10, 6

10, 6

10, 6

10, 6

10, 6

5, 1

10, 6

10, 6

10, 6

10, 6

Waiting time for the last pt put into bed (h)

1

1

1

1

1

1

0.5

1

1

1

Number of admit patients at ED

15

15

15

15

15

15

2

15

8

15

Number of admit patients at ED with different level of ESI

1, 7, 7, 0,

1, 7, 7, 0,

1, 7, 7, 0,

1, 7, 7, 0,

1, 7, 7, 0,

1, 7, 7, 0,

1, 1, 0, 0,

1, 7, 7, 0,

2, 3, 3, 0,

1, 7, 7, 0,

(1, 2, 3, 4, 5)

0

0

0

0

0

0

0

0

0

0

Ventilation patients

0

0

0

2

0

0

0

0

0

0

Longest time admit patient waiting at ED (h)

10

10

10

10

10

10

2.5

10

5

10

4. ED crowd status

References

  1. Bernstein SL, Asplin BR. Emergency department crowding: old problem, new solutions. Emerg Med Clin North Am 2006;24:821-37.
  2. Olshaker JS. Managing emergency department overcrowding. Emerg Med Clin North Am 2009;27:593-603.
  3. Hoot NR, Aronsky D. Systematic review of emergency department crowding: causes, effects, and solutions. Ann Emerg Med 2008;52:126-36.
  4. Weiss SJ, Derlet R, Arndahl J, Ernst AA, Richards J, Fernandez-Frackelton M, et al. Estimating the degree of emergency department overcrowding in academic medical centers: results of the National ED Overcrowding Study (NEDOCS). Acad Emerg Med 2004;11:38-50.
  5. Jones SS, Allen TL, Flottemesch TJ, Welch SJ. An independent evaluation of four quantitative Emergency Department Crowding Scales. Acad Emerg Med 2006;13: 1204-11.
  6. Weiss SJ, Ernst AA, Nick TG. Comparison of the national emergency department overcrowding scale and the emergency department work index for quantifying emergency department crowding. Acad Emerg Med 2006;13:513-8.
  7. Anneveld M, van der Linden C, Grootendorst D, Galli-Leslie M. Measuring emergency department crowding in an Inner city hospital in The Netherlands. Int J Emerg Med 2013;6:21.
  8. McCarthy ML, Aronsky D, Jones ID, Miner JR, Band RA, Baren JM, et al. The emergency department occupancy rate: a simple measure of emergency department crowding? Ann Emerg Med 2008;51:15-24.
  9. Richardson LD, Asplin BR, Lowe RA. Emergency department crowding as a health policy issue: past development, future directions. Ann Emerg Med 2002;40:388-93.
  10. Hwang U, McCarthy ML, Aronsky D, Asplin B, Crane PW, Craven CK, et al. Measures of crowding in the emergency department: a systematic review. Acad Emerg Med 2011;18:527-38.
  11. Weinstock MA, Colditz GA, Willett WC, Stampfer MJ, Rosner B, Speizer FE, et al. Recall (report) bias and reliability in the retrospective assessment of melanoma risk. Am J Epidemiol 1991;133:240-5.
  12. Friesema IH, Veenstra MY, Zwietering PJ, Knottnerus JA, Garretsen HF, Lemmens PH, et al. Measurement of lifetime Alcohol intake: utility of a self-administered questionnaire. Am J Epidemiol 2004;159:809-17.
  13. Trzeciak S, Rivers E. Emergency department overcrowding in the United States: an emerging threat to patient safety and public health. Emerg Med J 2003;20:402.
  14. McCarthy ML, Zeger SL, Ding R, Levin SR, Desmond JS, Lee J, et al. Crowding delays treatment and lengthens emergency department length of stay, even among high- acuity patients. Ann Emerg Med 2009;54:492-503.
  15. Derlet RW, Richards JR, Kravitz RL. Frequent overcrowding in US emergency departments. Acad Emerg Med 2001;8:151-5.
  16. Weiss SJ, Ernst AA, Derlet R, King R, Bair A, Nick TG, et al. Relationship between the National ED Overcrowding Scale and the number of patients who leave without being seen in an academic ED. Am J Emerg Med 2005;23:288-94.
  17. Bernstein SL, Aronsky D, Duseja R, Epstein S, Handel D, Hwang U, et al. The effect of emergency department crowding on clinically oriented outcomes. Acad Emerg Med 2008;16:1-10.
  18. Guttmann A, Schull MJ, Vermeulen MJ, Stukel TA. Association between waiting times and short term mortality and hospital admission after departure from emergency department: population based cohort study from Ontario, Canada. Br Med J 2011;342:D2983.
  19. Kurowski EM, Byczkowski T, Timm N. Return visit characteristics among patients who leave without being seen from a pediatric ED. Am J Emerg Med 2012;30: 1019-24.
  20. Rootman DB, Mustard R, Kalia V. Ahmed NPerceptions and realities of testing for alcohol and other drugs in trauma patients. J Trauma 2007;63(6):1370-3.
  21. Reijman M, Hazes JM, Koes BW, Verhagen AP, Bierma-Zeinstra SM. Validity, reliability, and applicability of seven definitions of Hip osteoarthritis used in epidemiological studies: a systematic appraisal. Ann Rheum Dis 2004;63(3): 226-32.
  22. Hobbs D, Kunzman SC, Tandberg D, Sklar D. hospital factors associated with emergency center patients leaving without being seen. Am J Emerg Med 2000;18: 767-72.
  23. Cardin S, Afilalo M, Lang E, Collet JP, Colacone A, Tselios C, et al. Intervention to decrease emergency department crowding: does it have an effect on return visits and Hospital readmissions? Ann Emerg Med 2003;41:173-85.
  24. Geelhoed GC, de Klerk NH. Emergency department overcrowding, mortality and the 4-hour rule in Western Australia. Med J Aust 2012;196:122-6.

Leave a Reply

Your email address will not be published. Required fields are marked *