Article, Emergency Medicine

Applying advanced analytics to guide emergency department operational decisions: A proof-of-concept study examining the effects of boarding

a b s t r a c t

Background: Emergency Department (ED) leaders are increasingly confronted with large amounts of data with the potential to inform and guide operational decisions. Routine use of advanced analytic methods may provide additional insights.

Objectives: To examine the practical application of available advanced analytic methods to guide operational de- cision making around Patient boarding.

Methods: Retrospective analysis of the effect of boarding on ED operational metrics from a single site between 1/ 2015 and 1/2017. Times series were visualized through decompositional techniques accounting for seasonal trends, to determine the effect of boarding on ED Performance metrics and to determine the impact of boarding “shocks” to the system on operational metrics over several days.

Results: There were 226,461 visits with the mean (IQR) number of visits per day was 273 (258-291). Decompo- sition of the boarding count time series illustrated an upward trend in the last 2-3 quarters as well as clear sea- sonal components. All performance metrics were significantly impacted (p b 0.05) by boarding count, except for overall Press Ganey scores (p b 0.65). For every additional increase in boarder count, overall length-of-stay (LOS) increased by 1.55 min (0.68, 1.50). Smaller effects were seen for waiting room LOS and treat and release LOS. The impulse responses indicate that the boarding shocks are characterized by changes in the performance metrics within the first day that fade out after 4-5 days.

Conclusion: In this study regarding the use of advanced analytics in daily ED operations, time series analysis pro- vided multiple useful insights into boarding and its impact on performance metrics.

(C) 2018

  1. Introduction

With greater adoption of electronic health records (EHR), Emergen- cy Department (ED) leaders are increasingly confronted with large amounts of data with the potential to inform and guide operational de- cisions [1,2]. In the past decade numerous operational measures of emergency care have been developed [3], tied to physician and hospital payment incentives [4], and required by national public quality reporting programs such as the Centers for Medicare and Medicaid Ser- vices [5]. Specifically, measures of ED throughput including the timeli- ness of provider evaluation, discharge, admission, and ED boarding have become common parlance among ED leaders and frequent out- comes of focused interventions [4,6,7]. These measures, however have

* Corresponding author.

E-mail address: [email protected] (R. Andrew Taylor).

historically been examined and targeted for improvement in a silo without acknowledging the complexity and apparent inter-related nature of ED throughput, ED crowding, and outcomes such as patient satisfaction [8,9]. While traditional summary statistics (e.g. measures of central tenden-

cy, variation) and data visualization techniques (e.g. bar graphs, scatter plots) continue to have a key role in guiding decisions around improving operational metrics, routine use of more advanced analytic techniques, such as time series and regression analysis, have the potential to provide new insights for operational leaders into the complex inter-relationships of the metrics, uncover trends, weight the relative importance of key metrics, and test interventions [10-12]. Additionally these analytic tech- niques are becoming increasingly user-friendly with drag-and-click graphical user interface implementations in several common statistical and visualization software packages (e.g. R, STATA, Matlab, python).

Assessment of the impact of boarding, a situation that occurs when insufficient hospital capacity is available for admitted patients to be

https://doi.org/10.1016/j.ajem.2018.01.011

0735-6757/(C) 2018

transferred to inpatient beds, is an area where the use of advanced ana- lytics is potentially beneficial for operations leaders. Boarding is known to be, at the aggregate level, associated with numerous operational throughput metrics (e.g., length-of-stay, LWBS, door-to-doc), patient satisfaction, and patient outcomes [13]. Quantifying the effect of boarding on daily ED performance metrics through time series analysis would allow for meaningful interpretation (e.g. for every X boarders the length-of-stay increases by X minutes) and adjustment for other factors that may effect performance metrics such as daily visit volume.

We therefore sought to examine the potential application of readily available time series analysis methods to common ED operational met- rics to guide decision making and administrative discussion around pa- tient boarding.

  1. Methods
    1. Study design

Retrospective analysis of the effect of boarding on ED operational metrics. The study adhered to the STROBE guidelines and checklist for observational research [14]. This study resulted from a quality improve- ment project and was waived by the institutional IRB.

Study setting

Single site urban, academic, Level I trauma center with an annual census of approximately 90,000 patients with a single vendor EHR. Data pertaining to visits was obtained from a centralized data ware- house between January 2015 and January 2017.

Definitions and outcomes

For the purposes of analysis, a boarded patient was defined as a pa- tient who remained in the emergency department after the patient had been admitted (determined by time of admission order) to the facility for 4 h, but had not been transferred to an Inpatient unit [15]. We exam- ined the impact of boarding on several operational metrics defined below. Throughput operational metrics adhered to the definitions outlined by Welch et al. [3] and included: Waiting Room Length of Stay (LOS), Treat and Release LOS, Overall LOS, Percentage of Patients Left-Without-Being-Seen (LWBS), and the percentage of patients who walked out. Overall patient satisfaction scores were collected from Press Ganey only on discharged patients.

Analysis

Data analysis included descriptive statistics on ED performance met- rics and boarding. Continuous data are presented as means and 95% confidence intervals (CIs). All analyses were conducted based on daily time intervals, except those involving patient satisfaction scores which were analyzed on a weekly basis secondary to lack of daily data. Boarding data was aggregated to a count of the total number of boarders per day, LOS metrics were aggregated to median values, LWBS and Walkouts were calculated as percentages. Patient satisfaction scores were averaged over the week time frame.

Time series are often viewed as having various components that re- flect seasonal patterns, general trends in the data, and noise. A trend ex- ists when there is an increasing or decreasing direction in the data, while seasonal patterns are cyclical patterns over a fixed, known period (daily, weekly, etc.). Separating and visualizing these components, a process called time series decomposition, can be extremely helpful to the decision maker [16]. For the study, we demonstrate the decomposi- tion of the boarding count daily data with using the seasonal trend de- composition Loess algorithm. [17].

To determine the effect of boarding on ED performance metrics and to develop meaningful adjustment factors (e.g. for every X boarders the

length-of-stay increases by X minutes), time series analysis was per- formed using univariate autoregression integrated moving averages (ARIMA) models with external regressors [18]. For each operational metric a separate ARIMA model was constructed that included boarding count and number of visits per day as external regressors. The adequacy of each model was analyzed using autocorrelation functions and periodograms. Stationarity of the models was examined by using the Dickey-Fuller and Phillips-Perron unit root tests, while Portmanteau statistics were used to determine if any autocorrelation remained in the residuals of the model [18]. Final model selection was determined by a search over model parameters for the best performing model ac- cording to the Akaike information criteria (AIC).

We constructed multivariable vector auto-regression moving aver- age models (VARMA) to determine the impact of boarding “shocks” to the system on operational metrics over several days. Time series vari- ables included the operational metric, boarding count, and visits per day. Unlike a simple structural regression model, where the relationship between dependent and independent variables is assumed static, the VARMA modelling offers a more dynamic approach to the nature of in- teraction between variables via a system of autoregressive equations [19]. The advantage of this system is that it is able to estimate the impact of changes in one variable on the other variable not only at a point of time, but also over period of time. Impulse response functions, a function that describes the cascade of changes in a variable due to an unexpected shock in another variable, were calculated for each performance metric to demonstrate the potential multiday effect of boarding spikes [20].

All data were analyzed using R (R Foundation for Statistical Comput-

ing, Vienna, Austria.).

  1. Results

There were 226,461 visits with a mean number of visits per day of 273 (IQR 258-291). The mean boarding count per day was 19.5 (IQR 9-26) with a maximum of 79. The daily mean (IQR) for Overall LOS, Treat and Release LOS, and Waiting Room LOS was 266 (IQR 240-288), 228 (IQR 204-250), and 8.2 [5-9], respectively. The mean LWBS percent- age was 3.4 (IQR 1.2-5.1) and walkout percentage was 4.9 (IQR 2.8-6.8). with 4437 (2.0%) returned Press Ganey surveys during the study period The mean weekly Press Ganey score was (82.8 IQR 78-88). Times series graphs of key operational metrics are presented in Fig. 1.

Decomposition of the boarding count time series is demonstrated in Fig. 2 and illustrates an upward trend in the last 2-3 quarters as well as clear seasonal components.

The results of the ARIMA models with the effect of boarding counts on performance metrics are presented in Table 1. ARIMA model types and the number of Fourier terms as determined by AIC varied for each performance metric. All performance metrics were significantly impacted (p b 0.05) by boarding count, except for overall Press Ganey scores (p = 0.65). For every additional increase in boarder count, overall LOS increased by 1.55 min (0.68, 1.50). Smaller effects were seen for Waiting Room LOS and Treat and Release LOS. The percentage LWBS and Walkouts changed by 0.07% for every additional increase in boarder count.

The impulse responses for a boarding count shock for each of the performance metrics are demonstrated in Fig. 3. The impulse responses indicate that the boarding shock of 1 patient is characterized by an in- crease in the performance metrics within the first day that fades out after approximately 4-5 days.

  1. Discussion

In this proof of concept study examining the use of advanced analyt- ics in daily ED operations, time series analysis provided multiple in- sights into boarding and its impact on performance metrics. Greater clarity and understanding of the effect of boarding on emergency de- partment operations and performance is critical, as our specialty

Fig. 1. Time series graphs of boarding count, daily number of visits, and key operational metrics. This figure illustrates the difficulty of discerning trends and periodic cycles in real-world Operational data without the use of advanced analytics.

continues to face the increase burden of ED volume with decreasing inpatient capacity. Decomposition of the boarding time series unmasked a late trend in boarding and demonstrated seasonal variation. ARIMA modelling of performance metrics with boarding counts and number of visits demonstrated a quantifiable impact of boarding on performance metrics and impulse response function calculation and plotting demonstrated that boarding not only effects an individual day but has ripple effects multiple days out from the initial shock to the system.

Emergency Department operational decisions are increasingly data- driven. Using only a subjective visual analysis of raw time series data to look for trends, associations, and periodic patterns is fraught with errors [21]. Decomposition of time series analysis, represents one advanced analytic method, that demonstrates seasonal patterns, trends, and the underlying effects within a time series analysis [16]. Fig. 1 highlights these advantages by demonstrating both seasonality and trend in the time series that is not readily perceptible on visual inspection. While methods of decomposition have existed and have been utilized in the ED research for some time [22-24], we believe now is the time, with available support from multiple user-friendly platforms, to move these

methods from the research arena to daily practical use and advocate presenting time-series data in this format to reduce inferential errors.

ARIMA modelling of performance metrics using boarding counts and adjusting for number of visits demonstrated a quantifiable impact of boarding on performance metrics. For example, on average, for every additional boarder per day there was 1.5 min increase in the overall LOS. Prior studies on the effects of boarding on operational metrics dem- onstrate similar adverse effects for individual metrics, [25,26] however our study breaks down these effects into easily understandable units while adjusting for patient volume/census. Applying these results to the adjustment of performance targets at our institution, assuming a target of 240 min for a day with 30 boarders, the new target would be 285 min. This knowledge could then be used by operational leaders in hospital performance negotiations, to instruct operational change, and guide policy.

To our knowledge this is the first study examining ED operations that uses impact functions to examine the multi-day effect of a shock (boarding counts) to the system. Intuitively it makes sense that in- creasED boarding times one day is correlated with increased boarding the next day, and that boarders likely have an impact on an ED system

Fig. 2. Time series decomposition algorithms break, or decompose, the time series into several parts. The figure displays the general trend in the data, the seasonal (cyclical) component(s), and residual noise. Presenting time series by decomposition facilitates visual analysis and protects against inferential errors.

for more than day. Demonstrating this link, however, requires advanced analytics. Our results demonstrate this effect and for our ED it appears to drop off rapidly after one day while retaining some effect for multiple days. This information may lead to enhanced support for ED leadership to negotiate operational changes and additional resource allocation to address prolonged and evolving boarding issues. We caution though that these effects should be examined on an ED-by-ED basis as different workflow and operational environments may result in different effects.

  1. Limitations

The current study has several limitations. First, we only examined a single site and the results regarding the effects are not necessarily gen- eralizable to other EDs. However, the purpose of this study was focused on advocating the use of advanced analytic methods that are locally ap- plicable. Second, we recognize that not all EDs currently have the

infrastructure or personnel to support the use of advanced analytics. Our results though demonstrate the ability of advanced analytics to pro- vide valuable information and we hope to convey that many of these methods can be implemented through user-friendly platforms. Last, in our ARIMA models we only controlled for visit count and it is conceiv- able that other variables if controlled for would lessen the effect of boarding on key operational metrics.

  1. Conclusions

In this proof of concept study regarding the use of advanced analyt- ics in daily ED operations, time series analysis provided multiple useful insights into boarding and its impact on performance metrics, These re- sults support the value and need for the investment and utilization of advanced analytics by ED operations leaders.

Table 1

Effect of boarding and number of visits and operational metrics.

Operational metric Boarding (95%CI) Number visits (SE) Model typea Overall length of stay (mins) 1.55 (1.38-1.71) 0.34 (0.27-0.42) ARIMA(1,1,1)

Treat and release 0.71 (0.63-0.78) 0.45 (0.41-0.49) ARIMA(1,1,1)

LWBS (%) 0.07 (0.059-0.084) 0.045 (0.038-0.051) ARIMA(1,1,1)

Waiting room length of stay (min) 0.11 (0.091-0.15) 0.059 (0.049-0.79) ARIMA(1,1,1)

Triage length of stay (min) 0.66 (0.51-0.86) 0.47 (0.38-0.55) ARIMA(0,1,2)

Walkout (%) 0.073 (0.59-0.86) 0.047 (0.040-0.53) ARIMA(1,1,2)

Press Ganey overall score 0.032 (-0.011-0.172) 0.03 (-0.063-0.12) ARIMA(4,0,1)

a Model type ARIMA (p, q, d) where p is the order (number of time lags), d is the degree of differencing (the number of times past values have been subtracted), and q is the order of the moving average model.

Fig. 3. Plots of impulse response functions for key operational metrics affected by boarding shocks. The y axis for each metric represents the change in that metric for every 1 additional boarder (i.e. the shock to the system). The x axis represents the number of days out from the initial shock.

Meetings

None.

Grant or other financial support

None.

Conflicts of interest

None.

Author contributions

All authors contributed to the conception and design of the study. RAT and AU supervised the study. RAT analyzed the data. RAT drafted the manuscript, and all authors contributed to its revision. RAT takes re- sponsibility for the paper as a whole.

References

  1. Knepper MM, Castillo EM, Chan TC, Guss DA. The effect of access to electronic health records on throughput efficiency and imaging utilization in the emergency depart- ment. Health Serv Res 2017. https://doi.org/10.1111/1475-6773.12695.
  2. Ben-Assuli O. Electronic health records, adoption, quality of care, legal and privacy issues and their implementation in emergency departments. Health Policy 2015; 119:287-97.
  3. Welch SJ, Asplin BR, Stone-Griffith S, et al. Emergency department operational met- rics, measures and definitions: results of the Second performance measures and Benchmarking Summit. Ann Emerg Med 2011;58:33-40.
  4. Schuur JD, Hsia RY, Burstin H, Schull MJ, Pines JM. Quality measurement in the emer- gency department: past and future. Health Aff 2013;32:2129-38 (Millwood).
  5. Chang AM, Lin A, Fu R, McConnell KJ, Sun B. Associations of emergency department length of stay with publicly reported quality-of-care measures. Acad Emerg Med 2017;24:246-50.
  6. Stang AS, Crotts J, Johnson DW, Hartling L, Guttmann A. Crowding measures associ- ated with the quality of emergency department care: a systematic review. Acad Emerg Med 2015;22:643-56.
  7. Milsten A, Klein J, Liu Q, Vibhakar N, Linder L. Retrospective analysis of quality im- provement throughput measures at a high-volume community emergency depart- ment. J Healthc Qual 2014;36:13-25.
  8. Sun BC, Hsia RY, Weiss RE, et al. Effect of emergency department crowding on out- comes of admitted patients. Ann Emerg Med 2013;61:605-11 [e6].
  9. Melton 3rd JD, Blind F, Hall AB, Leckie M, Novotny A. Impact of a hospital wide qual- ity improvement initiative on emergency department throughput and crowding measures. Jt Comm J Qual Patient Saf 2016;42:533-42.
  10. Chatfield C. The analysis of time series: An introduction. 6th ed. Chapman & Hall/ CRC: Boca Raton, FL; 2004.
  11. Rathlev NK, Chessare J, Olshaker J, et al. Time series analysis of variables associated with daily mean emergency department length of stay. Ann Emerg Med 2007;49: 265-71.
  12. Wiler JL, Griffey RT, Olsen T. Review of modeling approaches for emergency de- partment patient flow and Crowding research. Acad Emerg Med 2011;18: 1371-9.
  13. Pines JM, Prabhu A, Hilton JA, Hollander JE, Datner EM. The effect of emergency de- partment crowding on length of stay and medication treatment times in discharged patients with acute asthma. Acad Emerg Med 2010;17:834-9.
  14. von Elm E, Altman DG, Egger M, et al. The strengthening the reporting of observa- tional studies in epidemiology (STROBE) statement: guidelines for reporting obser- vational studies. Int J Surg 2014;12:1495-9.
  15. Definition of Boarded Patient. (Accessed August 10, 2017, at https://www.acep.org/ Clinical–Practice-Management/Definition-of-Boarded-Patient-2147416349/#sm. 000110tbbt6nsf1xq6v145xt2bijz.
  16. Walker LR, Greenawald E. Forecasting operations with time series decomposition. Hosp Financ Manage 1979;33(26):8-30 (2).
  17. Cleveland RB, Cleveland WS, McRae JE, Terpenning I. STL: a seasonal-trend decom- position procedure based on loess. J Off Stat 1990;6:3-73.
  18. Hyndman RJ, Khandakar Y. Automatic time series forecasting: the forecast package for R. J Stat Softw 2008;27:1-22.
  19. Lutkepohl H. Chapter 6 forecasting with VARMA models. In: Elliott G, Granger CWJ, Timmermann A, editors. Handbook of economic forecasting. Elsevier; 2006.

p. 287-325.

  1. Lutkepohl H. Impulse response function. Macroeconometrics and time series analy- sis. Springer; 2010. p. 145-50.
  2. Farnum NR, Stanton LW. Quantitative forecasting methods. Boston: PWS-Kent Pub. Co.; 1989.
  3. Tandberg D, Qualls C. Time series forecasts of emergency department patient vol- ume, length of stay, and acuity. Ann Emerg Med 1994;23:299-306.
  4. Kadri F, Harrou F, Chaabane S, Tahon C. Time series modelling and forecasting of emergency department overcrowding. J Med Syst 2014;38:107.
  5. Jones SA, Joy MP, Pearson J. Forecasting demand of emergency care. Health Care Manag Sci 2002;5:297-305.
  6. White BA, Biddinger PD, Chang Y, Grabowski B, Carignan S, Brown DF. Boarding in- patients in the emergency department increases discharged patient length of stay. J Emerg Med 2013;44:230-5.
  7. Hoot NR, Aronsky D. Systematic review of emergency department crowding: causes, effects, and solutions. Ann Emerg Med 2008;52:126-36.

Leave a Reply

Your email address will not be published. Required fields are marked *