a b s t r a c t

Objectives: Emergency Department crowding is an increasing problem, leading to treatment delays and higher risk of mortality. Our institution recently implemented a telemedicine physician intake (“tele-intake”) process as a mitigating front-end strategy. Previous studies have focused on ED throughput metrics such as door to dis- position; our work aimed to specifically assess the tele-intake model for clinical accuracy.

Methods: We retrospectively reviewed ED visits at a high acuity, tertiary care academic hospital before and after tele-intake implementation. We defined the primary outcome as the degree of additional laboratory, imaging, and Medication orders placed by the subsequent ED provider. Our secondary outcomes were the cancellation rate of intake orders and the percentage of encounters where no additional second provider orders were neces- sary.

Results: For in-person and tele-intake physician encounters between September 2015 and February 2017, most labs and diagnostic radiology studies, and approximately half of CT, ultrasound, and pharmacy orders were initi- ated by the intake physician. We found no significant difference for our primary outcome (p = 0.2449). For both tele-intake and in-person encounters, b1% of orders were cancelled by the second provider. Additionally, 30.8% of in-person and 31.5% of telemedicine patient encounters required no additional orders to make a disposition de- cision.

Discussion: This novel analysis of an innovative patient care model suggests that the benefits of tele-intake as a replacement for in-person physician directed intake are not at the cost of over or under utilization of diagnostic testing or interventions.

(C) 2018

Introduction

Emergency Department (ED) crowding has become an increasingly prominent issue in the United States over the last several decades. In 2002, a national survey revealed that 90% of large hospitals reported their EDs operated at or over capacity [1]. Subsequent studies revealed that overcrowding may lead to increased mortality risk, decreased pa- tient satisfaction, and Treatment delays for high-acuity, time-sensitive conditions [1-3]. Despite the concerns that have been raised, as well as changes to healthcare policy, the ED remains a safety net for much of the American population.

? Declaration of interest: None.Presentations: ACEP Scientific Assembly. October 31st 2017. Washington, DC.

* Corresponding author at: 110 Irving Street NW, Suite NA 1009, Washington, DC 20010, United States of America.

E-mail address: [email protected] (J.A. Izzo).

Over the past several years, several strategies have been suggested to mitigate these risks including utilization of Observation Units [3], changes to the triage process, increased staffing [4], and encouraged use of primary care [5]. Additionally, several front-end strategies such as bedside registration, fast-track services, and provider in triage have demonstrated some improvement in operational metrics [6-8].

Since 2009, our ED has had a Physician in triage, allowing for im- proved metrics such as door-to-provider time and a decreased left with- out being seen rate, as well as providing therapeutic interventions in a more timely fashion. In 2016, our hospital introduced a novel telemed- icine triage system for front-end evaluation and initiation of treatment of ED patients. A prior study of this intervention demonstrated that the telemedicine triage model was associated with significantly lower time for ED arrival to physician evaluation [10]. Our current study eval- uates the accuracy of evaluation and interventions initiated by a physi- cian participating in the telemedicine model (“tele-intake”) when compared with an in-person physician model of triage.

https://doi.org/10.1016/j.ajem.2018.08.022

0735-6757/(C) 2018

2062 J.A. Izzo et al. / American Journal of Emergency Medicine 36 (2018) 2061-2063

Methods

Study setting and design

This front-end quality improvement initiative was conducted at a high-acuity, adult, tertiary care, level 1 trauma center with an Emer- gency Medicine residency program and was approved by the Institu- tional Review Board. We retrospectively reviewed visits for 6 months before and after implementation of a telemedicine physician intake model.

Our intake physicians see and examine patients during our highest volume hours (11 am-8 pm on weekdays) and place orders based on their assessment. A designated intake area with nurses and technicians allows diagnostics and medication orders to be carried out while the pa- tient waits to be brought back to the treatment team. Formerly, our in- take physicians were present in person during the same hours (11 am- 8 pm); after September 2016, a telemedicine platform allowed for as- sessments from a designated office within the hospital with audio/ video capabilities as well as ready access to medical records. In-person intake physicians typically worked on a single monitor mobile worksta- tion while interviewing patients. In the designated office, a dual-screen workstation with webcam and microphone access was available to the intake physician. On the front-end, a mobile tablet interface was used to interview patients, operated by the triage nurse. There was no change to the intake area nurse and tech staffing, and the physical space desig- nated for the intake workup remained unchanged after transitioning to tele-intake.

We examined all patient encounters where an intake physician was

present, looking at in-person encounters from September 2015 to Feb- ruary 2016, and tele-intake encounters from September 2016 to Febru- ary 2017. We chose to match months of the year between study periods to account for seasonal variability in patient acuity and diagnoses.

Data collection

For each patient encounter, we extracted all physician orders placed using SQL queries to an Oracle Database connecting to the Cerner Mil- lennium electronic health record database. Orders were catego- rized by lab test, X-ray, CT, MRI, ultrasound, and pharmacy. Any other orders such as patient status orders, diet, admission, or similar were re- moved from analysis. Additionally, we excluded orders discontinued by the same provider who placed them, duplicate orders (e.g., an order for a second troponin), or inpatient orders. A priori subgroup analysis for each outcome was performed by order category.

Outcomes

The primary outcome was the rate at which additional orders were deemed necessary by the subsequent ED provider, which we used as a surrogate for diagnostic accuracy. The secondary outcomes included the order cancellation rate of the intake physician’s orders by the second provider and the intake disposition rate, defined as the percentage of encounters in which no additional orders were necessary by the second provider for a disposition.

Data analysis

Data processing and statistics were performed using the Python Pandas and STATA (Version 14; StataCorp LP, College Station, Texas) software packages. Two-sample Wilcoxon rank-sum (Mann-Whitney) testing was used to assess our primary outcome and the order cancella- tion rate (alpha = 0.05). We applied Bonferroni’s adjustment to our subgroup analysis to account for multiple comparisons and defined sig- nificance a priori at p b 0.0083. Pearson’s chi square was used for the in- take disposition rate (alpha = 0.05).

Results

For in-person and tele-intake physician encounters between Sep- tember 2015 and February 2017, there were 7326 and 6586 encounters, respectively. Demographics are presented in Table 1. No statistically sig- nificant differences were found for ED patient volumes, ESI levels, Boarding hours, gender, or race. During the tele-intake time period, more patients arrived by ambulance and more patients were admitted. 327 and 296 respective encounters during the in-person and tele-intake time periods had no orders placed by either provider, and 407 and 217 respective encounters had no second provider. The majority of patients with no second provider left prior to completion of treatment against medical advice or via elopement, though rarely, a patient was discharged from the intake waiting room. There were 6577 and 6060 patient visits, respectively, where a subsequent ED provider completed the patient’s care. There were 27,256 and 27,325 unique orders initiated by the in-person intake and tele-intake physician, respectively, and 13,546 and 12,229 respectively by the second provider. The majority of lab and X-ray orders, and approximately half of CT, ultrasound, and pharmacy orders were initiated by the intake physician (Table 2). Mann-Whitney testing showed no significant difference in our overall primary outcome, the number of orders per encounter added by the second provider (p = 0.2449). Subgroup analysis showed no difference by order type except for pharmacy orders (p = 0.0072); significantly more medication orders were added when an in-person physician saw the patient (8072 orders [1.23 orders/encounter] vs. 6989 orders [1.15 orders/encounter], respectively). For both tele-intake and in- person encounters, b1% of all orders were cancelled by the second pro- vider, and Mann-Whitney testing demonstrated an overall significant difference (p = 0.0008). On subgroup analysis, only cancelled pharmacy orders for in-person versus telemedicine providers were significant (p

= 0.002, 63 orders [0.0096 orders/encounter] vs. 95 [0.016 orders/en-

counters], respectively). For 30.8% of in-person and 31.5% of telemedi- cine patient encounters, no additional orders were necessary to make a Disposition decision. This difference was not significant (p = 0.4139; Chi-square).

Table 1

Demographic data for in-person versus telemedicine intake encounters, with percentages in parentheses. Welsh’s two sample t-test was used for our continuous data (ED volume and boarding hours per day); Pearson’s chi square was used for our categorical data (ESI, gender, race, disposition, and mode of arrival)

In-person Tele-intake p value

ED volume (average # registrations per day)	252.5	253.6	0.743
Boarding hours per day (average)	534.0	567.9	0.223
ESI level (%)
1	3 (0)	0 (0)	0.136
2	2501 (31.1)	2198 (33.4)
3	4623 (63.1)	4183 (33.4)
4	174 (2.4)	187 (63.5)
5	12 (0.2)	5 (0.1)
Not listed	13 (0.2)	13 (0.2)
Gender (%)
Male	2990 (40.8)	2706 (41.1)	0.743
Female	4336 (59.2)	3880 (58.9)
Race (%)
African-American	5809 (79.2)	5254 (79.8)	0.285
White	919 (8.2)	772 (8.5)
Other	598 (12.6)	560 (11.7)
Patient disposition (%) Discharged	4695 (38.3)	4368 (36.7)	0.006
Admitted	2631 (61.7)	2218 (63.3)
Mode of arrival (%)
Ambulance	1498 (20.5)	1508 (22.9)	b0.001
private vehicle/walk-in	5289 (72.2)	4656 (70.8)
Other	539 (7.3)	422 (6.3)

J.A. Izzo et al. / American Journal of Emergency Medicine 36 (2018) 2061-2063 2063

Table 2

Comparison of order data for in-person versus tele-intake encounters organized by second physician orders (primary outcome), and discontinued orders (secondary outcome). For the second provider orders, the values in parentheses for each subgroup represent the proportion of orders placed by the second provider compared to the total orders (intake physician plus second provider). p values, with a priori significance defined as p b 0.0083 by Bonferroni’s adjustment, represent comparisons between order totals

Second physician orders p-Value

In-person encounters Tele-intake encounters Order totals Mean/Enc Order totals Mean/Enc

encounters, whereas 1083 additional pharmacy orders were placed after in-person intake encounters. We suspect our dedicated telemedi- cine office may assist with synthesizing clinical data in the limited time available for intake encounters both through the dual-screen inter- face and by significantly reducing distractions. Additionally, nurses present initial details about the patient’s presentation which further op- timizes physician time spent with each patient. The added cost of the telemedicine equipment was b5% of the labor cost of the physician, nurse, and patient care technician intake team.

Our study is limited by its retrospective, single-center design, and

CT	982 (51.7)	0.149	953 (54.0)	0.157	0.179	would be strengthened by a multicenter Prospective analysis. We
X-ray	771 (23.1)	0.117	664 (21.4)	0.110	0.130	tried to account for between-group variation by matching months to ac-
Lab MRI	3360 (16.0) 150 (72.5)	0.511 0.023	3276 (15.2) 142 (79.8)	0.541 0.023	0.250 0.600	count for seasonal variability. Additionally, cultural factors may also
Pharmacy	8072 (58.1)	1.227	6989 (55.5)	1.153	0.007	limit our results. A second provider may be hesitant to add to or cancel
Ultrasound	211 (49.2)	0.032	205 (55.3)	0.034	0.469	the orders of the first provider due to departmental culture or a per-
Total	13,546	2.060	12,229	2.018	0.245	ceived authority bias. Also, diagnostic momentum may cloud the clinical

Discontinued orders p-Value

In-person encounters Tele-intake encounters

	Order totals	Mean/Enc		Order totals	Mean/Enc
CT	27	0.004		23	0.004	0.768
X-ray	24	0.004		37	0.006	0.061
Lab	64	0.010		92	0.015	0.102
MRI	1	b0.001		0	0.000	0.337
Pharmacy	63	0.010		95	0.016	0.002
Ultrasound	3	b0.001		7	0.001	0.163
Total	182	0.028		254	0.042	0.001

Discussion

Our study demonstrated comparable accuracy for diagnostic workup between tele-intake and in-person models of physician in tri- age. A 2013 study evaluating telemedicine intake physician accuracy similarly demonstrated comparable accuracy between nurse triage as- sessment and telemedicine physician assessment [9]. Additionally, that study demonstrated a range of Kappa values of 0.56 to 0.83 when comparing orders from telemedicine triage physician to an in-person second physician. However, to date there has not been a study assessing the accuracy of an in-person intake physician compared with a telemed- icine intake physician.

Here, we present a unique analysis of an innovative patient care model, specifically focused on the Ordering patterns of in-person versus tele-intake providers. We defined a surrogate for diagnostic accuracy as our primary outcome and found no difference overall between in- person and tele-intake encounters. There was also no difference in the intake disposition rate; approximately 31% of all encounters required no orders from the second provider.

While statistically more pharmacy orders were cancelled for tele- medicine encounters, b1% of orders were cancelled overall in our analysis, and the difference was far less striking when compared to the diagnostic accuracy noted for pharmacy orders. Only 32 more phar- macy orders were cancelled by the second provider in telemedicine

judgement of the second provider seeing a patient. Operational factors may be present as well. For example, at times of significant ED crowding, a provider may not see the patient in time to cancel or modify an intake order, which may significantly dilute the order cancellation rate. Still, we noted no significant difference in the time to second pro- vider for both groups [10].

Conclusions

This study suggests that the operational benefits of tele-intake as a replacement for in-person physician-directed intake are not at the cost of clinically significant over or under utilization of diagnostic test- ing or interventions. Further prospective evaluations may strengthen our results to include assessment of outcome data.

References

1. Lewin Group (for the American Hospital Association). Emergency department over- load: a growing crisis. The results of the American Hospital Association Survey of Emergency Department (ED) and Hospital Capacity. Falls Church, VA: American Hospital Association; 2002.
2. Sun BC, Hsia RY, Weiss RE, Zingmond D, Liang LJ, Han W, et al. Effect of Emergency Department crowding on outcomes of admitted patients. Ann Emerg Med 2013 June;61(6):605-11 (e6).
3. Trzeciak S, Rivers EP. Emergency department overcrowding in the United States: an emerging threat to patient safety and public health. Emerg Med J 2003;20:402-5.
4. Hoot NR, Aronsky D. Systematic review of emergency department crowding: causes, effects, and solutions. Ann Emerg Med 2008;52:126-36.
5. Blumenthal D, Abrams M, Nuzum R. The Affordable Care Act at 5 years. N Engl J Med 2015;372:2451-8.
6. Wiler JL, Gentle C, Halfpenny JM, et al. Optimizing emergency department front-end operations. Ann Emerg Med 2010;55:142-60 (e1).
7. Choi YF, Wong TW, Lau CC. Triage rapid initial assessment by doctor (TRIAD) im- proves waiting time and processing time of the emergency department. Emerg Med J 2006;23:262-5.
8. Subash F, Dunn F, McNicholl B, et al. Team triage improves emergency department efficiency. Emerg Med J 2004;21:542-4.
9. Watson J, Bhat R, Izzo JA, et al. Telemedicine model of physician intake decreases door-to-provider time. Ann Emerg Med 2017 Oct;70(4):S128-9.
10. Marconi GP, Chang T, Pham PK, Grajower DN, Nager AL. Traditional nurse triage vs physician telepresence in a pediatric ED. Am J Emerg Med 2014 Apr;32(4):325-9.