a b s t r a c t

Background: Due to limited community resources for mental health and long travel distances, emergency depart- ments (EDs) serve as the safety net for many rural residents facing crisis mental health care. In 2019, The Leona

M. and Harry B. Helmsley Charitable Trust funded a project to establish and implement an ED-based telepsychiatry service for patients with Mental health issues in underserved areas. The purpose of this study was to evaluate the implementation of this novel ED-based telepsychiatry service.

Methods: This was a mixed-methods study evaluating the new ED-based telepsychiatry consult service imple- mented in five EDs across three rural states that participated within a mature hub-and-spoke telemedicine net- work between June 2019 and December 2020. Quantitative evaluation in this study included characteristics of the telehealth encounters and the patient population for whom this service was used. For qualitative assess- ments, we identified key themes from interviews with key informants at the ED spokes to assess overall facilita- tors, barriers, and impact. Integrating the quantitative and qualitative findings, we explored emergent phenomena and identified insights to provide a comprehensive perspective of the implementation process.

Results: There were 4130 encounters for 3932 patients from the EDs during the evaluation period. Approximately

54% of encounters involved female patients. The majority of patients seen were white (51%) or Native American (44%) reflecting the population of the communities where the EDs were located. Among the indications for the telepsychiatry consult, the most frequently identified were depression (28%), suicide/self-harm (17%), and schizophrenia (12%). Across sites, 99% of clinician-to-clinician consults were by phone, and 99% of clinical assess- ments/evaluations were by video. The distribution of encounters varied by the day of the week and the time of day. Facilitators for the service included increasing need, a Supportive infrastructure, a straightforward process, familiarity with telemedicine, and a collaborative relationship. Barriers identified by respondents at the sites in- cluded the lack of clarity of process and technical limitations. The themes emerging from the impact of the telepsychiatry consultation in the ED included workforce improvement, care improvement, patient satisfaction, cost-benefit, facilitating COVID care, and access improvement.

Conclusions: Implementation of a telepsychiatry service in ED settings may be beneficial to the patient, local ED,

and the underserved community. In this study, we found that implementing this service alleviated the burden of care during the COVID-19 pandemic, enhanced local site capability, and improved local ability to provide quality and effective care.

(C) 2022

1. Background

* Corresponding author at: 200 Hawkins Drive, Suite GH C132, Iowa City, IA 52242, USA.

E-mail address: [email protected] (J.P. Vakkalanka).

Rural counties have nearly six times fewer mental health profes- sionals per capita and a suicide rate that is 45% greater than urban or large metropolitan counties [1,2]. Consequently, many rural residents access crisis mental health care in local emergency departments

https://doi.org/10.1016/j.ajem.2022.06.053

0735-6757/(C) 2022

(EDs), which serve as a safety net for emergency psychiatric care [3]. However, because local resources are unavailable, these patients may not receive complete mental health evaluations, may be transferred to another hospital for evaluation and management, or may be transferred unnecessarily [4,5].

Telepsychiatry offers an innovative approach to expanding access to mental health care in remote settings. One form of telepsychiatry that may particularly benefit rural and underserved EDs is clinician-to- clinician tele-consultation [6-8]. When telemedicine is used in the ED, patient and clinical quality metrics have improved in trauma, sepsis, mental health, and other conditions [9-13]. Even though these findings are well documented, less is known about the implementation process and perceptions of a telepsychiatry-specific service in the ED. In 2019, The Leona M. and Harry B. Helmsley Charitable Trust funded a project to establish and implement an ED-based telepsychiatry service for hos- pitals in underserved areas in the upper Midwest. The overall goal of this effort was to establish an interdisciplinary team of psychiatrists and behavioral health nurses to provide telepsychiatry consultation to local clinicians in underserved areas for patients with mental health emergencies (e.g., suicidal ideation, depression, anxiety, overdose). The purpose of this study was to evaluate the novel ED-based telepsychiatry service to explore the experience of implementing telepsychiatry services and to identify the utilization, barriers, and facil- itators in these sites.

1. Methods
   1. Overview of study setting and participants

This study was a mixed-methods evaluation of a new telepsychiatry consult service implemented in five EDs that participated within a mature hub-and-spoke telemedicine network between June 2019 and December 2020. We used a cohort analysis of consultation program utilization combined with a qualitative design to explore emergent phe- nomena and identify insights to provide a comprehensive perspective [14,15]. We were interested in both encounter-level and patient-level characteristics of patients for whom telepsychiatry services were used, and we elicited key qualitative themes to assess overall facilitators, bar- riers, and impact. This study was deemed not human subject research by our local Institutional Review Board and is reported using the Consol- idated Criteria for Reporting Qualitative Research (COREQ) for reporting purposes [16].

• 1. ED-based telepsychiatry service

Avera eCARE (now known as Avel eCare) is based in Sioux Falls, SD and has operated a multispecialty telemedicine network since 1993. The on-demand hub-and-spoke network now provides ED-based ser- vices to >200 hospitals in 14 states and connects local ED clinicians promptly with hub emergency physicians using a high-definition video connection and/or telephone consultation [17,18]. The telepsychiatry service evaluated here was a new expansion to provide telepsychiatry consultations from an interdisciplinary team of psychia- trists and behavioral health nurses with ED staff at spoke sites. Imple- mentation of telepsychiatry services included in the study occurred in five EDs across three states (Iowa, Minnesota, and South Dakota). All of these hospitals are located in federally defined health professional shortage areas (either primary care or mental health or both) and four are designated as rural [19].

• 1. Quantitative methods

We obtained patient-level data to characterize encounters to under- stand the demographic and clinical presentation of patients for whom telepsychiatry consults were requested. Telepsychiatry encounters were documented by psychiatrists and behavioral health nurses at the

telemedicine hub through an electronic call log (ViTel Net, Tysons Corner, VA). Patient-level data elements included age, gender, race, eth- nicity, and psychiatric clinical diagnosis determined by the hub’s clinical team (using International Classification of Diseases, 10th Revision, Clinical Modification [ICD-10-CM]). Diagnosis codes were categorized according to the Clinical Classifications Software Level-2 codes [20]. En- counter data elements included site identification, reason for and type of encounters, and temporal measures (i.e., day of the week, time of day). We assessed the reason for engagement including clinician-to- clinician consult (to address medication management questions or clinical questions) or a clinical assessment/evaluation (to evaluate the behavioral health needs of the patient). Facility-level characteristics were identified from the 2020 American Hospital Association Annual Survey [21]. Exact values for these characteristics were categorized to help maintain anonymity for sites. The earliest date included was the implementation date at each ED, and the end of the evaluation period was May 31, 2021 for all sites (study periods ranged from 176 to 712 days, by site). Patients were included in the data more than once if they had multiple encounters during the study period.

• 1. Qualitative methods

We conducted semi-structured interviews with seven key infor- mants (i.e., administrators, physicians, nurses) at participating EDs. We used a structured interview guide that was designed to explore the experience of implementing and using telepsychiatry services in these sites. Respondents were identified through a purposive sampling approach to include a diverse cohort of participants who had used the service so that we could capture insights from key informants most fa- miliar with the telepsychiatry service. We conducted interviews lasting between 45 and 60 min approximately six months after the service im- plementation at sites, and we focused on the service preparation and delivery phases. Interviews were audio-recorded, with participant per- mission, and transcribed.

Two experienced coders used an inductive qualitative analysis ap- proach to independently identify themes related to the ED telepsychiatry service utilization [22]. Following this, they met to dis- cuss their findings, establish a set of agreed-upon themes, refine a code- book, and identify illustrative quotes for each theme. Then the coders grouped themes into three domains: facilitators, barriers, and impacts. Following this phase, the coders met with a third investigator to discuss the emerging findings. An iterative process was used that involved con- tinuous modification and organization of the codes to identify the themes’ relationships. Discrepancies among the three investigators were discussed until consensus was reached.

1. Results
   1. Facility-Level utilization

The implementation dates for each site and general characteristics (including number of licensed beds, critical access hospital status, and annual ED volume) where the ED telepsychiatry services were delivered are identified in Table 1. There were 4130 encounters for 3932 patients from the EDs during the evaluation period. Due to differences in dates of telepsychiatry implementation and varying evaluation periods, there was variability in the volume of encounters by site. Using a 30-day monthly average, there was significant variation in the use of the telepsychiatry services in the ED by site, ranging from 2.6 encounters per month up to 159 encounters per month. Only one facility was a crit- ical access hospital (Site D) and the others were prospective payment system hospitals. Total licensed bed counts ranged from those with 0 to 50 and up to 400 to 450 beds and annual ED volume ranged from ap- proximately <5000 to 50,000 visits.

Patient demographic and clinical characteristics are presented in Table 2. Approximately 54% of encounters involved female patients.

Table 1

Telehealth services implementation and characteristics by site.

Site	Implementation Date	Days Evaluated	Total Encounters	Average Encounters/ Month	Critical Access Hospitala	Total Licensed Bed Countsa	Annual ED Volumea
A	6/20/2019	712	3770	158.8	No	400-450	45,000-50,000
B	3/3/2020	455	63	4.2	No	150-200	5000-10,000
C	9/5/2020	269	23	2.6	No	100-150	5000-10,000
D	11/19/2020	194	77	11.9	Yes	0-50	<5000
E	12/7/2020	176	197	33.6	No	0-50	10,000-15,000

^a Data obtained from the American Hospital Association Annual Survey, 2020. Licensed bed counts and annual ED volume were categorized to maintain anonymity.

The age group with the highest number of encounters was <18 years (36%), followed by those 25-44 years (32%). Adults over 65 years ac- counted for <3% of all encounters. The majority of patients seen were white (51%) or Native American (44%) reflecting the population of the communities where the EDs were located. Among the diagnoses in- volved in the telepsychiatry consult, the most frequently identified in- cluded depression (28%), suicide/self-harm (17%), and schizophrenia (12%).

Encounters occurred both by video (36%) and phone (64%). For Site A, the most frequent reason for telepsychiatry consultation was clinician-to-clinician consultation (68%) (Table 3). However, a clinical assessment/evaluation was the most frequent reason for consultation at all other sites. Across sites, 99% of clinician-to-clinician consults were by phone, and 99% of clinical assessments/evaluations were by video. The distribution of encounters varied by the day of the week and the time of day. temporal differences by day and time, which were primarily driven by local staffing needs, are further described below in the qualitative themes.

• 1. Perceptions of telepsychiatry implementation in EDs

Themes from the in-depth interviews are grouped into three do- mains: facilitators, barriers, and impact. Selected quotes representing the themes within each domain are presented in Table 4.

Table 2

Characteristics of patients with telehealth encounters across all sites.

• • 1. Facilitators

This domain represents factors that drive the telepsychiatry imple- mentation in participating ED sites and includes five motivating themes: increasing need, supportive infrastructure, straightforward process, familiarity with telemedicine, and collaborative relationship. ED physicians, nurses, and administrators in our study observed that ac- cess to behavioral health experts is becoming more critical in rural areas with an upward trend in number of patients with serious mental health issues. Furthermore, interviewees mentioned that having the needed tools and a compatible system with the telepsychiatry hub increased the feasibility of using this service in underserved settings. This facilita- tor is particularly helpful during the initiation period, when EDs start utilizing the telepsychiatry service. Others also shared that the straight- forward process and familiarity with similar telemedicine services con- tributed to facilitating the implementation. To encourage utilization of this telepsychiatry service, the hub needed to ensure that the service was not a burden to use and that the users understood the process. Overall, the EDs appreciated the collaborative relationship with the hub and the open communication process that enabled them to work together to address barriers. The opportunity for having interactive communication was crucial, considering each ED may have different ca- pabilities and experiences in implementing this telepsychiatry service.

• • 1. Barriers

This category contains themes that illustrate hurdles in imple- menting the service, such as the clarity of process and technical limita- tions. Some interviewees pointed to the need to work through barriers

that arose early in the implementation process. Moreover, in the begin-

Patient Characteristics Total (%)

Gender

Male 1814 (46.1)

Female 2118 (53.9)

Age (Years)

<18 1399 (35.6)

18-24 680 (17.3)

ning, overcoming issues can be a challenge due to miscommunication or misunderstanding. Interviewees mentioned there was a need for in- creased clarity on roles between the hub and spoke staff (e.g. sharing/

Table 3

Characteristics of telehealth encounters across all sites.

25-44 1275 (32.4)

45-64	474	(12.1)	Encounter characteristics	Total (n = 4130)
>=65	103	(2.6)
Unknown	1	(<0.1%)
Race			Encounter Type
White	1994	(50.7)	Phone	2635	63.8
Native American	1727	(43.9)	Video	1492	36.1
African American	105	(2.7)	Unknown	3	0.1
Asian	19	(0.5)	Reason for Consultation
Other/Unknown	87	(2.2)	Clinician-to-Clinician Consultation	2631	63.7
Ethnicity			Clinical Assessment/Evaluation	1495	36.2
Hispanic	233	(5.9)	Unknown	4	0.1
Not Hispanic	3527	(89.7)	Day of Week
Unknown	172	(4.4)	Sunday	635	15.4
Telepsychiatry Hub Behavioral Health Diagnosis			Monday	542	13.1
Depression	1093	(27.8)	Tuesday	588	14.2
Suicide/Self-Harm	655	(16.7)	Wednesday	605	14.6
Schizophrenia	484	(12.3)	Thursday	535	13.0
Substance Use/Overdose	449	(11.4)	Friday	557	13.5
Personality Disorder	421	(10.7)	Saturday	668	16.2
Other/Unknown	369	(9.4)	Time of Day
Trauma/Stress	221	(5.6)	12 AM-6 AM	726	17.6
Other Psychiatric	178	(4.5)	6 AM-12 PM	375	9.1
Medical	46	(1.2)	12 PM-6 PM	1264	30.6
Neurological/Neurocognitive	16	(0.4)	6 PM-12 AM	1765	42.7

n %

Table 4

Exemplary quotes on telehealth implementation in rural EDs.

Domains Themes Quotes

patient might want to be left alone with the video equipment for a pri- vate conversation with the mental health professional. To address these concerns, sites worked through processes to permit the patient to have

Facilitators 1. Increasing

need

2. Supportive infrastructure

3.

Straightforward process

1. Familiarity with telemedicine
2. Collaborative relationship

Barriers 1. Clarity of process

2. Technical limitations

Impacts 1. Workforce improvement

1. Care improvement
2. Patient satisfaction
3. Cost-benefit
4. Facilitating COVID care
5. Access improvement

We’ve had a lot more people that have had substance abuse issues, behavioral health issues that are presented to our ED. I think that’s just going to keep going up with all the stresses in society and stuff. (Physician - Site D)

We’re using an EMR system and they have access to our patient record from the information we put in locally. And then we can see if they put in a note on the patient from their end. (Physician - Site D)

The staff definitely prefer the telepsychiatry service, with the ease of use. They think the process goes a lot quicker if we use it. (Admin - Site C)

The staff need minimal training because we have added other telemedicine services already so they’re familiar with this type of service, so it’s very minimal. (Admin - Site D)

We have worked with the hub when we hear about issues and they are more than willing to correct and try to help us work through some of those issues. (Admin - Site A)

I think it’s confusing for a lot of people, nurses and doctors, on exactly how the “hold and release” process works for our behavioral health patients. And all that affects the legal side as well, including paperwork issues. (Physician - Site C)

We had some connectivity issues with our wireless, so the iPad would shut off and so it was a challenging issue that we had. (Nurse - Site C)

The service just frees up one of our staff that are on to be able to assist the other patients. We’re also helping these patients get where they need to be a little faster. (Admin - Site C)

Before the telemedicine service, sometimes we’d have to keep these patients for a couple of days to find placement stuff. Since we’ve had the telemedicine, we actually get a psychiatrist that gives us a good assessment and we can present that to some of the referral facilities and they are more comfortable with taking the patient. (Physician - Site D)

While we may be seeing more behavioral health, we’re actually able to send some of these people home versus sending them to a facility, which is helpful. So while the numbers have increased, we’ve actually been able to send a lot of them home or send them with maybe a med change or follow-up as an outpatient or that kind of thing. (Nurse - Site D)

It’s a cost savings for the community, having to have our police officer out of our community and taking them somewhere. And then a cost savings for our organization, just in decreasing the amount of time they’re on a one-on-one hold with us. (Admin - Site C)

The service helps guide us in those situations where COVID might interrupt what our normal process is for patients who might need an inpatient stay, how we can manage through that. (Admin - Site C)

It’s providing that service of getting our patients admitted after hours without needing to disrupt local psychiatrists when they’re not really available. So it’s taken that gap and bridged it for us, basically. (Admin - Site A)

a private interaction with the hub personnel using the video equipment while the local staff maintained supervision of the patient at a distance to assure their safety.

Technical limitations were also identified as one of the barriers dur- ing the implementation process. These included both connectivity is- sues and problems with internet network availability.

3.2.3. Impacts

Interviewees shared their thoughts on the influence of the telepsychiatry service on ED care delivery in underserved settings. We defined six themes identified in this domain: workforce improvement, care improvement, patient satisfaction, cost-benefit, facilitating COVID care, and access improvement. Interviewees found that having the telepsychiatry service available any time of the day positively impacted ED staff by allowing existing clinicians to care for more patients and pro- vide care in a timely manner. One interviewee highlighted, “This telepsychiatry service frees up a nurse a little quicker, so we’re not spending so many hours on a one-on-one visualization. We come to a [clinical] decision quicker and dispositions quickly, so that helps.” Additionally, the staffing and availability of local psychiatrists at each site contributed to the variability observed in temporal measures (e.g. day of the week, time of the day) such that these services were instrumental in overcoming limitations in local staffing availability.

Moreover, access to the hub’s expertise also built local clinicians’ confidence in caring for behavioral health patients. One interviewee shared a comment, “ED staff are much more comfortable dealing with patients with behavioral health complaints after we’ve been able to get the telepsychiatry service.” The service benefited not only internal resources but also external stakeholders, as shared by one of the admin- istrators, “This hospital is part of the mental health task force for our communities. If we’re using this telepsychiatry service, we’re definitely not contacting the local area counseling center to do as many evalua- tions for us, which then frees them up to see more patients in the community.”

In addition to workforce improvement, this telepsychiatry service benefited the care process in EDs serving predominantly rural popula- tions. It enabled better workflow, as one of the interviewees shared, “The care process is more automatic now. We know that if we get be- havioral health patients, we go with the telepsychiatry process versus waiting for a social worker or a local expert, and all the challenges with contacting people and waiting and all that.” The service helped di- rect patients to the appropriate care facility and minimize unnecessary care. One administrator commented, “It’s met the goal to hook patients up with other services in our community and decrease the number of unnecessary involuntary holds placed on a patient. And when a referral is needed, the hub will give recommendations on what medications we should get started or what we can assist with here at the facility while we’re waiting on transport.”

Consequently, improvements in the care process also impacted pa- tients, as one interviewee shared, “You have much-improved satisfac- tion on the patient part. Many of these are parents of behavioral health patients who are their kids, and they’re not sure what’s going on. They feel a lot better when they know that a hub psychiatrist actu- ally talked one-on-one with their child.” In regard to the cost-benefit of this telepsychiatry service, interviewees had mixed opinions. One site perceived the benefits from organization and community perspec- tives, while others determined that the relationship between cost and benefit was still unclear. An administrator commented, “I don’t believe

completing paperwork) and communication in some situations. This was particularly the case for “hold and release” processes, referring to temporary Involuntary holds that involve specific legal considerations, which differ between locations. There were also concerns regarding pa- tient safety and privacy during the consultation process when the

that we could say there’s been a cost-benefit on our side. Right now, it’s an additional expense that’s not reimbursed, but it’s a necessary one.”

The necessity of having a telepsychiatry service such as this one has become more prominent with the COVID-19 pandemic. One

interviewee observed, “We see a lot of behavioral health people since COVID hit. We’re almost to the point of doubling the number of people that are coming in for behavioral health reasons, whether it’s a suicide attempt or suicidal ideation or others.” The availability of this telepsychiatry service has helped facilitate the virtual consultation, as shared by one interviewee, “We have had behavioral health patients that have had COVID. So, we’ve been able to consult them using tele- medicine capabilities. Otherwise, they would not have been able to see a behavioral health expert at all.”

Overall, the interviewees agreed that having a telepsychiatry service that is always available has improved access to behavioral health ex- perts. This access has enabled them to seek appropriate consultations for caring for their patients. For instance, one interviewee commented on the medication management assistance, “Some of these [are] pa- tients who are depressed, and we can’t find beds for them. We can get a hub psychiatrist to see them and start them on medication programs. It’s definitely beneficial to try and get them on those meds as soon as possible.” The opportunity to access appropriate experts has added value to ED services and the population served as one interviewee reflected on the experience, “The hub has definitely allowed us to pro- vide better care for our patients because of the added benefit of having a psychiatrist. It’s provided something better for our physicians and get- ting the actual assessment done quicker and as the patient needs it. It’s provided that for us, which is something that was definitely needed.”

4. Discussion

Telepsychiatry services serve as one strategy to address the chal- lenges in balancing staffing needs, financial costs, and the varied patient needs in EDs serving rural and underserved settings [23-25]. In this study, we assessed the process and user perceptions after telepsychiatry for behavioral health emergencies was implemented in EDs. The key strengths of this study include incorporating timely qualitative inter- views to better understand overall themes of telepsychiatry implemen- tation, prospectively collecting quantitative data to assess the extent and type of utilization, and having the ability to generalize these find- ings from multiple hospitals, personnel, and states to other ED settings. There were three key findings in this study about the process and im- pact of telepsychiatry implementation in EDs that warrant further ex- planation, along with the similarities and distinctions between our work and previous efforts.

One of the primary findings in this study is the positive impact that telepsychiatry had on improving access to behavioral health expertise and addressing patient needs in EDs in underserved areas. For example, the admission and transfer processes were viewed by interviewees as improved post-implementation. Some previous studies have reported lower rates of admission among telemedicine users [26-28], while others had higher rates of admission among patients when telemedi- cine was incorporated into care management [12,29,30]. As described in a review paper, the differences among these earlier study findings may be a result of over-triaging non-critical patients, averting unneces- sary triage, or increasing transfers and local admissions for patients pre- senting with more emergent psychiatric complaints [31]. Despite the variability in patient outcomes with telepsychiatry implementation, the service, as identified in this study, was perceived as beneficial for improving the care process and achieving its intended goals of enhanc- ing local support for management and care.

Another key finding in this study was how telepsychiatry addressed staffing concerns, local workforce shortages, and barriers with general infrastructure at local sites, which is often a primary challenge in EDs providing care in underserved areas. Interviewees recognized how the implementation of telepsychiatry in these sites eased the burden of staffing generally and enhanced the local capacity for behavioral health emergency care. Examples that emerged from our study included timely access to a hub psychiatrist and behavioral health nurse, the re- ferral or recommendations by these experts, and improvement in

processes so local ED staff may better assist other patients. These find- ings are consistent with a telemedicine implementation study within the Veterans Health Administration, in that the service addressed re- source limitations, burdensome logistical processes involved in patient care and management, and diversion of resources for clinical care [5]. Although research has demonstrated telepsychiatry to be cost- effective [27,28,32], perceptions by interviewees in the current study were mixed and inconclusive due to timing of interviews early in the implementation process. This is clearly an important area for future research.

A third finding in this study pertains to the timing of the implemen- tation, which occurred across sites prior to or during the COVID-19 pan- demic. Interviewees at several sites reported how telepsychiatry implementation facilitated the patient management process during the COVID-19 pandemic, particularly by safely managing patients dur- ing the pandemic for behavioral health care in the ED. This theme is con- sistent with previous work that demonstrated how a virtual Delivery of care could assist with public health disasters or crises, by alleviating staffing burdens, limiting Infection risk, and improving safety [33,34]. Though telepsychiatry service implementation was not due to the onset of the COVID-19 pandemic within this network, it may have been utilized more when local sites benefited from the additional sup- port and expertise offered due to a public health crisis.

One consideration with the telepsychiatry model in this setting is the ability to address privacy concerns, which is often purported to be a strength of telepsychiatry care when compared to traditional in- person visits [35-38]. Perhaps the primary distinction between this study and others stems from the ED setting as much previous research that demonstrated increased convenience and privacy involved direct consultation occurring in the patient’s home or other private settings. Interviewees noted that there were limitations to privacy during a telepsychiatry evaluation in the ED to ensure the patient was continu- ously monitored by local staff to assure their safety in the ED. This may continue to be a trade-off when balancing the safety of patients presenting with psychiatric emergencies and providing adequate pri- vacy in the ED setting.

There are some limitations to this study. First, the analysis only in- cludes utilization data after telepsychiatry implementation. The lack of pre-implementation data has deterred the ability to conduct compara- tive analysis. However, this study has gathered perceptions regarding the use process that provide an insight into the value of this service for rural and underserved EDs. Future research can include prior care utilization data to complement the findings from this study. This study is expected to provide valuable guidelines for future research that may engage EDs across multiple regions in the country.

In conclusion, implementation of a telepsychiatry service in rural and underserved ED settings holds the promise of being beneficial to the patient, local ED, and the community. In this study, we found that implementing this service alleviated the burden of care during the COVID-19 pandemic, enhanced local site capability, and improved local ability to provide quality and effective care.

Funding

This work was supported by the Leona M. and Harry B. Helmsley Charitable Trust.

CRediT authorship contribution statement

J. Priyanka Vakkalanka: Conceptualization, Formal analysis, Inves- tigation, Methodology, Software, Writing - original draft, Writing - re- view & editing. M. Muska Nataliansyah: Conceptualization, Formal analysis, Investigation, Software, Visualization, Writing - original draft, Writing - review & editing. Kimberly A.S. Merchant: Conceptualiza- tion, Data curation, Formal analysis, Investigation, Methodology, Project administration, Software, Visualization, Writing - original draft, Writing

- review & editing. Luke J. Mack: Conceptualization, Data curation, Funding acquisition, Project administration, Resources, Software, Writ- ing - review & editing. Seth Parsons: Investigation, Writing - review & editing. Nicholas M. Mohr: Conceptualization, Funding acquisition, Supervision, Writing - review & editing. Marcia M. Ward: Conceptual- ization, Funding acquisition, Investigation, Methodology, Project ad- ministration, Supervision, Visualization, Writing - original draft, Writing - review & editing.

Declaration of Competing Interest

None.

References

1. Rural Health Information Hub. Rural mental health. https://www.ruralhealthinfo. org/topics/mental-health; 2021. [accessed March 18, 2022].
2. Larson EH, Patterson DG, Garberson LA, CHA Andrilla. Supply and distribution of the behavioral health workforce in Rural America. Data brief. Rural Health Research & Policy Centers; 2016.
3. Hartley DGJ, Ziller EC, Loux SL, Hansen AY. Mental health encounters in critical ac- cess hospital emergency rooms: A national survey. http://muskie.usm.maine.edu/ Publications/rural/wp32.pdf; 2005.
4. Mohr NM, Wu C, Ward MJ, McNaughton CD, Richardson K, Kaboli PJ. Potentially avoidable inter-facility transfer from veterans health administration emergency de- partments: a cohort study. BMC Health Serv Res. 2020;20(1):110. https://doi.org/10. 1186/s12913-020-4956-6.
5. McNaughton CD, Bonnet K, Schlundt D, Mohr NM, Chung S, Kaboli PJ, et al. Rural Interfacility emergency department transfers: framework and qualitative analysis. West J Emerg Med. 2020;21(4):858-65. https://doi.org/10.5811/westjem.2020.3. 46059.
6. NEJM catalyst: What is telehealth? https://catalyst.nejm.org/doi/full/10.1056/CAT.1 8.0268;; 2022. [accessed March 2, 2022].
7. Banbury A, Roots A, Nancarrow S. Rapid review of applications of e-health and re- mote monitoring for rural residents. Aust J Rural Health. 2014;22(5):211-22. https://doi.org/10.1111/ajr.12127.
8. Ward MJ, Shuster JL, Mohr NM, Kaboli PJ, Mixon AS, Kemmer J, et al. Implementation of telehealth for psychiatric care in VA emergency departments and urgent care clinics. Telemed J E Health. 2021. https://doi.org/10.1089/tmj.2021.0263.
9. Mohr NM, Campbell KD, Swanson MB, Ullrich F, Merchant KA, Ward MM. Provider- to-provider telemedicine improves adherence to Sepsis Bundle care in community emergency departments. J Telemed Telecare. 2021;27(8):518-26. https://doi.org/ 10.1177/1357633×19896667.
10. Mohr NM, Vakkalanka JP, Harland KK, Bell A, Skow B, Shane DM, et al. Telemedicine use decreases rural emergency department length of stay for transferred North Da- kota trauma patients. Telemed J E Health. 2018;24(3):194-202. https://doi.org/10. 1089/tmj.2017.0083.
11. Mohr NM, Young T, Harland KK, Skow B, Wittrock A, Bell A, et al. Telemedicine is as- sociated with faster diagnostic imaging in stroke patients: a cohort study. Telemed J E Health. 2019;25(2):93-100. https://doi.org/10.1089/tmj.2018.0013.
12. Vakkalanka JP, Harland KK, Wittrock A, Schmidt M, Mack L, Nipe M, et al. Telemed- icine is associated with rapid transfer and fewer involuntary holds among patients presenting with suicidal ideation in Rural hospitals: a propensity matched cohort study. J Epidemiol Community Health. 2019;73(11):1033-9. https://doi.org/10. 1136/jech-2019-212623.
13. Ward MM, Carter KD, Ullrich F, Merchant KAS, Natafgi N, Zhu X, et al. Averted trans- fers in rural emergency departments using telemedicine: rates and costs across six networks. Telemed J E Health. 2021;27(5):481-7. https://doi.org/10.1089/tmj. 2020.0080.
14. Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs- principles and practices. Health Serv Res. 2013;48(6 Pt 2):2134-56. https://doi.org/ 10.1111/1475-6773.12117.
15. Crabtree BF, WL. M. Doing qualitative research. 2nd ed. SAGE Publications, Inc; 1999.
16. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and FoCUS groups. International J Qual Health Care. 2007;19(6):349-57. https://doi.org/10.1093/intqhc/mzm042.
17. Ward MM, Ullrich F, MacKinney AC, Bell AL, Shipp S, Mueller KJ. Tele-emergency uti- lization: in what clinical situations is tele-emergency activated? J Telemed Telecare. 2016;22(1):25-31. https://doi.org/10.1177/1357633×15586319.
18. Mueller KJ, Potter AJ, MacKinney AC, Ward MM. Lessons from tele-emergency: im- proving care quality and health outcomes by expanding support for rural care sys- tems. Health Aff (Millwood). 2014;33(2):228-34. https://doi.org/10.1377/hlthaff. 2013.1016.
19. Rural Health Information Hub. Am I rural? - tool. https://www.ruralhealthinfo.org/ am-i-rural; 2022. [accessed April 5, 2022].
20. Agency for Healthcare Research and Quality. Clinical Classifications Software (CCS) for ICD-10-PCS (beta version). https://www.hcup-us.ahrq.gov/toolssoftware/ccs10/ ccs10.jsp; 2022. [accessed March 2, 2022].
21. American Hospital Association. AHA annual survey database. https://www.ahadata. com/; 2020. [accessed March 2, 2022].
22. Bradley EH, Curry LA, Devers KJ. Qualitative data analysis for health services re- search: developing taxonomy, themes, and theory. Health Serv Res. 2007;42(4): 1758-72. https://doi.org/10.1111/j.1475-6773.2006.00684.x.
23. Butterfield A. Telepsychiatric evaluation and consultation in emergency care set- tings. Child Adolesc Psychiatric Clin. 2018;27(3):467-78.
24. Graziane JA, Gopalan P, Cahalane J. Telepsychiatry consultation for medical and sur- gical Inpatient units. Psychosomatics. 2018;59(1):62-6. https://doi.org/10.1016/j. psym.2017.08.008.
25. Morris NP, Hirschtritt ME. Telepsychiatry, hospitals, and the COVID-19 pandemic. Psychiatr Serv. 2020;71(12):1309-12. https://doi.org/10.1176/appi.ps.202000216.
26. Saurman E, Lyle D, Perkins D, Roberts R. Successful provision of emergency mental health care to rural and remote New South Wales: an evaluation of the mental health emergency care-rural access program. Aust Health Rev. 2014;38(1):58-64. https://doi.org/10.1071/AH13050.
27. Reinhardt I, Gouzoulis-Mayfrank E, Zielasek J. Use of telepsychiatry in emergency and crisis intervention: current evidence. Curr Psychiatry Rep. 2019;21(8):63. https://doi.org/10.1007/s11920-019-1054-8.
28. Narasimhan M, Druss BG, Hockenberry JM, Royer J. P W, Glick G, et al. impact of a telepsychiatry program at emergency departments statewide on the quality, utiliza- tion, and costs of mental health services. Psychiatr Serv. 2015;66(11):1167-72. https://doi.org/10.1176/appi.ps.201400122.
29. du Toit M, Malau-Aduli B, Vangaveti V, Sabesan S, Ray RA. Use of telehealth in the management of non-critical emergencies in rural or remote emergency depart- ments: a systematic review. J Telemed Telecare. 2019;25(1):3-16. https://doi.org/ 10.1177/1357633×17734239.
30. Southard EP, Neufeld JD, Laws S. Telemental health evaluations enhance access and efficiency in a critical access hospital emergency department. Telemed e-Health. 2014;20(7):664-8. https://doi.org/10.1089/tmj.2013.0257.
31. Tsou C, Robinson S, Boyd J, Jamieson A, Blakeman R, Yeung J, et al. Effectiveness of telehealth in rural and remote emergency departments: systematic review. J Med Internet Res. 2021;23(11):e30632. https://doi.org/10.2196/30632.
32. Thomas JF, Novins DK, Hosokawa PW, Olson CA, Hunter D, Brent AS, et al. The use of telepsychiatry to provide cost-efficient care during pediatric mental health emer- gencies. Psychiatr Serv. 2018;69(2):161-8. https://doi.org/10.1176/appi.ps. 201700140.
33. Natafgi N, Childers C, Pollak A, Blackwell S, Hardeman S, Cooner S, et al. Beam Me Out: review of emergency department telepsychiatry and lessons learned during COVID-19. Curr Psychiatry Rep. 2021;23(11):72. https://doi.org/10.1007/s11920- 021-01282-4.
34. Heslin SM, Nappi M, Kelly G, Crawford J, Morley EJ, Lingam V, et al. Rapid creation of an emergency department telehealth program during the COVID-19 pandemic. J Telemed Telecare. 2020. https://doi.org/10.1177/1357633X20952632. 1357633X20952632.
35. Vanderpool D. An overview of practicing high quality telepsychiatry. In: Dewan NA, Luo JS, Lorenzi NM, editors. Mental health practice in a digital world: a clinicians guide. Springer International Publishing; 2015. p. 159-81.
36. Chan S, Parish M, Yellowlees P. Telepsychiatry today. Curr Psychiatry Rep. 2015;17

(11):89. https://doi.org/10.1007/s11920-015-0630-9.

1. Daschle T, Dorsey ER. The return of the house call. Ann Intern Med. 2015;162(8): 587-8. https://doi.org/10.7326/m14-2769.
2. American Psychiatric Association. What is telepsychiatry? https://www.psychiatry. org/patients-families/what-is-telepsychiatry; 2020. [accessed March 2, 2022].