a b s t r a c t

Study objective: Primary care (PC) follow-up for discharged emergency department (ED) patients provides pa- tients with further medical attention. We conducted a pilot randomized controlled trial to determine whether using a freely-available physician appointment-booking website results in higher self-reported PC follow-up. Methods: We randomized discharged patients whom treating physicians determined PC follow-up was impor- tant and who possessed health insurance but had no PC provider to one of three groups: (1) a PC appointment booked through the booking website prior to ED discharge; (2) written information on how to use the booking website; or (3) usual care (i.e. standard follow-up instructions). We phoned subjects two weeks after the ED visit to determine whether they had completed a PC follow-up visit. We also asked subjects about their satisfaction with obtaining a PC appointment, satisfaction with the ED visit, symptom resolution and Subsequent ED visits. The self-reported PCP follow-up rate was compared among the study groups by estimating the risk difference (RD) and 95% CI between usual care and each intervention group.

Results: 272 subjects were enrolled and randomized and 68% completed the two-week telephone follow-up in- terview. The self-reported PCP follow-up rate was higher (52%) among subjects whose appointment was booked on the website before ED discharge (RD = 16%; 95% CI -1%, 34%) and lower (25%) for subjects who received book- ing website information (RD = 13%; 95% CI -32%, 7%) compared to subjects (36%) in the usual care group. A higher percentage of subjects in the booking group were more likely to report being extremely or very satisfied with obtaining a PC appointment (78%) compared to those who received booking website information (54%) or usual care (40%).

Conclusion: Among ED patients that providers judged PC follow-up is important, using a booking website to schedule an appointment before ED discharge resulted in a higher but not statistically significant self-reported PC follow-up rate. This intervention warrants further investigation in a study with a larger sample size and objec- tive follow-up visit data.

(C) 2019

Introduction

Background

Approximately 110 million patients are discharged from the emer- gency department (ED) annually [1] with over half recommended to follow-up with a primary care (PC) provider [2]. Follow-up with a PC provider is particularly important when the health problem addressed

? Meetings: “A Randomized Controlled Trial Evaluating a Novel Approach to Post-ED Follow-up Appointment Planning,” National SAEM Conference, May 18, 2017.

* Corresponding author.

E-mail address: [email protected] (R.J. Merritt).

in the ED warrants further medical attention and guidance. However, patients can experience numerous challenges obtaining timely follow up. Barriers to PC follow-up include the lack of a PC provider or PC ap- pointment availability, insurance status, unclear discharge instructions and socioeconomic factors [3]. Removing barriers to PC follow-up, such as booking an appointment before ED discharge, increases follow-up compliance [4,5].

Because patients without insurance are particularly vulnerable to poor PC follow-up, much previous ED to PC linkage research focused on this population [4,6]. Since the 2010 Affordable Care Act, 20 million individuals have obtained health insurance [7]. The insured now repre- sents 90% percent of the population (from 84% in 2010) [8]. However it is unclear how many of these insured patients now have PC providers.

https://doi.org/10.1016/j.ajem.2019.158384

0735-6757/(C) 2019

Importance

A recent analysis of care transitions between emergency providers and PC teams identified care linkage as an important area for further re- search including the development of a technological solution that would allow ED staff to easily book appointments for their patients with PC providers prior to ED discharge [5]. Previous research in ED- based PC linkage has focused on diversion to PC clinics, using custom- ized software to link low-income patients to local community health centers, text message-based reminders and appointment-booking using a research assistant to call PC providers and schedule appoint- ments [8]. Identifying a scalable, rapid and low-resource bedside inter- vention that schedules follow-up appointments may not only make it feasible for ED staff to more effectively link ED patients to follow-up care but it may also increase patient satisfaction, improve health out- comes and decrease ED return visits.

This is the first pilot randomized controlled trial (RCT) to evaluate whether a free physician appointment-booking website, Zocdoc, used at time of ED discharge with insured patients, increases PC follow-up compliance. Zocdoc allows users to book an appointment at a desired date and time with a PC provider who accepts the user’s insurance within a radius of the patient’s zip code. Thus, barriers of time, proximity and cost are minimized. To note, none of the authors have any financial relationships or competing interests to disclose with Zocdoc.

Goals of this investigation

The primary objective of this pilot RCT was to determine the effect that PC follow-up appointment booking at ED discharge has on follow-up compliance compared to providing patients with information on how to use the booking website versus standard discharge instruc- tions. We hypothesized that patients who received a follow-up appoint- ment booking before ED discharge would report more frequently completing a PC visit compared to the other two study groups.

Materials and methods

Study design

We conducted a parallel RCT at an academic ED in Washington, DC after approval by the Institutional Review Board and registration with ClinicalTrials.gov (NCT02740348). Patients were eligible for the study if they were insured, did not have a PC provider (or wanted a new PC provider) and the ED provider considered PC follow-up within 14 days as important. Patients who consented were randomized to one of three treatment groups with equal probability: (1) standard dis- charge instructions plus a PC follow-up appointment scheduled prior to ED discharge using an online physician Appointment scheduling sys- tem; (2) standard discharge instructions plus written booking website information provided to the patient to self-schedule a PCP appointment at his/her convenience; or (3) standard discharge instructions. Approx- imately two weeks after ED discharge, a research assistant (RA) attempted to call each subject to determine whether the patient com- pleted a follow-up PC visit and other outcomes.

Study setting and selection of participants

The study was conducted at an urban, tertiary hospital with an an- nual ED census of approximately 71,000 visits. The vast majority of pa- tients who present to the ED are treated and released (82%). The overall health insurance coverage at our ED is 31% commercial, 35% Medicaid, 22% Medicare and 12% self-pay.

Patients were eligible for the study if they were at least 18 years old, were being discharged from the ED, possessed private or public health insurance, spoke English, had an email address, reported that they did not have a PC provider or reported that they had one but wanted new

one, and the ED provider considered a PC follow-up within 14 days as important.

Because not all patients need a PC care follow-up visit within 14 days

[5] and we didn’t want to focus on a specific disease condition, we asked the treating provider to rate the importance of a primary care follow-up visit within 14 days using a Visual analogue scale from 0 (not important) to 10 (very important). Ratings of five (moderately important) or greater were defined as important and meeting the study inclusion criteria. If the treating provider rated a PC visit as a five or greater, we also asked why the ED provider thought the patient should see a PC pro- vider within 14 days and recorded the provider’s verbatim response in an open text field. We did not tell the ED providers about the purpose of the rating.

We excluded patients for the following reasons: (1) they did not want a PC provider; (2) they did not want to schedule a PC follow-up visit; (3) they did not have email access; (4) were a prisoner; (5) had a psychiatric chief complaint; or (6) had Kaiser health insurance since Kaiser was not listed on the booking website used in this study.

Enrollment occurred between November 5, 2015 and June 26, 2017 during the hours of 9 am to 10 pm on weekdays and 9 am to 6 pm on weekends. Trained RAs screened patients for study eligibil- ity by reviewing patient information available in the electronic ED tracking board (i.e. age, insurance status, Disposition decision). RAs monitored the tracking board and after the discharge disposition was documented, the RA asked the ED attending, resident or physi- cian assistant who was treating the patient to rate the importance of a PC follow-up visit within 14 days. The RA then approached pa- tients with ratings of five or above, described the study and reviewed the eligibility criteria. All patients willing to participate signed a written consent form.

We used SAS software (version 9.4, SAS Institute, Inc., Cary, North Carolina) to generate a randomization strategy that randomly allocated patients with equal probability to one of three treatment groups prior to onset of enrollment. We uploaded and concealed the randomization strategy in REDCap (version 6.5.12; Nashville, TN). REDCap is a secure, web-based application designed to support data capture for research studies. The RAs used REDCap for randomization and real-time data capture [9]. After a subject agreed to participate and signed the written consent form, the RA selected the randomization button in REDCap which revealed the treatment assignment for that subject.

Interventions

The first group, the booking group, had a PC follow-up appointment booked at the bedside by the RA using an online physician appointment scheduling website known as Zocdoc, see www.Zocdoc.com. Zocdoc is free to use for patients, but doctors/practices must pay to be listed on the site. To book an appointment using Zocdoc, the user enters online the condition, procedure or type of doctor desired, their zipcode and the name of their health insurance plan. Based on this information, Zocdoc lists all physicians who accept the type of health insurance a per- son has, their available appointment dates and times, their location, practice affiliation, main specialty, medical degree and user ratings and reviews. Zocdoc prioritizes the physician list by whether the physi- cian is within or outside of the user’s health plan network, the next available appointment time and distance.

For this study, the RA used a computer to enter the above informa- tion into the online appointment system for each subject and generate a list of PC providers and their available appointments. The RA used the patient’s insurance card to select the correct Insurance type, and attempted to schedule the patient for the soonest available appoint- ment - most patients were able to obtain an appointment within a week to two weeks. After the subject selected a physician and appoint- ment date and time, he/she created a Zocdoc account using his/her email address so that the provider could reach the subject prior to the appointment and Zocdoc could send automatic appointment reminders.

The user also has the option to provide a cell phone number and also re- ceive appointment reminders via text messages. The RA documented the name of the physician the subject scheduled with and the date of the appointment. All subjects who had an appointment scheduled through the online booking system were also discharged by the nurse and given ED written discharge instructions according to usual care.

The second intervention group, the booking website information group, received a two-page step-by-step guide that detailed how to schedule a PC appointment through the same online physician appoint- ment scheduling system at the subject’s convenience. All subjects ran- domized to the booking website information group were also discharged by an ED nurse according to usual care.

Usual care

The third group was randomized to be discharged by an ED nurse ac- cording to usual care. Before a patient is discharged, the ED attending, resident or physician assistant uses Cerner, the ED’s electronic medical record (EMR) system, to write the discharge summary which provides information on the discharge diagnosis, a summary of the treatments delivered in the ED and instructions on what the patient should do post discharge.

A standard statement in the discharge instructions is for the patient to follow-up with his/her PC provider. If the patient reported a PC pro- vider to the registration staff during the ED visit, then the patient’s PC provider is listed as the recommended PC provider to follow-up with in the discharge instructions. If the patient did not report a PC provider to the registration staff, contact information for the internal medicine department of our university physician practice group is provided as a recommended place for follow-up. At time of discharge, a nurse reviews and gives the patient the written discharge summary and instructions and answers any questions the patient may have. Participants in the usual care group were not provided any information about the online physician booking website.

Methods of measurement

RAs completed a brief interview with all enrolled subjects prior to ED discharge. RAs asked subjects to rate the importance of a PC follow-up visit on the same 0 to 10 scale that we used with the clini- cians. The interview also included questions about highest level of schooling completed, marital status, race and ethnicity and overall health. The RAs documented triage Acuity level, chief complaint, ED dis- charge diagnoses, treatment location (main ED versus urgent care), Type of insurance and whether the ED provider recommended follow-up with a specialist in addition to a PCP by reviewing and abstracting the information from the EMR.

Approximately two weeks after the index ED visit, an RA attempted to contact each subject to complete a telephone follow-up interview. The RA asked each participant whether he/she had completed a follow-up visit with a PC provider and if he/she had, the date of the follow-up PC visit and the last name of the provider. If the subject did not follow up, they were asked if they had an appointment coming up or if they had rescheduled their initial appointment. We did not re- contact subjects who had an upcoming appointment. Subjects were also asked to rate their satisfaction with both their ED visit and the pro- cess of obtaining a PC follow-up appointment using a 5-level Likert scale response that ranged from extremely satisfied to not at all satisfied. Sub- jects were asked if they had made any additional ED visits since the index visit. Finally, subjects were reminded of the reason they had come to the ED and asked to rate how they felt now on a scale of 0 (no better or worse than when subject initially came to ED) to 10 (completely better). Subjects who completed the follow-up interview were mailed a five-dollar gift card.

Outcome measures

The primary outcome measure was whether the subject had com- pleted a PC follow-up visit according to self-report at the time of the telephone follow-up interview. When designing the study, we calcu- lated the sample size assuming a power of 80%, an alpha level of 0.05 and a follow-up visit rate in the control group of 35% based on a previ- ous study [4]. To detect a difference of 20% or greater in our primary out- come between the control group and each intervention group, we estimated a total sample size of 288 (96 subjects per group) would be required. We also measured the following secondary outcomes: satis- faction with the ED visit, satisfaction with the process of obtaining a PC appointment, subsequent ED visits, and recovery from the initial health problem as secondary outcomes during the follow-up interview.

Analysis

First, we examined the frequency distribution of the patient and clinical characteristics by study group. Second, we compared whether the subject reported completing a PC visit by patient and clinical charac- teristics. Third, we compared the primary outcome, self-reported PC follow-up visit rate, by study group. Finally, we also compared the fre- quency distribution of the secondary outcomes by study group. All com- parisons were tested for statistical significance using a chi-square test of homogeneity except for the primary outcome. For the primary outcome, we calculated the risk difference (RD) and 95% confidence interval (CI) in the self-reported follow-up rate between each intervention group and the usual care group. We analyzed all data according to intention to treat. All analyses were completed using SAS software (version 9.4, SAS Institute, Inc., Cary, North Carolina).

Results

During the 20 month enrollment period, the RAs screened 1681 pa- tients for eligibility. The vast majority of those screened did not meet the inclusion criteria, most often because they had a PC provider and did not want a new one (N = 1002) or the PC visit was not rated as im- portant according to the provider (N = 142) (see Fig. 1). Of those that met the inclusion criteria, we excluded an additional 107 patients be- cause they did not want to schedule a PCP follow-up visit (N = 47), they had Kaiser health insurance (N = 21), they were unable to under- stand consent (N = 20), they did not have an email account (N = 14), or they were a prisoner or previously enrolled (N = 5).

Of the 353 who met both inclusion and exclusion criteria, 73 refused, and 8 were discharged before the RA could complete consent. We suc- cessfully enrolled and randomized 272 subjects. However, five subjects randomized to the booking group did not have an appointment sched- uled while in the ED because they did not like the options provided by the online physician appointment scheduling system (i.e. PC provider too far away, not at convenient times, could not find insurance type in the booking website).

The enrolled sample was relatively young (mean age 35), single (70%) and had earned a high school diploma (94%) (see Table 1). Ap- proximately two-thirds of subjects were privately insured and reported their health as good to excellent (79%). ED providers most commonly cited timely re-evaluation as the reason for the importance of the PC follow-up visit. The majority of the subjects rated the PC follow-up visit as important (93%). The study groups did not vary significantly by their patient or clinical characteristics.

Of the 272 enrolled subjects, 185 completed the telephone follow-up interview (68%). There was no significant difference in the follow-up in- terview completion rate by study arm. The follow-up interview was completed, on average, 29 days after the index ED visit. There was also no significant difference in time to completion of follow-up interview by study group (data not shown). Of the 185 subjects with a completed follow-up interview, 69 (37%) reported completing a follow-up visit

o

Enrollment

Assessed For Eligibility

(N=1,681)

Excluded (N=1,328 )

Did not meet inclusion criteria (N=1,221)

• Has PCP/Not want new one (N=1002)
• PC follow-up not important (N=142)
• Non-English speaking (N=45)
• No health insurance (N=30)
• Age < 18 (N=2)
• Met exclusion criteria (N=107)

Met Eligibility But Not Enrolled (N=81)

• Refused (N=73)
• Missed (N=8)

Met Eligibility and Enrolled (N=272)

Allocation

Analyzed (N=64)

Analysis

Lost to follow-up (N=29)

• Unable to reach by phone after 7 attempts

Follow - Up

Booking Website Information Allocated to Intervention (N=93)

• Received allocated intervention (N=93)
• Did not receive allocation intervention (N=0)

Analyzed (N=59)

Analyzed (N=62)

Lost to follow-up (N=33)

• Unable to reach by phone after 7 attempts

Lost to follow-up (N=25)

• Unable to reach by phone after 7 attempts

Control Group (Usual Care) Allocated to Intervention (N=92)

• Received allocated intervention (N=92)
• Did not receive allocation intervention (N=0)

Booking Website in ED Allocated to Intervention (N=87)

• Received allocated intervention (N=82)
• Did not receive allocation intervention (N=5) Did not like booking website options

Consented & Randomized

(N=272)

Fig. 1. Consort flow diagram.

with a PC provider by the time of the interview. There were no signifi- cant differences in self-reported PC follow-up visit status by patient and clinical characteristics (see Table 2).

Slightly more than half of the subjects (52%) who had a PCP follow- up appointment booked by the RA through the online physician ap- pointment software prior to ED discharge reported completing their follow-up visit during the follow-up interview compared to 25% of sub- jects who received the booking site information and 36% of subjects ran- domized to usual care. Thus, the risk difference of reporting a completed PCP follow-up visit was 16% higher (95% CI -1%, 34%) for subjects who had their PCP appointment booked in the ED and 13% lower (95% CI

-32%, 7%) for subjects who received booking website information com- pared to subjects who received usual care. The risk differences between the two intervention groups compared to usual care were not statisti- cally significantly different. Among the 32 subjects who had their ap- pointment booked in the ED and who reported completing a PC follow-up visit, 55% reported the same name of the provider that we booked the appointment with, 6% reported a different provider and 39% could not recall the name of the provider they saw.

Among the secondary outcomes we examined, only satisfaction with obtaining a PC follow-up appointment was significantly different among the study groups (see Table 3). More than three-quarters of sub- jects (78%) who had their PC appointment booked before ED discharge were very or extremely satisfied with the process compared to 54% who received booking website information and 40% who received standard discharge instructions (p = 0.003). No significant differences were ob- served among groups with respect to satisfaction with the ED visit, sub- sequent ED visits or recovery from the health problem of the index ED visit.

Limitations

The results of our study must be considered in the context of the fol- lowing limitations. First and foremost, the main outcome was measured by self-report and may be subject to social desirability and recall biases. To reduce these biases, we included a booking website information arm

and we asked the subjects in the booking arm to recall the name of the PC provider they saw. More than half of subjects in that study group who reported completing a follow-up visit named the same provider as booked through the online physician appointment system website. We were unable to confirm that these appointments were kept with in- dividual providers, as obtaining data sharing agreements and contacting each individual provider was beyond the study team’s resources and ca- pacity. A second and related limitation was that we did not ask partici- pants who were not booked an appointment the extent to which they may or may not have attempted to make a PCP appointment on their own. We did however ask whether they had a future follow up appoint- ment scheduled if they responded “no” to following up with a PCP at the time of the follow up phone call. Third, our telephone follow-up rate was modest and negatively impacted our power to detect significant differences among the study groups. Fourth, this study was conducted at a single ED in an urban environment with a relatively high insured rate, potentially limiting reproducibility in dissimilar settings. Fifth, we had trouble finding acceptable and timely appointments for five sub- jects in the booking arm, the extent to which this may be more problem- atic in a rural setting is unknown. Finally, these findings may not generalize beyond English-speaking patients who have email access.

Discussion

In this study of ED patients who did not have a PC provider and the treating physician considered it important for the patient to have a PC follow-up visit within two weeks, booking a PC follow-up appointment for patients before ED discharge resulted in a meaningful (although not statistically significant) increase in the self-reported PC follow-up visit rate compared to recommending that patients follow-up with a PC pro- vider as part of standard discharge instructions. The patients who had their follow-up appointment booked in the ED were also more satisfied with the process of obtaining a PC appointment compared to receiving written information on how to book a follow-up appointment them- selves or receiving a recommendation to book an appointment themselves.

Table 1

Percent distribution of patient and clinical characteristics by type of primary care follow-up assistance provided.

Characteristic	Overall N (%)		Appointment booked in ED N (%)		Booking website information given N (%)			Usual care N (%)
	N = 272		N = 87		N = 93			N = 92
Age
18-34	164 (60)		55 (63)		48 (52)			61 (66)
35-54	80 (30)		24 (28)		33 (35)			23 (25)
55+	10 (10)		8 (9)		12 (13)			8 (9)
Male gender	126 (46)		38 (44)		43 (46)			45 (49)
Education
Less than high school	16 (6)		8 (9)		2 (2)			6 (7)
High school diploma/GED	58 (21)		16 (19)		20 (22)			22 (24)
Some college	77 (28)		27 (31)		26 (28)			24 (26)
Bachelor’s degree	69 (26)		20 (23)		27 (29)			22 (24)
Graduate degree or higher	50 (19)		15 (18)		18 (19)			17 (19)
Marital status
Single, never married	189 (70)	58 (67)		60 (65)			71 (77)
Married or domestic partnership	56 (21)	18 (21)		22 (24)			16 (17)
Widowed/separated/divorced	26 (9)	10 (12)		11 (11)			5 (6)
Race
Black	155 (57)	48 (56)		53 (57)			54 (59)
White	86 (32)	27 (31)		29 (31)			30 (33)
Asian/Other	29 (11)	11 (13)		11 (12)			7 (8)
Hispanic/Latino ethnicity	28 (10)	8 (9)		11 (12)			9 (10)
Overall health
Excellent	43 (16)	16 (19)		14 (15)			13 (14)
Very good	88 (32)	32 (37)		27 (29)			29 (32)
Good	83 (31)	20 (23)		37 (40)			26 (28)
Fair	39 (14)	12 (14)		10 (11)			17 (18)
Poor	18 (7)	6 (7)		5 (5)			7 (8)
Insurance type
Private	168 (61)	53 (61)		58 (63)			57 (62)
Medicare +- Medicaid	7 (3)	2 (2)		1 (1)			4 (4)
Medicaid only	85 (30)	31 (36)		28 (30)			26 (29)
Champus/TriCare	1 (1)	0 (0)		1 (1)			0 (0)
Worker’s comp	4 (2)	0 (0)		3 (3)			1 (1)
2 or more types of insurance	7 (3)	1 (1)		2 (2)			4 (4)
Triage level
2	33 (12)	12 (14)		7 (7)			14 (15)
3	167 (62)	49 (57)		62 (67)			56 (61)
4-5	71 (26)	25 (29)		24 (27)			22 (24)
Treatment area
Main side	134 (49)	41 (47)		47 (51)			46 (50)
Urgent care	138 (51)	46 (53)		46 (49)			46 (50)
Chief complaint
Musculoskeletal/trauma	71 (26)	25 (29)		20 (22)			26 (29)
Abdominal pain/OBGYN	71 (26)	24 (27)		24 (27)			23 (25)
Headache/neurologic	44 (17)	14 (16)		19 (20)			11 (12)
Chest pain/SOB	25 (9)	4 (5)		8 (9)			13 (15)
Infectious	23 (9)	4 (5)		12 (13)			7 (8)
Other	34 (13)	16 (18)		8 (9)			10 (11)
Discharge diagnosis
Musculoskeletal/trauma	68 (27)	23 (28)		18 (21)			27 (31)
Abdominal pain/OBGYN	58 (23)	20 (25)		17 (20)			21 (24)
Headache/neurologic	45 (18)	12 (15)		19 (21)			14 (16)
Chest pain/SOB	21 (8)	4 (5)		7 (8)			10 (12)
Infectious	27 (10)	6 (7)		11 (13)			10 (11)
Other	37 (14)	17 (20)		15 (17)			5 (6)
Provider’s main reason for follow-up visit
Timely reevaluation needed	138 (53)	47 (55)		45 (52)			46 (52)
PCP needed to facilitate specialty care	9 (3)	4 (5)		3 (3)			2 (2)
Chronic health issues need PCP attention	49 (19)	16 (19)		12 (14)			21 (24)
Does not have a PCP and needs one	42 (16)	14 (16)		17 (20)			11 (12)
Further imaging or testing needed	14 (5)	3 (3)		6 (7)			5 (7)
Medication follow-up	5 (2)	2 (2)		2 (2)			1 (1)
Pregnant	4 (2)	0 (0)		2 (2)			2 (2)
Provider recommended specialty care	86 (34)	33 (42)		27 (30)			26 (30)
Provider’s rating of importance of follow-up
0-4	0 (0)	0 (0)		0 (0)			0 (0)
5-7	114 (58)	39 (45)		39 (42)			36 (39)
8-10	158 (42)	48 (55)		54 (58)			56 (61)
Patient’s rating of importance of follow-up
0-4	20 (7)	7 (8)		7 (8)			6 (6)
5-7	62 (23)	20 (23)		20 (21)			22 (24)
8-10	190 (70)	60 (69)		66 (71)			64 (70)
Provider’s reason for PCP visit
PCP needed for timely reevaluation	138 (53)	47 (55)		45 (52)		46 (52)
						(continued on next page)

Table 1 (continued)

Characteristic Overall N (%)		Appointment booked in ED N (%)		Booking website information given N (%)		Usual care N (%)
	N = 272		N = 87		N = 93		N = 92
PCP to facilitate specialty care	9 (3)		4 (5)		3 (3)		2 (2)
Chronic health issues identified in ED	49 (19)		16 (19)		12 (14)		21 (24)
No PCP or ED feels they need one	42 (16)		14 (16)		17 (20)		11 (12)
Further imaging or testing needed	14 (5)		3 (3)		6 (7)		5 (7)
Medication follow-up	5 (2)		2 (2)		2 (2)		1 (1)
Pregnant	4 (2)		0 (0)		2 (2)		2 (2)

Our findings are consistent with the results of other Randomized controlled trials that have evaluated the effect of booking a PC appointment prior to discharge versus standard discharge instructions or providing patients with written information on how to make a follow-up appointment on their own [5,10-13]. For example, Kyriacou and colleagues randomized 250 ED patients who the treating physician deemed a timely PC follow-up was necessary to receiving a follow-up appointment booked in the ED versus receiving a recommendation that subjects follow-up with their PC provider or with the hospital refer- ral system if they did not have a provider. The investigators determined follow-up status by contacting the outpatient referral clinics and found that subjects in the intervention group were significantly more likely to follow-up (59%) compared to usual care (37%, p b 0.001) [13].

Interestingly, the self-reported compliance with PC follow-up did not differ significantly between subjects who only received booking website information compared to those who received standard dis- charge instructions. This suggests that the extra step taken in the ED to schedule a follow-up appointment for patients matters. It is possible that patients are less likely to follow-up when they must book the ap- pointment themselves because of technical barriers, insurance compat- ibility or other issues become a higher priority with them once they leave the ED. It should also be noted that the group that only received the booking website information reported a meaningfully lower but not statistically significant follow-up rate than the usual care group which could be a chance finding due to the smaller than anticipated sample size with complete follow-up.

The majority of RCTs that have compared booking a follow-up ap- pointment before ED discharge versus recommending that patients do so on their own have focused on ED patients that are likely to need PC follow-up. The study conducted by Kyriacou and colleagues, similar to our study, identified patients by asking the treating physician whether a timely PC follow-up visit was important [13]. Other trials have focused on ED patients who have a condition that requires on-going monitoring and treatment (e.g. asthma, hypertension) [10,11,14]. The only RCT that did not base study inclusion on the clinical importance of a timely PC follow-up visit still found significant differences between the follow- up visit rates of the group who received a follow-up appointment prior to ED discharge (30%) versus the group that did not (14%) [12]. However, the follow-up visit rates were lower than the rates of ours and others [11-13]. Further research is needed to determine which ED patients would benefit from having a timely primary care or specialist follow-up visit after being discharged from the ED. [5]

One major difference between the majority of the previous RCTs and

this one is that they largely evaluated the intervention on subjects who already had PC providers. If randomized to the intervention group, the ED staff would ask the subject for the name of the PC provider and then call and book the appointment on the patient’s behalf. If the subject did not have a PC provider then ED staff relied on other resources such as a list of PC providers willing to take new patients [11], a hospital re- ferral system or hospital clinic willing to accept new patients [10,13]. In this trial, we used a freely available, online physician appointment scheduling program to book PC appointments for patients that were compatible with their health insurance coverage and acceptable to them in terms of distance and time. Zocdoc allows users to book an ap- pointment 24/7 and the appointments are not limited to a hospital

referral system or a geographic region [ 15]. While this study utilized Zocdoc for the intervention, these results may hold true for other online appointment systems that are in existence (e.g. DocASAP, insurance company websites of physician group online platforms) particularly those that identify providers willing to accept the patient’s insurance type. Our PC follow-up visit rates with new providers are comparable to the follow-up rates achieved by other trials with PC providers for whom patients already had an existing relationship.

We did not find that our intervention changed patients’ satisfaction with the ED visit, subsequent use of the ED or extent of recovery approx- imately one month after the index ED visit. Our intervention was fo- cused on improving the transition of care from the ED to the home setting and this one action of scheduling a follow-up appointment may not have been enough to change patients’ satisfaction with the care they received in the ED or it may be that patients do not associate ease of post-ED follow-up with the care they received in the ED.

As for the other secondary outcomes, there has been relatively little research on the impact of primary care and specialty follow-up on sub- sequent ED use and health. The only studies that have examined these outcomes to date have focused on patients with cardiovascular disease (i.e. chest pain, atrial fibrillation or heart failure). The studies have re- ported fewer subsequent ED visits and lower mortality among patients who receive follow-up within the first 14 to 30 days after the index ED visit. However, all of these studies were observational and subject to confounding. Further research is needed to better understand the bene- fits to patients of better linkage between emergency and ambulatory care.

In summary, the results of this pilot RCT are promising and warrant further replication in a larger trial that is able to objectively confirm the follow-up visit. Real-time appointment booking using a physician appointment-booking website for insured patients without PC pro- viders may represent an efficient mechanism to improve the transition between acute and ambulatory care.

Declaration of competing of interest

None of the authors have any financial relationships with bio- medical entities that are in conflict with the research carried out in this study. Our study intervention relied on Zocdoc, a free, web-based program that anyone can use. None of the authors have any financial re- lationship with this company or any other company that has a similar purpose. None of the authors have any patents related to online physi- cian scheduling. The first author received a $2500 resident research award from the Department of Emergency Medicine at the George Washington University toward study expenses. This money was used to pay for the gift cards given to subjects for completing the follow-up interview and for travel to present the study results at a conference.

Author contributions

RM and TC conceived the study. RM, TC, PK, and MM designed the trial. RM obtained a small amount of research funding. RM, PK, and AL supervised the conduct of the trial and data collection. PK and AL super- vised recruitment of patients and managed the data, including quality control. MM and PK provided statistical advice on study design and

Table 2

Percent distribution of patient and clinical characteristics by primary care follow-up visit status.

Table 3

Frequency Distribution of Secondary Outcomes By Type Of Primary Care Discharge Assis- tance Provided In Emergency Department Among Those Who Completed Telephone Fol- low-Up

Characteristic Overall

Follow-up visit

Not

N (%)

completed N (%)

completed N (%)

Outcome Overall

N (%)

Appointment Booked in

Booking website

Usual care

N = 185 N = 69 N = 116

ED N (%)

information given N (%)

N (%)

Age

sit

18-34	105 (57)	42 (61)	63 (54)
35-54	61 (33)	16 (23)	45 (39)	Satisfaction with ED vi
55+	19 (10)	11 (16)	8 (7)	Extremely satisfied

N = 185 N = 62 N = 64 N = 59

60 (32) 26 (42) 20 (31) 14 (24)

Male gender	84 (45)	28 (41)	56 (48)	Very satisfied	60 (32)	17 (27)	22 (34)	21 (36)
Education				Moderately satisfied	44 (24)	12 (19)	13 (20)	19 (32)
Less than high school	11 (6)	5 (8)	6 (5)	Slightly satisfied	14 (8)	4 (7)	8 (13)	2 (3)
High school diploma/GED	30 (16)	9 (13)	21 (18)	Not at all satisfied	7 (4)	3 (5)	1 (2)	3 (5)
Some college	56 (31)	24 (36)	32 (28) Satisfaction with process

Bachelor’s degree	49 (27)	16 (24)	33 (28)	of obtaining a PCP
Graduate degree or higher	37 (20)	13 (19)	24 (21)	follow-up

Marital status appointment^a,?

Single, never married	124 (67)	46 (68)	78 (67)	Extremely satisfied	44 (24)	20 (33)	12 (19)	12 (21)
Married or domestic partnership	42 (23)	13 (19)	29 (25)	Very satisfied	60 (33)	27 (45)	22 (35)	11 (19)
Widowed/divorced/separated	18 (10)	9 (13)	9 (8)	Moderately satisfied	41 (23)	6 (10)	15 (24)	20 (35)
Race Slightly satisfied 10 (6)						2 (4)	6 (9)	2 (3)
Black	105 (57)	43 (63)	62 (53)	Not at all satisfied	26 (14)	5 (8)	8 (13)	13 (22)
White	60 (33)	17 (25)	43 (37)	Subsequent ED visit	22 (12)	8 (13)	9 (14)	5 (8)
Asian/other	21 (10)	8 (12)	11 (10)	How do you feel now?
Hispanic/Latino ethnicity	16 (9)	5 (7)	11 (10) (0 = no better/worse,
Overall health				10 = completely better)
Excellent	28 (15)	8 (12)	20 (17)	0-3	22 (12)	5 (8)	8 (13)	9 (15)
Very good	65 (35)	29 (42)	36 (31)	4-7	50 (27)	19 (31)	15 (23)	16 (27)
Good	53 (29)	15 (22)	38 (33)	8-10	113 (61)	38 (61)	41 (64)	34 (58)
Fair	26 (14)	8 (12)	18 (16)	Date of follow-up visitb
Poor	12 (7)	8 (12)	4 (3)	b1 week after ED visit	21 (39)	9 (38)	5 (38)	7 (40)
Insurance type				1-2 weeks after ED visit	16 (31)	7 (29)	4 (31)	5 (30)

N2 weeks after ED visit 17 (30) 8 (33) 4 (31) 5 (30)

Private	120 (65)	42 (61)	78 (67)
Medicare +- Medicaid	3 (2)	3 (4)	0 (0)
Medicaid	57 (30)	24 (35)	33 (28)
Champus/TriCare	0 (0)	0 (0)	0 (0)
Worker’s comp	2 (1)	0 (0)	2 (2)
2 or more types of insurance	3 (2)	0 (0)	3 (3)
Triage Level 2	21 (11)	6 (9)	15 (13)
3	112 (61)	42 (61)	70 (60)
4-5	52 (28)	21 (30)	31 (27)
Treatment area

^a N=4 missing.

^b Based on number of subjects who reported follow-up with PCP and recalled follow-up visit date (N=54).

* p b 0.05.

Funding

The first author received a resident research award from the GWU

Main side	91 (49)	35 (51)	56 (48)	Department of Emergency Medicine.
Urgent care Chief complaint	94 (51)	34 (49)	60 (52)

Discharge diagnosis

Musculoskeletal/trauma	50 (27)	22 (32)	28 (24)
Abdominal pain/OBGYN	45 (25)	15 (22)	30 (26)
Headache/neurologic	30 (16)	14 (20)	16 (14)
Chest pain/SOB	21 (12)	3 (4)	18 (16)
Infectious	15 (8)	4 (6)	11 (10)
Other	22 (12)	10 (15)	12 (10)

References

1. Weiss AJ, Wier LM, Stocks C, Blanchard J. Overview of emergency department visits in the United States, 2011. HCUP statistical brief: #174. Overview of emergency de- partment visits in the United States; 2011 [Published June 2014. Accessed May 20, 2017].
2. Asplin BR, Rhodes KV, Levy H, Crain AL, Carlin BP, Kellermann AL. Insurance status

   Musculoskeletal/trauma	48 (27)	20 (31)	28 (25)	and access to urgent ambulatory care follow-up appointments. JAMA 2005;294
   Abdominal pain/OBGYN	33 (19)	13 (20)	20 (18)	(10):1248. https://doi.org/10.1001/jama.294.10.1248.
   Headache/neurologic	33 (19)	16 (25)	17 (15)	[3] Handel DA, McConnell J, Allen H, Lowe RA. Outpatient follow-up in today’s health
   Chest pain/SOB	18 (10)	2 (3)	16 (14)	care environment. Ann Emerg Med 2007;49(3):288-92. https://doi.org/10.1016/j.
   Infectious	19 (11)	6 (9)	13 (12)	annemergmed.2006.06.014.

   Other	26 (14)	8 (12)	18 (16) [4] Katz EB, Carrier ER, Umscheid CA, Pines JM. Comparative effectiveness of care coor-
   Provider recommended specialty care Provider’s rating of importance of	58 (34)	24 (40)	34 (30) dination interventions in the emergency department: a systematic review. Ann Emerg Med 2012;60(1):12-24. https://doi.org/10.1016/j.annemergmed.2012.02. 025.
   follow-up 0-4	0 (0)	0 (0)	[5] Atzema CL, Maclagan LC. The transition of care between emergency department and 0 (0) primary care: a scoping study. Acad Emerg Med 2017;24(2):201-15. https://doi.org/ 10.1111/acem.13125.
   5-7	90 (49)	34 (49)	56 (48) [6] Chan TC, Killeen JP, Castillo EM, et al. Impact of an internet-based emergency depart-
   8-10	95 (51)	35 (51)	60 (52) ment appointment system to access primary care at safety net community clinics.
   Patient’s rating of importance of follow-up			Ann Emerg Med 2009;54(2):279-84. https://doi.org/10.1016/j.annemergmed. 2008.10.030.

   analyzed the data. RM and MM drafted the manuscript, and all authors contributed substantially to its revision. RM takes responsibility for the paper as a whole.

   Ward BW, Clarke TC, Nugent CN, Schiller JS. Early release of selected estimates based on data from the 2015 National Health Interview Survey. National Health Interview Survey early release program. https://www.cdc.gov/nchs/data/nhis/earlyrelease/ earlyrelease201605.pdf. Published May 2016. Accessed May 20, 2017.

3. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Gonde JG. Research electronic data capture (REDCap) - a metadata-driven methodology and workflow process

   0-4	16 (9)	3 (4)	13 (11)	[7] Karaca-Mandi P, Jena AB, Ross JS. Health and Health care use among individuals at
   5-7	44 (24)	14 (20)	30 (26)	risk to lose health insurance with repeal of the affordable care act. JAMA Intern
   8-10	125 (67)	52 (76)	73 (63)	Med 2017;177(4):590-3. https://doi.org/10.1001/jamainternmed.2016.9541.

   for providing translational research informatics support. J Biomed Inform 2010;42 (2):377-81. https://doi.org/10.1016/j.jbi.2008.08.010.

   O’Brien GM, Stein MD, Fagan MJ, Shapiro MJ, Nasta A. Enhanced emergency depart- ment referral improves Primary care access. Am J Manag Care 1999;5(10):1265-9.

4. Sin DD, Bell NR, Man P. Effects of increased primary care access on process of care and health outcomes among patients with asthma who frequent emergency depart- ments. Am J Med 2004;117:479-83. https://doi.org/10.1016/j.amjmed.2004.04.011.
5. Zorc JJ, Scarfone RJ, Li Y, Hong T, Harmelin M, Grunstein L, et al. Scheduled follow-up after a pediatric emergency department visit for asthma: a randomized trial. Pediat- rics 2003;111(3):495-502.
6. Kyriacou DN, Handel D, Stein AC, Nelson RR. Brief report: factors affecting outpatient follow-up compliance of emergency department patients. J Gen Intern Med 2005;20 (10):938-42. https://doi.org/10.1111/j.1525-1497.2005.0216_1.x.
7. Hamaker JB, Baker BK, Weinberger M, Neill PJ. The emergency room and the hyper- tensive patient - improving linkage to clinical care. Med Care 1984;22(8):755-9.
8. Is Zocdoc available in my area? Zocdoc website. https://support.Zocdoc.com/ patients/s/article/ka10d0000000HG7AAM/Is-Zocdoc-near-me, Accessed date: 31 August 2017.