a b s t r a c t

Introduction: United States emergency medicine (EM) post-graduate training programs vary in training length, either 4 or 3 years. However, it is unknown if clinical care by graduates from the two curricula differs in the early post-residency period.

Methods: We performed a retrospective observational study comparing measures of clinical care and practice patterns between new graduates from 4- and 3-year EM programs with experienced new physician hires as a ref- erence group. We included emergency department (ED) encounters from a national EM group (2016-19) be- tween newly hired physicians from 4- and 3- year programs and experienced new hires (>2 years’ experience) during their first year of practice with the group. Primary outcomes were at the physician-shift level (patients per hour and relative value units [RVUs] per hour) and encounter-level (72-h return visits with admission/transfer and discharge length of stay [LOS]). Secondary outcomes included discharge opioid prescrip- tion rates, test ordering, computer tomography (CT) use, and admission/transfer rate. We compared outcomes using multivariable Linear regression models that included patient, shift, and facility-day characteristics, and a fa- cility fixed effect. We hypothesized that experienced new hires would be most efficient, followed by new 4-year graduates and then new 3-year graduates.

Results: We included 1,084,085 ED encounters by 4-year graduates (n = 39), 3-year graduates (n = 70), and ex- perienced new hires (n = 476). There were no differences in physician-level and encounter-level primary out- comes except discharge LOS was 10.60 min (2.551, 18.554) longer for 4-year graduates compared to experienced new hires. Secondary outcomes were similar among the three groups except 4- and 3-year new graduates were less likely to prescribe opioids to discharged patients, -3.70% (-5.768, -1.624) and – 3.38% (-5.136, -1.617) compared to experienced new hires.

Conclusions: In this sample, measures of clinical care and practice patterns related to efficiency, safety, and flow were largely similar between the physician groups; however, experienced new hires were more likely to pre- scribe opioids than new graduates. These results do not support recommending a specific length of residency training in EM.

(C) 2023

Abbreviations: ACGME, Accreditation Council for Graduate Medical Education; CK-MB, creatine kinase-MB; CPT, Current Procedure Terminology; CT, computed tomography; ED, emer- gency department; EM, emergency medicine; ESI, emergency severity index; ICD, International Classification of Diseases; LOS, length of stay; PD, program director; PT/INR, prothrombin/ international normalized ratio; RVU, relative value unit; SD, standard deviation; STROBE, Strengthening the Reporting of Observational Studies in Epidemiology; US, United States.

* Corresponding author at: US Acute Care Solutions, Canton, OH, United States of America.

E-mail address: dhimitri.nikolla@med.lecom.edu (D.A. Nikolla).

https://doi.org/10.1016/j.ajem.2023.04.017

0735-6757/(C) 2023

1. Introduction
   1. Background

Emergency medicine (EM) is one of only a few United States (US) specialties with a variable length of post-graduate training [1]. While various training lengths have been approved in the past, including a 2- year curriculum [2], the Accreditation Council for Graduate Medical Ed- ucation (ACGME) currently approves only 4- and 3-year EM programs with 3-years being more common, 59 (23%) vs. 198 (77%), respectively [2-4].

A couple of other ACGME-approved medical specialties have vari- able training lengths. For example, independent thoracic surgery fel- lowships, completed after general surgery residency, are approved for either 2 or 3 years [5]. Similarly, integrated interventional radiology res- idency programs are approved for either 5 or 6 years [6]. Like 4-year EM programs, thoracic surgery and interventional radiology residencies with longer training lengths must justify the additional time with an ex- periential or educational rationale for ACGME approval [3,5,6]. In EM, scholarly tracks are often integrated into the 4-year curriculum provid- ing residents with additional specialized training during residency [7,8].

• 1. Importance

There is disagreement regarding the ideal length of EM residency [9,10]. While 4-year programs may increase opportunities to perform rare procedures under supervision and 4-year graduates may be more likely to pursue Fellowship training and academic positions [11-13], jus- tification for the 4-year curriculum is largely based on more instruc- tional hours, electives, scholarship, niche development, and supervised clinical experience [1,14]. We are unaware of any published studies ex- amining clinical care in the early post-residency period between gradu- ates from the two curriculums.

• 1. Study objectives“>Study objectives

We compare practice patterns between new graduates from 4- and 3-year programs with experienced new physician hires, using a large dataset of emergency department (ED) encounters from a national emergency medicine group. Adding experienced new hires as a com- parator mitigates any practice differences that may be due to starting a new job rather than training length. Given the extra year of training and prior work showing increasED efficiency as EM residents progress through residency [15], we hypothesized that new 4-year graduates would practice differently than new 3-year graduates, in particular, new 4-year graduates would be more efficient. We also hypothesized that experienced new hires would have the greatest efficiency.

1. Materials and methods
   1. Study design and setting

Using administrative data abstracted from ED billing records, elec- tronic health records, scheduling software (Shift Admin, Columbia, SC), and credentialing records, we performed a retrospective observa- tional study examining ED care among new 4-year graduates, new 3- year graduates, and experienced new physician hires. Encounters from 2016 to 2019 were included from general EDs (non-pediatric and non-free standing) where at least one newly hired physician from each group had worked within the study period. We report our findings in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) recommendations [16]. Our study was considered exempt by the Allegheny Health Network Institutional Review Board.

• 1. Data collection

The datasets used in this study have been described previously [17,18], and all variables are defined in a data dictionary. The US Acute Care Solutions Research Group works with the billing, human resources, and information technology departments of the group to consolidate de-identified data into research datasets. Encounter variables were ab- stracted with standardized tools and processes by trained billing spe- cialists, who were required to obtain relevant certifications by their second and third year of employment. These billing specialists undergo ongoing training, auditing, and external evaluation. Encounter variables examined included patient age, patient gender, ED arrival and departure times, Emergency Severity Index , relative value units (RVUs) gen- erated, disposition, procedure and laboratory codes (Current Procedure Terminology [CPT] codes), ED diagnosis (International Classification of Diseases [ICD]-10 codes), patient payer category (i.e., commercial, Medicare, Medicaid, self-pay, other), ED characteristics (i.e., trauma center designation, teaching hospital status, volume, US region, metro status), and physician directly caring for the patient. Physician variables included physician age, gender, hire dates, residency length, and resi- dency graduation dates. Scheduling data were abstracted to create a physician-shift level database with aggregated Encounter data experi- enced during the shift.

• 1. Study population

We included encounters during the physician’s first year of practice with this group at general EDs for physicians hired between January 1, 2016 and January 1, 2019. We excluded encounters occurring during physician shifts lasting less than one hour, encounters that left before being seen, and encounters without direct care provided by the physi- cian (i.e., supervision of visits seen primarily by non-physician practi- tioners). However, encounters involving joint care with non-physician practitioners and resident physicians were included. The group of expe- rienced new hires included physicians with EM residency graduation dates prior to 2015 and at least 2 years between their graduation date and their first patient encounter with this group. The groups of 4- and 3-year new graduates included physicians with EM residency gradua- tion dates between January 1, 2016 and January 1, 2019. 4- and 3-year new graduates were excluded if their first patient encounter occurred

>90 days after their residency graduation date. Physicians were ex-

cluded from the study if they worked less than one year, had unknown residency training length, fellowship training, or residency training <3 or > 4 years.

• 1. Outcome measures

Primary outcomes were determined a priori and were chosen to il- lustrate differences in efficiency, safety, and flow of clinical care be- tween groups. Therefore, the primary outcomes included patients seen per clinical hour and RVUs per clinical hour (measured at the physician-shift level) as well as encounters that returned to the same ED within 72-h with admission/transfer and length of stay [LOS] in mi- nutes for discharged patients (measured at the encounter-level). Sec- ondary outcomes focused on measures of clinical care including other productivity measures (i.e., RVUs per encounter), test ordering practices (i.e., using d-dimer before computed tomography (CT) angiography for pulmonary embolism, avoiding amylase when lipase is available, avoiding creatine kinase-MB (CK-MB) when troponin is available, avoiding prothrombin/international normalized ratio (PT/INR) for discharged chest pain patients, avoiding head CT in syncope, pregnancy testing in women 10-50 years old presenting with a chief complaint of abdominal pain), and utilization measures (i.e., encounters with: any CT order, admitted/transferred disposition, prescription of an opioid at dis- charge). Test ordering and utilization measures were chosen that may indicate differences in Safety outcomes or low-yield practices.

• 1. Missing data

Not all facilities included in the study consistently reported ESI level, medication, radiology, laboratory, and chief complaint data. ESI level was missing on 24% of encounters (N = 261,589), discharge medication data were missing on 59% of encounters (N = 641,309), radiology data were missing on 25% of encounters (N = 266,143), and chief complaint data were missing on 23% of encounters (N = 244,247). As a result, the number of encounters and physicians in- cluded in the regression models vary depending on the outcome measure.

• 1. Data analysis

Descriptive statistics (e.g., means, proportions) of patient demo- graphic and clinical characteristics were calculated for each physician group. To compare outcomes between physician groups, we performed multivariable linear regression models at the physician-shift or encounter-level. Physician-shift level outcomes included patients per hour and RVUs per hour, while encounter-level outcomes included 72-h returns with admission/transfer, discharge LOS, and the measures of clinical care listed as secondary outcomes. To reduce the influence of large outliers, LOS and RVUs were winsorized at the 99th percentile

Records identified for screening:

New Grad database (n=343)

Exp. New Hire database (n = 2,392)

Records screened against visit and scheduling database New Grad database (n=172)

Exp. New Hire database (n=858)

New Grad database exclusions (n=171):

• Graduated outside study period (n=128)
• Hired outside study period (n=143)
• Fellowship training (n=15)
• Not from a 3- or 4-year program (n=9)

Exp. New Hire database exclusions (n=1,191):

• Hired outside study period (n=1,191)

New Grad database exclusions (n=63):

• First general ED visit after 1/1/2019 (n=2)
• First general ED visit before residency completion (n=4)
• First general ED visit >90 days since residency completion (n=57)
• Last shift at a general ED <360 days from first shift at a general ED (n=4)

Exp. New Hire database exclusions (n=382):

• First general ED visit after 1/1/2019 (n=12)
• Last shift at a general ED <360 days from first shift at a general ED (n=382)

New Grads Included (n=109):

3-year grads (n=70)

4-year grads (n=39)

Exp. New Hires Included (n=476)

Fig. 1. Flow diagram of study cohort inclusions and exclusions. Exclusions are not mutually exclusive; therefore, individual exclusions may not sum to total exclusions. Abbreviations. Exp, experienced; grad, graduate; ED, emergency department.

(i.e., observations above the 99th percentile are set to the 99th percentile).

Models for physician-shift level outcomes controlled for differences in patient and clinical characteristics of encounters seen during the shift, facility characteristics on the day the shift began, and shift characteris- tics. Details on covariate and outcome coding are in the Technical Appendix (Table A1); however, in brief, patient and clinical characteris- tics included patient age, average ESI level, and patient gender. Facility- day characteristics included average admitted LOS, daily encounter vol- ume, and average ED admission rate. Shift characteristics includED shift length, shift start time, overnight shift, and shift day of week, month, and year. Encounter-level models included similar patient and clinical covariates as the physician-shift models, but also included payer source (Medicare, Medicaid, commercial, self, and other) and do not include the shift length or overnight shift. ED visit rates and case mix differ be- tween payer source [19]; therefore, payer source was included for

encounter-level models. Both physician-shift and encounter-level models included an individual facility fixed-effect to control for all time-invariant differences between facilities. To visually examine differ- ences in the primary outcomes over time, we added an interaction term to the models (Group x Time) and plotted the predicted outcomes with confidence intervals in 30-day intervals. Lastly, a sensitivity analysis was performed after multiple imputation of missing ESI data using an ordered logistic regression imputation algorithm with 10 imputations. Standard errors were clustered at the facility-level. All analyses were conducted using Stata v. 17 (StataCorp LLC, College Station, TX). Additional methodological details are in the Technical Appendix (Tables A1-A4).

We did not perform an a priori sample size calculation as there were no known differences in clinical care between groups to guide an esti- mate. In addition, clinically and operationally meaningful differences in these outcomes vary greatly by practice setting.

Table 1

Emergency Physician and Emergency Department visit characteristics.

3-year Graduates 4-year Graduates Experienced New Hires Total

	N	(%)	N	(%)	N	(%)		N	(%)
Total encounters	154,963		89,115		840,007			1,084,085
Total clinicians	70		39		476			585
Physician age, mean (SD)	31.7	(3.4)	32.9	(3.9)	44.1	(10.8)		41.9	(10.9)
Female physicians	27	(38.6)	16	(41.0)	151	(31.7)		194	(32.5)
Days since graduation, mean (SD)	33.4	(22.9)	28.2	(18.1)	–	–		31.5	(21.4)
Worked in a trauma center	24	(34.3)	17	(43.6)	147	(30.9)		188	(31.7)
Worked in a teaching hospital	5	(7.1)	9	(23.1)	57	(12.0)		71	(12.3)
Worked in ED with Volume
<30 k	18	(25.7)	11	(28.2)	172	(36.1)		201	(35.3)
30 k-59,999	47	(67.1)	28	(71.8)	251	(52.7)		326	(54.3)
60 k and over	29	(41.4)	13	(33.3)	171	(35.9)		213	(36.0)
Metro status of ED worked in
Non-Metro	12	(17.1) 6		(15.4) 123		(25.8)	141		(24.9)
Metro	60	(85.7) 33		(84.6) 374		(78.6)	467		(79.2)
Region of ED worked in
Midwest	19	(27.1) 9		(23.1) 98		(20.6)	126		(21.0)
Northeast	9	(12.9) 5		(12.8) 78		(16.4)	92		(16.1)
South	33	(47.1) 16		(41.0) 231		(48.5)	280		(48.1)
West	9	(12.9) 9		(23.1) 80		(16.8)	98		(16.9)
Patient age, mean (SD)	49.9	(23.1) 49.1		(23.3) 46.5		(24.3)	47.2		(24.1)
Patient age categories
<10 years	5918	(3.8) 3765		(4.2) 62,262		(7.4)	71,945		(7.0)
10-17 years	4878	(3.1) 3083		(3.5) 39,705		(4.7)	47,666		(4.6)
18-54 years	75,801	(48.9) 43,986		(49.4) 405,383		(48.3)	525,170		(48.4)
55 and over	68,328	(44.1) 38,254		(42.9) 332,490		(39.6)	439,072		(40.0)
Female patients	87,177	(56.3) 49,882		(56.0) 463,866		(55.2)	600,925		(55.3)
ESI Triage Level
1	1742	(1.4) 1635		(2.0) 8283		(1.3)	11,660		(1.4)
2	27,916	(22.3) 19,495		(23.8) 133,902		(21.8)	181,313		(21.9)
3	77,285	(61.7) 46,219		(56.4) 338,997		(55.1)	462,501		(55.6)
4	17,237	(13.8) 13,112		(16.0) 121,967		(19.8)	152,316		(19.2)
5	1052	(0.8) 1536		(1.9) 12,118		(2.0)	14,706		(1.9)
Disposition, %
Admitted/Transfer	45,432	(29.3) 27,238		(30.6) 227,751		(27.1)	300,421		(27.5)
Discharged	106,653	(68.9) 60,121		(67.6) 595,525		(70.9)	762,299		(70.6)
LWT	540	(0.3) 347		(0.4) 3971		(0.5)	4858		(0.5)
AMA	1658	(1.1) 997		(1.1) 9653		(1.2)	12,308		(1.1)
ED Death/DOA	468	(0.3) 257		(0.3) 2156		(0.3)	2881		(0.3)
Other	106	(0.1) 32		(0.0) 321		(0.0)	459		(0.0)
Payer source
Commercial	42,059	(27.1) 24,226		(27.2) 211,692		(25.2)	277,977		(25.4)
Medicare	53,653	(34.6) 29,973		(33.6) 262,839		(31.3)	346,465		(31.6)
Medicaid	37,105	(23.9) 21,563		(24.2) 240,048		(28.6)	298,716		(28.1)
Self-Pay	19,962	(12.9) 11,364		(12.8) 110,458		(13.1)	141,784		(13.1)
Other	2184	(1.4) 1989		(2.2) 14,970		(1.8)	19,143		(1.8)
Time of arrival
Midnight to 7 am	20,020	(12.9) 9803		(11.0) 129,388		(15.4)	159,211		(15.0)
7 am to 3 pm	62,608	(40.4) 36,732		(41.2) 331,991		(39.5)	431,331		(39.7)
3 pm to midnight	72,335	(46.7) 42,580		(47.8) 378,628		(45.1)	493,543		(45.3)

SD, standard deviation; ED, emergency department; ESI, emergency severity index; LWT, left without treatment; AMA, against medical advice; DOA, dead on arrival.

Table 2

Unadjusted Outcomes Comparing Practice Patterns in Emergency Physicians By Training Length and Experience

Outcome 3-year Graduates 4-year Graduates Experienced New Hires

Primary	Patients seen per hour worked, Mean (SD)	1.7	(0.6)	1.8	(0.7)	1.6	(0.6)
	RVUs generated per hour worked, Mean (SD)	7.4	(2.6)	8.0	(2.8)	6.6	(2.7)
	72-h return and admitted or transferred, N (%)	1322	(1.3)	639	(1.1)	6815	(1.2)
	Length of stay, discharged patients	227	(146.6)	234.1	(158.5)	203	(149.9)
Secondary	Any encounter admitted or transferred, N (%)	45,432	(29.3)	27,238	(30.6)	227,751	(27.1)
	Discharged with an Opioid prescription, N (%)	5680	(9)	2773	(8.9)	24,534	(11.7)
	D-dimer with CTA-PE, N (%)	1305	(27.5)	756	(24.3)	5190	(30.5)
	Any encounter with a CT scan, N (%)	43,355	(33.8)	26,513	(32.1)	170,348	(28)
	Amylase with Lipase, N (%)	278	(1.3)	197	(1.3)	4695	(4.9)
	Pregnancy test with abdominal pain, N (%)	3771	(74)	2215	(67.6)	16,048	(67.6)
	CK-MB with troponin, N (%)	2357	(5.5)	2216	(9.5)	13,977	(8.3)
	RVUs per visit, Mean (SD)	4.4	(1.5)	4.3	(1.5)	4.2	(1.5)
	Discharged chest pain visits with a PT/INR, N (%)	1258	(13.7)	940	(18.5)	6228	(17.7)
	Syncope visits with a head CT scan, N (%)	892	(39.6)	505	(35.8)	3575	(39.8)

SD, standard deviation; RVU, relative value units; CTA-PE, computed tomography angiography for pulmonary embolism; CT, computed tomography; CK-MB, creatine kinase-MB. Notes: Opioid prescription denominator includes all discharged encounters, D-dimer with CTA-PE denominator includes all encounters with a CTA-PE order, amylase with lipase denom- inator includes all encounters with a lipase order, pregnancy test with abdominal pain denominator includes all encounters with female patients ages 10-50 with a chief complaint of abdominal pain, troponin with CK-MB denominator includes all encounters with a troponin order, any CT scan and admitted/transferred denominators include all encounters, discharged chest pain with PT/INR denominator includes discharged encounters with a chief complaint of chest pain, and syncope with head CT scan denominator includes encounters with a diag- nosis of syncope excluding cases with a trauma diagnosis and >= 2 CT scan orders (Technical Appendix Tables A2 and A3).

1. Results
   1. Physician characteristics

From 2735 physicians working in general EDs during the study pe- riod, we included, 39 new 4-year graduates, 70 new 3-year graduates, and 476 experienced new hires (Fig. 1). New 4-year graduates trained at 26 unique post-graduate residency programs, while new 3-year grad- uates trained at 44 unique programs. Mean age (years) and standard deviation (SD) for physicians were 32.9 (3.9) for new 4-year graduates,

31.7 (3.4) for new 3-year graduates, and 44.1 (10.8) for experienced new hires. The number of female physicians in each group were 16 (41.0%) new 4-year graduates, 27 (38.6%) new 3-year graduates, and 151 (31.7%) experienced new hires. Lastly, the mean days since gradu- ation and SD for the new graduates was 28.2 (18.1) for 4-years and

33.4 (22.9) for 3-years (Table 1).

• 1. Encounter characteristics“>Encounter characteristics

We analyzed 89,115 new 4-year graduate, 154,963 new 3-year grad- uate, and 840,007 experienced new hire ED encounters from 125 sites. Mean age (years) and SD for patients were 49.1 (23.3) for new 4-year graduates, 49.9 (23.1) for new 3-year graduates, and 46.5 (24.3) for ex- perienced new hires. The number of female patients seen in each group were 49,882 (56.0%) new 4-year graduates, 87,177 (56.3%) new 3-year graduates, and 463,866 (55.2%) experienced new hires (Table 1).

• 1. Primary outcomes

Unadjusted physician-shift and encounter-level outcomes were similar between the three physician groups (Table 2). Similarly, compared to new 4-year graduates, new 3-year graduates had similar estimates including patients per hour (0.04 [95% confidence inter- val – 0.035, 0.112]), RVUs per hour (0.11 [-0.174, 0.390]), 72-h return visits with admission/transfer (0.06% [-0.131, 0.250]), and discharge LOS (-6.03 min [-18.199, 6.134]) in the adjusted analysis. Also, com- pared to experienced new hires, outcomes were similar for new 4- and 3-year graduates except for discharge LOS where the 4-year graduate estimate was 10.60 min [2.551, 18.554] longer (Table 3).

• 1. Secondary outcomes

Secondary outcomes were also largely similar among the three groups with select secondary outcomes differing slightly. Both new

graduate groups prescribed fewer discharge opioids compared to expe- rienced new hires, -3.70% (95% confidence interval -5.768, -1.624)

for new 4-year graduates and -3.38% (-5.136, -1.617) for new 3-

year graduates. Also, both new graduate groups had fewer amylase or- ders among patients who had lipase orders compared to experienced new hires, -1.43% (-2.545, -0.311) for new 4-year graduates and

-1.36% (-2.655, -0.056) for new 3-year graduates. Lastly, new 3- year graduates admitted or transferred fewer patients compared to new 4-year graduates, -1.78% (-3.263, -0.294) (Fig. 2 & Supplemen- tal Table S1).

Table 3

Fixed-Effect Multivariable Linear Models for Primary Outcomes Comparing Practice Pat- terns in Emergency Physicians By Training Length and Experience

Primary Outcomes Coefficient 95% CI Patients per Hour*

3-year grads (Ref = Exp. New hires) 0.05 (-0.012, 0.116)

4-year grads (Ref = Exp. New hires) 0.01 (-0.049, 0.075) 3-year grads (Ref = 4-year grads) 0.04 (-0.035, 0.112)

RVUs per Hour*

3-year grads (Ref = Exp. New hires) 0.20 (-0.062, 0.457)

4-year grads (Ref = Exp. New hires) 0.09 (-0.188, 0.367) 3-year grads (Ref = 4-year grads) 0.11 (-0.174, 0.390)

72-Hour Returns with Admission/ Transfer**

3-year grads (Ref = Exp. New hires) -0.09 (-0.222, 0.049)

4-year grads (Ref = Exp. New hires) -0.15 (-0.315, 0.023) 3-year grads (Ref = 4-year grads) 0.06 (-0.131, 0.250)

Discharged Length of Stay**

3-year grads (Ref = Exp. New hires)	4.50	(-4.580, 13.620)
4-year grads (Ref = Exp. New hires)	10.60	(2.551, 18.554)
3-year grads (Ref = 4-year grads)	-6.03	(-18.199, 6.134)

Ref, Reference; CI, confidence interval; RVU, relative value units.

Notes: *Physician-shift level outcomes: N = 55,662 physician shifts covering the first 12 months of working in a facility (n = 111). Physician-shift level model covariates in- clude: patient age (percent over 55 and percent under 10), average ESI, patient gender (percent female), shift length (in hours), facility average admitted LOS (for day shift began), facility daily volume (for day shift began), facility average admission rate (for day shift began), shift start time (midnight to 7 am, 7 am to 3 pm, 3 pm to midnight), over- night shift (yes/no), shift day of week, year, and month, and an individual facility fixed- effect. **Encounter-level outcomes only include discharged visits. N = 544,306 (72-h returns with admission/transfer) and N = 567,333 (discharged LOS). Encounter-level models include similar covariates as the physician-shift level models, including an individ- ual facility fixed-effect, but also include a covariate for payer source (Medicare, Medicaid, commercial, self, and other) and do not include shift length or overnight shift. 95% confi- dence intervals in parentheses. Additional methodological details are in the Technical Appendix (Tables A1-A4).

Fig. 2. Graphical display of coefficients with 95% confidence intervals from fixed-effect multivariable linear models for 4-year graduates vs. 3-year and new emergency medicine graduates (4-year and 3-year training programs) vs. experienced new hires for each secondary outcome

Abbreviations. yr, year; CT, computed tomography; RVU, relative value unit; PT, prothrombin time; INR, international normalized ratio; CTA-PE, computed tomography angiography for pulmonary embolism; CK-MB, creatine kinase-MB; CI, confidence interval.

• 1. Trends

All three physician groups followed similar trends and curves when outcomes were plotted over one year by month where patients per hour improved over the first four months then leveled off and discharge length of stay decreased. However, there did not appear to be a trend in 72-h return visits with admission/transfer (Fig. 3).

• 1. Sensitivity analyses

After multiple imputation, primary outcomes were similar between physician groups; however, new 4-year graduates still had a slightly longer discharge LOS compared to experienced new hires, 10.74 min

(95% confidence interval 2.218, 19.253) (Supplemental Table S2). In ad- dition, new graduates prescribed fewer opioids at discharge and had fewer amylase orders than experienced new hires; however, 3-year graduates had a similar proportion admitted/transferred compared to 4-year graduates, -1.23% (-2.593, 0.127) (Supplemental Table S3). Lastly, group trends over time in primary outcomes were similar to those without multiple imputation (Supplemental Fig. S1).

1. Discussion

We examined practice patterns between three groups of newly hired emergency physicians, new graduates from either 4- or 3-year programs and those with >2 years of post-residency work experience.

Fig. 3. Plots display trends in co-primary outcomes (y-axes) comparing 4-year emergency medicine graduates to 3-year graduates and experienced new hires over the first year of practice with the group with point estimates by month (x-axes) surrounded by 95% confidence intervals.

Abbreviations. RVU, relative value units; LOS, length of stay.

Ultimately, we observed similar estimates in patients per hour, RVUs per hour, 72-h return visits with admission/transfer, and discharge LOS between the three groups of physicians suggesting that clinical care, as measured by these outcomes, is not substantially different based on length of training or experience. However, compared to expe- rienced new hires, new graduates had slightly fewer discharge Opioid prescriptions and amylase with lipase orders suggesting that new grad- uates may be better at avoiding opioid therapy and ordering fewer unnecessary tests for select indications.

Nevertheless, despite these small differences, we view our results as a largely negative study for a couple reasons. First, these findings were not consistent across outcomes and the differences were not Clinically meaningful. Furthermore, similar trends were observed in all three groups among the co-primary outcomes over their first year of practice with this group. Therefore, these results support the notion that training length and experience do not substantially impact these practice patterns.

The ACGME has already de-emphasized time-based achievement by transitioning to a competency-based model for accredited post- graduate residency and fellowship programs [20-22]. While competency-based medical education appears to guide the future direc- tion of post-graduate trainee assessments [21,22], theoretical and prac- tical barriers exist [21]. For example, the COVID-19 pandemic highlighted the dilemma of broad- vs. specialty-based training when non-acute care trainees were deployed to work in Acute care settings forcing them to utilize their basic medical training [21]. In addition, the current funding for graduate medical education uses time-based criteria [23]. Nevertheless, our results suggest that the current model of 4- or 3-year curricula provides sufficient training to practice similarly to more experienced physicians.

Our study differs from prior work assessing differences between EM program training lengths by examining practice patterns during the early post-residency period. Hopson, et al. surveyed EM program direc- tor (PD) attitudes regarding the ideal length of post-graduate EM train- ing [1]. PDs reported ideal training lengths between 4- and 3-years; however, responses were correlated with their training and work expe- riences [1]. In addition, a survey of 92 EM programs by Hayden, et al. found wide variation in the procedural experiences of EM residents from various curricula (i.e., post-graduate years 1-3, 1-4, and 2-4) with similar average procedure counts for most procedures across cur- ricula [11]. However, our study provides insight into similarities and dif- ferences between 4- and 3-year graduates in their first year of practice as it compares to experienced new hires.

• 1. Limitations

This was a retrospective observational study of administrative data in a small sample of physicians; therefore, our results may be at risk of bias from unmeasured confounders, including differences in training site and other characteristics of physicians. Similarly, missingness of some variables may have impacted the study results; however, out- comes were similar after multiple imputation. The study was also con- ducted within a single, multi-state group that has specific practices taught to all physicians including a centralized learning platform and clinical protocols. Therefore, study results may be different in different physician groups or EDs. In addition, despite performing multivariable regression to adjust for known confounders like facility and date/time, differences in where and when each physician group works may have biased the results. For example, although the individual facility fixed ef- fect functionally prevents comparisons between facilities, it does not ac- count for changes over time within each facility (e.g., change in payer mix).

Measuring competency and safety of clinical care between physi- cians also has limitations and challenges. Although productivity mea- sures were readily available, quality measures in emergency medicine largely focus on performance of the acute care system rather than that

of individual physicians [24]. In addition, known quality measures were inconsistently measured among sites over the study period in our dataset limiting their applicability [25]. In addition to 72-h return visits with admission/transfer, we considered number of malpractice claims and incident reports. However, these occurred too infrequently among newly hired physicians in their first year of practice with the group to reasonably make comparisons [17]. Similarly, patient satisfac- tion surveys were considered, but emergency physician Press Ganey percentile rank has poor reliability [26].

Therefore, our results should be considered directional rather than definitive. For example, there may be other ways that shorter and longer training may be better or worse that we did not examine (e.g., breadth of knowledge and skills, Opportunity cost, and trainee well-being among others). Finally, we tested multiple study outcomes and did not adjust for multiplicity. Therefore, some of the statistically significant differences may have occurred by chance and may not represent real differences in clinical practice among the groups.

1. Conclusions

In conclusion, after controlling for multiple confounders including patient acuity and facility-level factors, measures of clinical care and practice patterns were similar in the first year of practice between new graduates from 4- and 3-year EM residency programs and experi- enced new hires with >2 years of experience. A broader examination of academic and clinical care differences, as well as patient-centered outcomes is necessary to ultimately determine the optimal EM post- graduate training length.

Presentations

Lightning Oral presentation at SAEM22 on May 12th, 2022 in New Orleans, LA.

Financial support

No funding was obtained for this study.

CRediT authorship contribution statement

Dhimitri A. Nikolla: Writing – review & editing, Writing – original draft, Methodology, Formal analysis, Conceptualization. Mark S. Zocchi: Writing – review & editing, Methodology, Formal analysis. Jesse M. Pines: Writing – review & editing, Methodology. Amy H. Kaji: Writing – review & editing, Methodology. Arvind Venkat: Writing – review & editing, Methodology, Conceptualization. Michael S. Beeson: Writing – review & editing, Supervision. Jestin N. Carlson: Writing – review & editing, Supervision, Methodology, Formal analysis, Conceptualization.

Declaration of Competing Interest

JMP reports unrelated work with CSL Behring, Medtronic, and Abbott Point-of-Care.

Acknowledgments

We thank Merle A. Carter, MD and John Bedolla, MD for their feed- back and support.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi. org/10.1016/j.ajem.2023.04.017.

References

1. Hopson L, Regan L, Gisondi MA, Cranford JA, Branzetti J. Program director opinion on the ideal length of residency training in emergency medicine. Acad Emerg Med. 2016;23(7):823-7. https://doi.org/10.1111/acem.12968.
2. Nelson LS, Keim SM, Ankel FK, Beeson MS, Calderon Y, Chudnofsky CR, et al. American board of emergency medicine report on residency and fellowship training information (2019-2020). Ann Emerg Med. 2020;75(5):648-67. https://doi.org/10. 1016/j.annemergmed.2020.03.012.
3. ACGME Program Requirements for Graduate Medical Education in Emergency Medicine. Accreditation Council for Graduate Medical Education (ACGME); 2021; 1-65.
4. Policy on Emergency Medicine Residency Training Requirements. American Board of

Emergency Medicine; 2021.

1. ACGME Program Requirements for Graduate Medical Education in Thoracic Surgery (Independent). Accreditation Council for Graduate Medical Education (ACGME); 2021; 1-55.
2. ACGME Program Requirements for Graduate Medical Education in Interventional Radiology. Accreditation Council for Graduate Medical Education (ACGME); 2021; 1-81.
3. Jordan J, Hwang M, Kaji AH, Coates WC. Scholarly tracks in emergency medicine res- idency programs are associated with increased choice of academic career. West J Emerg Med. 2018;19(3):593-9. https://doi.org/10.5811/westjem.2018.1.36753.
4. Jordan J, Hwang M, Coates WC. Academic career preparation for residents – are we on the right track? Prevalence of specialized tracks in emergency medicine training programs. BMC Med Educ. 2018;18(1):184. https://doi.org/10.1186/s12909-018- 1288-x.
5. Rosen P, Hamilton GC. Three Year Verses Four Year Residency. Clerkship Directors in Emergency Medicine (CDEM); 2009.
6. Langdorf M, Lotfipour S. Advantages of a three-year residency. Cal J Emerg Med. 2004;5(1):15-7. https://pubmed.ncbi.nlm.nih.gov/20847857, https://www.ncbi. nlm.nih.gov/pmc/articles/PMC2906974/.
7. Hayden SR, Panacek EA. Procedural competency in emergency medicine: the current range of resident experience. Acad Emerg Med. 1999;6(7):728-35. https://doi.org/ 10.1111/j.1553-2712.1999.tb00444.x.
8. Ehmann MR, Klein EY, Kelen GD, Regan L. Emergency medicine career outcomes and scholarly pursuits: the impact of transitioning from a three-year to a four-year niche-based residency curriculum. AEM Educ Train. 2021;5(1):43-51. https://doi. org/10.1002/aet2.10435.
9. Lubavin BV, Langdorf MI, Blasko BJ. The effect of emergency medicine residency for- mat on pursuit of fellowship training and an academic career. Acad Emerg Med. 2004;11(9):938-43. https://doi.org/10.1197/j.aem.2004.03.019.
10. Ross TM, Wolfe RE, Murano T, Alleyne P, Dyne PL, Cheaito MA, et al. Three- vs. four- year emergency medicine training programs. J Emerg Med. 2019;57(5):e161-5. https://doi.org/10.1016/j.jemermed.2019.07.013.
11. Turner-Lawrence D, Todd BR. Monthly progression of emergency medicine resident efficiency: what can we expect of our residents throughout training? J Emerg Med. 2019;57(1):77-81. https://doi.org/10.1016/j.jemermed.2019.03.037.
12. Vandenbroucke JP, von Elm E, Altman DG, Gotzsche PC, Mulrow CD, Pocock SJ, et al. Strengthening the reporting of observational studies in epidemiology (STROBE): ex- planation and elaboration. PLoS Med. 2007;4(10):e297. https://doi.org/10.1371/ journal.pmed.0040297.
13. Carlson JN, Foster KM, Pines JM, Corbit CK, Ward MJ, Hydari MZ, et al. Provider and practice factors associated with emergency physicians’ being named in a malpractice claim. Ann Emerg Med. 2018;71(2):157-64.e4. https://doi.org/10. 1016/j.annemergmed.2017.06.023.
14. Pines JM, Zocchi MS, De Maio VJ, Carlson JN, Bedolla J, Venkat A. The effect of oper- ational stressors on emergency department clinician scheduling and Patient throughput. Ann Emerg Med. 2020;76(5):646-58. https://doi.org/10.1016/j. annemergmed.2020.02.005.
15. Pickens G, Karaca Z, Gibson TB, Cutler E, Dworsky M, Moore B, et al. Changes in hos- pital service demand, cost, and patient illness severity following health reform. Health Serv Res. 2019;54(4):739-51. https://doi.org/10.1111/1475-6773.13165.
16. Frank JR, Snell LS, Cate OT, Holmboe ES, Carraccio C, Swing SR, et al. Competency- based medical education: theory to practice. Med Teach. 2010;32(8):638-45. https://doi.org/10.3109/0142159x.2010.501190.
17. Ryan MS, Holmboe ES, Chandra S. Competency-based medical education: consider- ing its past, present, and a post-COVID-19 era. Acad Med. 2022;97(3s):S90-s7. https://doi.org/10.1097/acm.0000000000004535.
18. Edgar L, Roberts S, Holmboe E. Milestones 2.0: a step forward. J Grad Med Educ. 2018;10(3):367-9. https://doi.org/10.4300/JGME-D-18-00372.1.
19. Medicare payments for graduate medical education: What every medical student, resident, and advisor needs to know. Association of American Medical Colleges; 2019; 1-12.
20. Hansen K, Boyle A, Holroyd B, Phillips G, Benger J, Chartier LB, et al. Updated frame- work on quality and safety in emergency medicine. Emerg Med J. 2020;37(7): 437-42. https://doi.org/10.1136/emermed-2019-209290.
21. 2021 Measures. [accessed 2021-08-03]. https://www.acep.org/cedr/measures/;

2021.

1. Pines JM, Penninti P, Alfaraj S, Carlson JN, Colfer O, Corbit CK, et al. Measurement under the microscope: high variability and limited construct validity in emergency department patient-experience scores. Ann Emerg Med. 2018;71(5):545-54.e6. https://doi.org/10.1016/j.annemergmed.2017.11.011.