a b s t r a c t

Study objective: We sought to examine the frequency of pediatric critical procedures performed in a national group of emergency physicians.

Methods: We performed a retrospective analysis of an administrative billing and coding dataset for procedural performance documentation verification from 2014 to 2018. We describe and compare incident rates of pediatric (age b18 years) patient critical Procedure performance by emergency physicians in general emergency depart- ments (EDs), pediatric EDs, and freestanding ED/urgent care centers. Critical procedures were endotracheal intu- bation, Electrical cardioversion, central Venous placement, intraosseous access, and chest tube insertion.

Results: Among 2290 emergency physicians working in 186 EDs (1844 working in 129 general EDs, 125 in 8 pe- diatric EDs, and 321 in 49 freestanding EDs/urgent cares), a total of 2233 pediatric critical procedures were per- formed during the study period. Many physicians at general EDs and freestanding EDs/urgent cares performed zero pediatric procedures per year (53.9% and 89% respectively). Per 1000 ED visits seen (All patient ages), phy- sicians working in general EDs performed fewer pediatric critical procedures than physicians in pediatric EDs (0.12/1000 visits vs 0.68/1000 visits; rate difference = 0.56, 95% confidence interval [CI] 0.51-0.61). Per 1000 clinical hours worked, physicians working in general EDs performed 0.26 procedures compared to 1.66 for phy- sicians in pediatric EDs (rate difference = 1.39; 95% CI 1.27-1.52).

Conclusion: Pediatric critical procedures are rarely performed by emergency physicians and are exceedingly rare in general EDs and freestanding EDs/urgent cares. The rarity of performance of these skills has implications for ED pediatric readiness.

(C) 2020

Introduction

Nearly 70% of emergency departments (EDs) see fewer than 5000 pediatric patients per year, and over half (51.5%) of all pediatric visits occur in EDs with fewer than 10,000 pediatric patients per year [1,2]. Resuscitation of severely ill and injured pediatric patients is time sensi- tive and often requires high-skill critical procedures including

* Corresponding author at: US Acute Care Solutions, United States of America.

?? Corresponding author at: Department of Emergency Medicine, Allegheny Health Network, 320 East North Avenue, Pittsburgh, PA 15212, United States of America.

E-mail address: [email protected] (A. Venkat).

Endotracheal intubation , cardioversion, central venous placement, intraosseous (IO) access, or chest tube insertion [3-7]. While these pro- cedures can be lifesaving when performed correctly, incorrect perfor- mance can negatively impact patient outcomes [8]. Prior work with pediatric Endotracheal intubation has suggested that complication rates decrease as Provider experience increases [9,10]. The performance of these procedures is often required in emergency departments (ED), including general EDs that care for both adult and pediatric patients, pe- diatric EDs, and potentially freestanding EDs and urgent care centers, which are separate from a hospital.

Although it is generally assumed that residency training and clinical practice alone is sufficient to ensure and maintain procedural skills,

https://doi.org/10.1016/j.ajem.2020.06.009

0735-6757/(C) 2020

many critical procedures such as ETI in adult ED patients are infrequently performed by many practicing emergency physicians [11]. Understanding the frequency of these procedures is important in efforts to optimize qual- ity and competence while ensuring EDs are prepared for critically ill pedi- atric patients. A previous single-center study examined critical procedures at a pediatric academic center and found that 61% of attending physicians did not perform one during their 12 month study period [6]. The fre- quency of these procedures in community EDs is unknown.

We sought to evaluate the incidence of critical pediatric procedures performed by emergency physicians in a national emergency medicine group.

Methods

Study design, setting, and population

We conducted a retrospective observational study of pediatric (age b18 years) patient critical procedures performed by emergency physi- cians in EDs staffed by a national emergency medicine group from 1/ 1/2014 to 12/31/2018. The administrative billing and coding dataset used in this study has been previously described (blinded for review). We included all emergency physicians with at least 100 clinical hours worked and 1000 visits seen during the study period in the specific fa- cility type (general ED, pediatric ED, freestanding ED/urgent care cen- ter). For outcomes calculated on a per annual basis (e.g. procedures per year), we only included physicians if they worked at least 12 months in a facility type.

The Inclusion/exclusion criteria were applied to each physician working in each type of facility. Therefore, data is observed at the level of the physician-facility type. It is possible for a physician to meet inclu- sion criteria at multiple facility types over the study period (e.g. worked at both a general and pediatric ED). In this case, the physician will ap- pear in the dataset two times – once for each facility type – with the cor- responding number of months, hours worked, and visits seen at that facility type. If a physician worked at multiple facilities of the same type during the study period (i.e. saw 1000 visits in a general ED and then started working at another general ED and saw 500 visits), data from both facilities are aggregated together for that physician (i.e., total of 1500 visits in a general ED). It is also possible for a physician to meet criteria for one facility type, but not another (e.g., saw 2000 visits in a general ED but only 500 visits in a pediatric ED), in which case only the physician and visits are only counted in the first facility type. These inclusion criteria allow us to include emergency physicians working simultaneously at different facility types as well as emergency physicians that transitioned from one facility type to another over the course of the study period.

This group employs billers and coders with professional certifica- tions and participates in ongoing quality assurance programs through chart reviews. These billers and coders abstract procedures from physi- cian electronic health records of ED visits. Per this group’s policy, coding of procedures requires clear documentation of successful completion by a clinician. This study received IRB approval from Allegheny Health Net- work (Pittsburgh, PA). A study flow diagram is available in the Appendix (Fig. 1).

Study protocol and measurements

Given prior criticism of epidemiologic descriptions of procedure per- formance [12], we directly confirmed our data on pediatric critical pro- cedure performance in this dataset by returning to the original medical records for a separate independent abstraction. First, we randomly se- lected 100 pediatric emergency department visit charts with no re- corded critical procedures to verify that our dataset accurately reflected the absence of procedure performance, consistent with other analyses of critical procedures from administrative databases [11]. We did not find any critical procedure performance in these charts. Second,

given that critical procedures are very rare in the overall pediatric pop- ulation, we identified all (N = 828) pediatric ED charts with an ICD-10 code for cardiac arrest, respiratory arrest, or cardiopulmonary arrest, but without a recording of a pediatric critical procedure to verify ab- sence of a critical procedure in these High-risk conditions.

We found 6 charts (0.7%) that should have been coded with a perfor- mance of a pediatric critical procedure by an emergency physician based on the chart documentation. Finally, we examined all medical charts with a coded pediatric critical procedure (N = 2423). We found 40 charts (1.7%) that were incorrectly coded as having a successful pediatric critical procedure performance and removed these as the procedure was not ac- tually performed. All chart reviews were performed prior to applying the inclusion/exclusion criteria to individual physicians.

Definitions and outcomes

We defined pediatric critical procedures based on author consensus of time-sensitivity and high-skill in their performance and included intraosseous (IO) insertions, electrical cardioversion, endotracheal intu- bation (ETI), chest tube insertion, and central vein placement on any pa- tient age b18 at the time of the ED visit. While other critical pediatric procedures have been studied in the literature (e.g. Needle thoracostomy, pericardiocentesis, etc.) they have been found to be ex- ceedingly uncommon, even at busy, academic pediatric EDs where no cases were reported in a large retrospective review [6]. As such, these additional procedures were not included. We also report critical care time (Current Procedural Terminology [CPT] Codes 99291 or 99292) separately as another point of reference. These codes are used when a physician provided certain high acuity services to a critically ill or in- jured patient for N30 min in an ED setting. Certain critical services (e.g. pulse oximetry, transcutaneous pacing, Ventilator management) are “bundled” into the ED critical care time codes [13]. Our definition of crit- ical procedures therefore does not include any procedures/services that are part of this bundle.

Data analysis

We report the performance of pediatric critical procedures by emer- gency physicians working in general EDs (attached to a hospital with a mean patient age >=18 years), pediatric EDs (attached to a hospital with a mean patient age b18 years), and freestanding EDs/urgent care centers (not attached to a hospital). All outcomes are reported at the physician- facility type level. For each physician we calculate the rate of pediatric critical procedures performed on an annual basis, per 1000 patient visits seen (All Ages), and per 1000 clinical hours worked. In addition, recog- nizing that certain emergency physicians work more clinically than others, we report the same outcomes for those emergency physicians who worked on average >=100 clinical hours per month for a minimum of 12 months. We aggregate the data by facility type to calculate overall incident rates of critical procedures performed by physicians in the study sample and create point estimates and 95% confidence intervals for incident-rate differences and incident-rate ratios between general emergency physicians and pediatric emergency physicians. All analyses were conducted using Stata v. 16 (College Station, Texas).

Results

Characteristics of study subjects

During the study period, 2290 emergency physicians (1844 working in 129 general EDs, 125 working in 8 pediatric EDs, and 321 working in 49 freestanding EDs/urgent care centers) performed 2233 pediatric crit- ical procedures (Table 1). The demographics of physicians differed by facility type. Physicians working at general EDs were more likely to be under 40 (43.7%), physicians at pediatric EDs more likely to be between 40 and 49 (44%), and physicians at freestanding/urgent cares more

Total Emergency Physicians

n=3,697

• General ED
• Pediatric ED
• Freestanding ED/ Urgent Care Center

n= 2,458

n= 211

n= 1,028

EXCLUDED

n=1,407 (total)

• <1,000 visits seen n=1,404
• <100 hours worked n=434

Emergency Physicians (min 12 months worked)

n = 1,718

• General ED n= 1,357
• Pediatric ED n= 89
• Freestanding ED/ n= 272

Urgent care centers

Fig. 1. Study flow diagram.

likely to be over 60 (14%). Female physicians were more commonly working in pediatric EDs (44%) compared to general EDs (32.5%) and freestanding/urgent care (28.7%). The majority of physicians worked in facilities with an annual census of b40,000 visits that were predomi- nantly located in the South and West census regions of the United States. Additional details on the facilities are available in the appendix (Appendix Table S1).

Emergency Physicians who averaged 100 hours worked/month, min 12 months)

n = 1,035

• General ED n= 912
• Pediatric ED n= 55
• Freestanding ED/ n= 68

Urgent care centers

Included Emergency Physicians

n = 2,290

• General ED n= 1,844
• Pediatric ED n= 125
• Freestanding ED/ n= 321

Urgent Care Center

Main results

At the median, emergency physicians working at general EDs saw 2618 visits (interquartile range [IQR] = 1873-3318) and worked 1331 clinical hours (IQR = 999-1530) per year. Physicians at pediatric EDs had a similar workload as those at general EDs. Freestanding ED/Urgent care physicians worked fewer hours and saw fewer patients (1113 visits and 736 h). Physicians working in a pediatric ED performed 0.5 pediatric critical procedures per 1000 visits (IQR = 0-1.2) compared to 0 (IQR = 0-0.2) for emergency physicians working in general EDs and 0 (IQR = 0-0) for those working in a freestanding ED/urgent care. On an hourly basis, emergency physicians in pediatric EDs performed an average of

1.6 pediatric critical procedures per 1000 h worked (SD = 1.8) compared to 0.3 (SD = 0.5) among physicians working in a general ED (Table 1).

Among 1357 physicians working in general EDs for a minimum of 12 months, 53.9% performed zero pediatric critical procedures per year with 1.1% performing two or more such procedures per year. Among 89 physicians working in pediatric EDs, 31.5% performed zero pediatric critical procedures per year, with 30.3% performing two or more. Among 272 physicians working in freestanding EDs/urgent cares, 89% performed zero pediatric critical procedures per year. We identified

no freestanding ED/urgent care physicians performing two or more crit- ical procedures per year. The results were similar on a per 1000 clinical hours worked and per 1000 patient visits seen basis (Fig. 2, A-C) or when confining the sample to emergency physicians working on aver- age >=100 clinical hours/month (Fig. 2, D-F).

Table 2 provides details of individual pediatric critical procedure performance performed by general and pediatric ED physicians. ETIs were the most common critical procedures (mean = 0.3/year for physi- cians working in general EDs and 1.82/year in those working in pediat- ric EDs) followed by IO insertions (mean = 0.03/year for those working in general EDs and 0.17/year for those working in pediatric EDs). All other pediatric critical procedures were exceedingly rare (mean b1/ year in both general and pediatric EDs).

Finally, we compared the overall incidence of PEdiatric critical pro- cedures performed by emergency physicians working at general and pediatric ED. During the study period, these emergency physicians saw 12.6 million visits, worked 5.8 million clinical hours, and performed 2094 pediatric critical procedures. The overall incidence of pediatric critical procedures performed by emergency physicians at general EDs was 0.33 per year worked, 0.12 per 1000 visits seen, and 0.26 per 1000 clinical hours worked. By contrast, the overall incidence of pediat- ric critical procedures performed by emergency physicians working at pediatric EDs was 1.95 per year worked (rate difference = 1.6, 95% CI 1.5-1.8), 0.68 per 1000 visits seen (rate difference = 0.56, 95% CI

0.51-0.61), and 1.66 per 1000 clinical hours (rate difference = 1.39, 95% CI 1.27-1.52). Expressed as ratios, the incidence of pediatric critical procedures was 5.5 to 6.3 times higher among emergency physicians at pediatric EDs compared to emergency physicians at general EDs (Ap- pendix Table S2).

Table 1

Characteristics of emergency physicians and performance of pediatric critical procedures.

procedural and resuscitation skills” fellows are expected to acquire, but no further guidance about the number or type of training experi-

Emergency physicians General ED physicians

Pediatric ED physicians

Freestanding ED/urgent care physicians

ences is provided [15]. Importantly, no national standards exist for en- suring ongoing pediatric procedural competency among practicing emergency physicians after completing their training. Even in pediatric EDs, physicians feel there clinical experience is inadequate to maintain competency with specific critical procedures [16]. Hospital credential- ing committees seldom require ongoing pediatric procedural compe- tency retraining of medical staff beyond one-time certification courses like pediatric advanced life support (PALS).

Our analysis demonstrated a median rate of pediatric critical proce- dures by general emergency physicians working at least 100 h/month of

0.15 per 1000 patients. This means that the average full-time emergency physician (120 clinical hours per month seeing 2 patients per hour) prac- ticing in a general ED in this national group can expect to perform one pe- diatric critical procedure every 6667 visits, or about once every 2.3 years with the majority of these being ETI. The results in pediatric EDs, by com- parison, paint a different picture. Based on our estimates, the median rate of pediatric critical procedures by the average full-time pediatric emer- gency physician was 1.34 per 1000 visits. This equals approximately one critical pediatric procedure performed every 0.3 years with the majority being ETI. In facilities with the lowest procedure rates (freestanding EDs/urgent cares) the expectation is that the average provider may not perform any of these procedures throughout his or her career. These num- bers align with findings by Mittiga et al. where for pediatric providers working in a single busy pediatric ED over a one year period, pediatric air- way management was the most common critical procedure either per- sonally performed (37%) or supervised (88%), and IO insertion and central line placement were the next two most frequent, performed only by 20% and 5% of faculty, respectively, per year [6].

While these rates are higher at pediatric facilities, 51.5% of pediatric visits to EDs occur in facilities that see fewer than 10,000 pediatric pa- tients per year with the mortality rates notably higher at sites with low pediatric volumes [17]. Given these disparities between general and pediatric ED facilities, it is essential to ensure that general EDs are ready and able to stabilize and manage critically ill and injured children.

^a For pediatric critical procedures performed/year, only physicians with at least 12 months worked in the facility type are included (N = 1357 general ED physicians, 89 pe- diatric ED physicians, and 272 freestanding ED/urgent care physicians).

N	844	125	321
Physician age in 2018, %
b40	43.7	33.6	36.1
40-49	30.0	44.0	31.8
50-59	16.4	15.2	18.1
>=60	9.9	7.2	14.0
Physician sex, %
Men	67.5	56.0	71.3
Women	32.5	44.0	28.7
Mean patient age (SD)	46.6 (6.6)	9.9 (9.4)	37.3 (2.7)
Patient age mix, %
Under 1 year	1.3	14.6	1.3
1-4 years	3.0	29.5	5.0
5-12 years	3.5	27.3	7.5
13-17 years	3.2	13.9	5.4
>=18 years	89.0	14.7	80.8
Physician clinical hours	1331	1323	736
worked/year, median (IQR)	(999-1530)	(870-1566)	(454-1166)
Patient visits seen by emergency
physicians
Total visits seen (all ages)	11,623,588	971,857	872,339
Total pediatric visits seen	1,251,523	900,483	168,745
Visits seen/year, all ages,	2618	2616	1113
median (IQR)	(1873-3318)	(1569-3611)	(769-1600)
Pediatric visits seen/year,	235	2590	205 (139-325)
median (IQR)	(127-376)	(1430-3508)
Pediatric critical procedures
performed by emergency
physicians
Pediatric critical procedures performed/year, median (IQR)a	0 (0-0.5)	0.8 (0-2.4)	0 (0-0)
Pediatric critical procedures	0 (0-0.2)	0.5 (0-1.2)	0 (0-0)
performed per 1000 visits seen
(all ages), median (IQR)
Pediatric critical procedures	0 (0-0.4)	1.1 (0-2.4)	0 (0-0)
performed per 1000 clinical
hours worked, median (IQR)

Discussion

In our study examining data from a national emergency physician group, pediatric critical procedures were rarely performed by emer- gency physicians across facility types, including general EDs, pediatric EDs, and freestanding EDs/urgent cares. While ETI was the most com- mon procedure, it was still rarely performed at the physician level in both general and pediatric EDs. Other procedures (e.g. cardioversion) were performed even more infrequently with many general ED physi- cians never performing these during the study period. In a recent retro- spective study at a large pediatric emergency department (90,000 child visits/year), 61% of pediatric emergency medicine faculty members did not perform a single critical procedure in one year [6]. To our knowledge our study is the first to look at pediatric critical procedure frequency among general, pediatric, and freestanding/urgent care emergency phy- sicians in different clinical settings at a national level where perfor- mance data has undergone verification by direct chart review.

The Accreditation Council for Graduate Medical Education (ACGME), in its residency program guidance, details minimum graduation re- quirements for Emergency Medicine residency programs, including cer- tain “key index procedures” deemed of the utmost importance, both through research and expert opinion [14]. Benchmarks such as 35 intu- bations and 10 Chest tubes have been declared the minimum threshold for graduation and Board eligibility [14]. These numbers are overall and not specific to pediatric patients. ACGME requirements for pediatric emergency medicine fellowships include a list of 27 “necessary

The 2018 joint policy statement Pediatric Readiness in Emergency Depart- ments outlines seven critical domains to ensure day-to-day readiness with one of these being “provider competencies” including procedural competencies [18-20]. Despite ongoing efforts to address these domains [21], common gaps still exist [1]. In addition, these gaps are associated with a four-fold difference in mortality among critically ill children pre- senting to EDs in the lowest pediatric readiness quartile [17].

How to bridge these gaps and ensure emergency physician compe- tency and proficiency with these critical procedures given their infrequent performance is multifactorial. First, physicians must have sufficient tools to cultivate ongoing practice to ensure skill with these procedures includ- ing 1) deliberate practice and 2) simulation. Deliberate practice may help maintain procedural skills, especially in rare but critical situations [22]. The use of simulation and other psychomotor education may also facili- tate maintenance of pediatric critical skill proficiency [23-25].

The second key aspect to ensuring ongoing proficiency is to have an understanding of the minimum threshold standards for maintaining com- petency with critical procedures. Currently, this threshold and the forget- ting curves [26] for pediatric critical procedures are unknown. The performance of other resuscitation skills such as cardiopulmonary resus- citation (CPR) can rapidly decline (in as little as 3 months from training) below acceptable performance thresholds [27]. No benchmarks exist for practicing emergency physicians with regard to pediatric critical proce- dure frequency and competence. Our data suggest that, at most sites, clin- ical exposure alone may be insufficient for clinicians to maintain skills in these areas. A recent national survey of pediatric ED providers’ self- perceived critical procedural skills found that over half felt that they needed additional training beyond direct patient care to maintain these skills and that adequate ongoing training efforts are not in place, even among busy pediatric EDs [28]. How we improve this confidence and

Fig. 2. Total pediatric critical procedures performed per year, per 1000 clinical hours worked, and per 1000 patient ED visits seen by frequency category for included general ED, pediatric ED, and freestanding ED/urgent care emergency physicians (A to C) and emergency physicians that averaged >=100 clinical hours per month (D to F). Note: Panels B and C include physicians with b12 months worked, but all had at least 100 clinical hours and 1000 visits seen.

determine the minimum experience threshold and appropriate training interval to maintain competency will require future study.

The ways in which physician competencies may be maintained or evaluated have not been delineated. Suggested methods include: direct observation, chart reviews, knowledge tests, simulation or mock codes, participation in continuing education conferences, team training exercises, cross training in other clinical settings, and maintenance of board certification [18-20]. However, none of these modalities specifically highlight maintenance of psychomotor skills. In a recent survey of pediatric ED clinicians, most felt at least annual practice was required for pediatric critical skills mainte- nance and that alternative Clinical sites (e.g., Operating rooms) are preferred for maintenance of pediatric airway management proce- dures [29]. Yet, based on a 2013 national assessment, annual

pediatric competency evaluations, in any form, are not required of physicians (38.7%); nurses (66.6%); or advanced practice providers (18.1%) working in emergency departments [1]. In addition, even if pediatric critical procedures are performed effectively and at the right time, lack of regular ongoing procedural experience among all staff and, thus overall system-level readiness to care for children may impact outcomes [17].

Limitations

Our study has several limitations. First, our data come from an ad- ministrative billing and coding dataset. While we have performed mul- tiple data checks to ensure the validity of our data, including a review of

Table 2

Emergency physician pediatric critical procedures.

Emergency physicians (min 100 h worked and 1000 visits seen) General ED emergency physicians

(N = 1844)

Pediatric ED emergency physicians (N = 125)

	Mean (SD)	Median (IQR)		Mean (SD)	Median (IQR)
Critical procedures:
Total pediatric critical procedures/year	0.35 (0.76)	0.00 (0.00-0.44)		2.05 (2.60)	1.09 (0.00-3.27)
Pediatric intraosseous insertions/year	0.03 (0.19)	0.00 (0.00-0.00)		0.17 (0.41)	0.00 (0.00-0.00)
Pediatric electrical cardioversions/year	0.00 (0.04)	0.00 (0.00-0.00)		0.01 (0.06)	0.00 (0.00-0.00)
Pediatric endotracheal intubations/year	0.30 (0.64)	0.00 (0.00-0.40)		1.82 (2.36)	1.04 (0.00-2.63)
Pediatric chest tube insertions/year	0.02 (0.14)	0.00 (0.00-0.00)		0.06 (0.27)	0.00 (0.00-0.00)
Pediatric central line placements/year	0.03 (0.20)	0.00 (0.00-0.00)		0.06 (0.28)	0.00 (0.00-0.00)
Total pediatric critical procedures per 1000 ED visits	0.13 (0.29)	0.00 (0.00-0.18)		0.75 (0.90)	0.48 (0.00-1.18)
Pediatric intraosseous insertions per 1000 ED visits	0.01 (0.07)	0.00 (0.00-0.00)		0.06 (0.15)	0.00 (0.00-0.00)
Pediatric electrical cardioversions per 1000 ED visits	0.00 (0.01)	0.00 (0.00-0.00)		0.00 (0.02)	0.00 (0.00-0.00)
Pediatric endotracheal intubations per 1000 ED visits	0.11 (0.25)	0.00 (0.00-0.14)		0.66 (0.81)	0.40 (0.00-1.05)
Pediatric chest tube insertions per 1000 ED visits	0.01 (0.05)	0.00 (0.00-0.00)		0.02 (0.11)	0.00 (0.00-0.00)
Pediatric central line placements per 1000 ED visits	0.01 (0.07)	0.00 (0.00-0.00)		0.03 (0.13)	0.00 (0.00-0.00)
Total pediatric critical procedures per 1000 clinical hours	0.27 (0.54)	0.00 (0.00-0.39)		1.58 (1.77)	1.12 (0.00-2.40)
Pediatric intraosseous insertions per 1000 clinical hours	0.02 (0.14)	0.00 (0.00-0.00)		0.12 (0.29)	0.00 (0.00-0.00)
Pediatric electrical cardioversions per 1000 clinical hours	0.00 (0.03)	0.00 (0.00-0.00)		0.01 (0.04)	0.00 (0.00-0.00)
Pediatric endotracheal intubations per 1000 clinical hours	0.23 (0.47)	0.00 (0.00-0.31)		1.41 (1.64)	0.75 (0.00-2.25)
Pediatric chest tube insertions per 1000 clinical hours	0.02 (0.10)	0.00 (0.00-0.00)		0.04 (0.17)	0.00 (0.00-0.00)
Pediatric central line placements per 1000 clinical hours	0.02 (0.14)	0.00 (0.00-0.00)		0.05 (0.21)	0.00 (0.00-0.00)

Emergency physicians (min 100 h worked, 1000 visits seen, and average of 100+ clinical hours/month, min 12 months)

General ED emergency physicians

(N = 912)

Pediatric ED emergency physicians

(N = 55)

	Mean (SD)	Median (IQR)		Mean (SD)	Median (IQR)
Critical procedures: Total pediatric critical procedures/year	0.47	0.22		2.92	2.12
	(0.70)	(0.00-0.69)		(2.95)	(0.60-4.40)
Pediatric intraosseous insertions/year	0.04	0.00		0.28	0.00
	(0.14)	(0.00-0.00)		(0.45)	(0.00-0.41)
Pediatric electrical cardioversions/year	0.00	0.00		0.04	0.00
	(0.04)	(0.00-0.00)		(0.11)	(0.00-0.00)
Pediatric endotracheal intubations/year	0.40	0.00		2.55	1.50
	(0.65)	(0.00-0.58)		(2.81)	(0.48-3.80)
Pediatric chest tube insertions/year	0.03	0.00		0.05	0.00
	(0.14)	(0.00-0.00)		(0.17)	(0.00-0.00)
Pediatric central line placements/year	0.03	0.00		0.08	0.00
	(0.11)	(0.00-0.00)		(0.29)	(0.00-0.00)
Total pediatric critical procedures per 1000 ED visits	0.13	0.06		0.74	0.53
	(0.20)	(0.00-0.20)		(0.73)	(0.11-1.17)
Pediatric intraosseous insertions per 1000 ED visits	0.01	0.00		0.08	0.00
	(0.04)	(0.00-0.00)		(0.14)	(0.00-0.11)
Pediatric electrical cardioversions per 1000 ED visits	0.00	0.00		0.01	0.00
	(0.01)	(0.00-0.00)		(0.03)	(0.00-0.00)
Pediatric endotracheal intubations per 1000 ED visits	0.11	0.00		0.63	0.40
	(0.18)	(0.00-0.17)		(0.69)	(0.10-1.01)
Pediatric chest tube insertions per 1000 ED visits	0.01	0.00		0.01	0.00
	(0.04)	(0.00-0.00)		(0.03)	(0.00-0.00)
Pediatric central line placements per 1000 ED visits	0.01	0.00		0.03	0.00
	(0.04)	(0.00-0.00)		(0.13)	(0.00-0.00)
Total pediatric critical procedures per 1000 clinical hours	0.32	0.15		1.97	1.34
	(0.48)	(0.00-0.45)		(2.03)	(0.36-3.01)
Pediatric intraosseous insertions per 1000 clinical hours	0.02	0.00		0.19	0.00
	(0.10)	(0.00-0.00)		(0.29)	(0.00-0.22)
Pediatric electrical cardioversions per 1000 clinical hours	0.00	0.00		0.03	0.00
	(0.03)	(0.00-0.00)		(0.07)	(0.00-0.00)
Pediatric endotracheal intubations per 1000 clinical hours	0.27	0.00		1.73	1.07
	(0.45)	(0.00-0.39)		(1.96)	(0.27-2.41)
Pediatric chest tube insertions per 1000 clinical hours	0.02	0.00		0.03	0.00
	(0.09)	(0.00-0.00)		(0.11)	(0.00-0.00)
Pediatric central line placements per 1000 clinical hours	0.02	0.00		0.05	0.00
	(0.07)	(0.00-0.00)		(0.21)	(0.00-0.00)

selected medical charts, there may be individual cases where critical procedures were missed.

Second, most of the physicians do not appear in the dataset for the entire study period. We set minimum eligibility criteria to be included in the study sample (e.g. 1000 visits seen and 100 clinical hours worked) and still found a large number of physicians that did not per- form any pediatric critical procedures. For each outcome, we present

results for all physicians as well as for the subset of physicians that worked close to a full-time clinical schedule (i.e. averaged 100 h per month for a minimum of 12 months). We still found a large number of physicians that did not perform any pediatric critical procedures in this group. However, we cannot rule out that setting even more strin- gent criteria (e.g. a minimum of two years worked) would yield differ- ent results.

Third, based on our data, we are unable to conclude whether psycho- motor competencies impact mortality either directly or indirectly. We are also unable to determine the appropriateness of the performed pro- cedures. Along with this, some procedures carry significant risk if mul- tiple attempts are needed (i.e., ETI). From our data, we are unable to determine if multiple attempts were needed to complete a procedure or if any clinically significant patient related adverse events occurred as a result of performing one of these critical procedures. In addition, we are unable to determine if other providers initially attempted a pro- cedure and were unsuccessful as the data were only reported for pro- viders who billed for a completed procedure. Finally, we were unable to examine procedures performed in the ED that were not attributable to the group (i.e. anesthesiology or cardiology, intensive care team, etc.).

Conclusions

In this study using data from a national ED staffing group, emergency physicians infrequently performed pediatric critical procedures, and the rates are lower at general EDs and freestanding EDs/urgent cares relative to pediatric EDs. The rarity of performance of these skills has implications for pediatric readiness in EDs and ensuring emergency physicians can ef- fectively perform pediatric critical procedures when required.

Funding

Dr. Remick is partially supported by the Health Resources and Ser- vices Administration (HRSA) of the U.S. Department of Health and Human Services (HHS) under grant number U07MC29829: Emergency Medical Services for Children (EMSC) Innovation and Improvement Center and Targeted Issues grant HRSA 19-051, Targeted Issues Grant: Developing a National Pediatric Readiness Project QI Data Registry.

The funder had no role in the design and conduct of the study; col- lection, management, analysis and interpretation of the data; prepara- tion, review, or approval of the manuscript; and decision to submit the manuscript for publication.

CRediT authorship contribution statement Jestin N. Carlson: Conceptualization, Supervision, Writing – original

draft, Writing – review & editing. Mark Zocchi: Formal analysis, Writing

• original draft, Writing – review& editing. Coburn Allen: Conceptuali- zation, Writing – original draft, Writing – review & editing. T. Kent Denmark: Conceptualization, Writing – original draft, Writing – review & editing. Jay D. Fisher: Conceptualization, Writing – original draft, Writing – review & editing. Matthew Wilkinson: Writing – original draft, Writing – review & editing. Katherine Remick: Writing – original draft, Writing – review & editing. Abbie Sullivan: Supervision, Writing – review & editing. Jesse M. Pines: Writing – original draft, Writing – re- view & editing. Arvind Venkat: Conceptualization, Supervision, Writing
• original draft, Writing – review & editing, Project administration.

Declaration of competing interest

JNC, MZ, CA, TKD, JDF, MW, KR, AS, JMP, and AV report no conflicts of interest.

Acknowledgements

The authors wish to acknowledge and thanks Paul Dietzen, Amer Aldeen, MD, and the leadership of US Acute Care Solutions for their sup- port of this research.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi. org/10.1016/j.ajem.2020.06.009.

References

1. Gausche-Hill M, Ely M, Schmuhl P, et al. A national assessment of pediatric readiness of emergency departments. JAMA Pediatr. 2015;169(6):527-34 Jun.
2. Whitfill T, Auerbach M, Scherzer DJ, Shi J, Xiang H, Stanley RM. Emergency care for children in the United States: epidemiology and trends over time. J Emerg Med. 2018;55(3):423-34 Sep.
3. Green SM, Ruben J. Emergency department children are not as sick as adults: impli- cations for critical care skills retention in an exclusively pediatric emergency medi- cine practice. J Emerg Med. 2009;37(4):359-68 Nov.
4. Green SM. A is for airway: a pediatric emergency department challenge. Ann Emerg Med. 2012;60(3):261-3 Sep.
5. Green SM. How does one learn Critical care procedures in a pediatric emergency de- partment? Ann Emerg Med. 2013;61(3):279-80 Mar.
6. Mittiga MR, Geis GL, Kerrey BT, Rinderknecht AS. The spectrum and frequency of critical procedures performed in a pediatric emergency department: implications of a provider-level view. Ann Emerg Med. 2013;61(3):263-70 Mar.
7. Carlson JN, Gannon E, Mann NC, et al. Pediatric out-of-hospital critical procedures in the United States. Pediatr Crit Care Med. 2015;16(8):e260-7 Oct.
8. Kerrey BT, Rinderknecht A, Mittiga M. High risk, low frequency: optimizing perfor- mance of emergency intubation for children. Ann Emerg Med. 2017;70(6):783-6 Dec.
9. Auroy Y, Ecoffey C, Messiah A, Rouvier B. Relationship between complications of pe- diatric anesthesia and volume of pediatric anesthetics. Anesth Analg. 1997;84(1): 234-5 Jan.
10. Keenan RL, Shapiro JH, Kane FR, Simpson PM. Bradycardia during anesthesia in in- fants. An epidemiologic study. Anesthesiology. 1994;80(5):976-82 May.
11. Carlson JN, Zocchi M, Marsh K, et al. Procedural experience with intubation: results from a national emergency medicine group. Ann Emerg Med. 2019;74(6):786-94 Dec.
12. Green SM. Congruence of disposition after emergency department intubation in the National Hospital Ambulatory Medical Care Survey. Ann Emerg Med. 2013;61(4): 423-426 e428 Apr.
13. ACEP. Critical care FAQ. https://http://www.acep.org/administration/ reimbursement/reimbursement-faqs/critical-care-faq/-question0. [Accessed 22

    February 2020].

    ACGME. Emergency medicine defined key index procedure minimums. https:// http://www.acgme.org/Portals/0/PFAssets/ProgramResources/EM_Key_Index_ Procedure_Minimums_103117.pdf?ver=2017-11-10-130003-693. [Accessed 17

    February 2020].

    ACGME. ACGME program requirements for graduate medical education in pediatric critical care medicine (subspecialty of pediatrics). https://http://www.acgme.org/ Portals/0/PFAssets/ProgramRequirements/323_PediatricCricitalCareMedicine_2019. pdf?ver=2019-06-13-125908-093. [Accessed 17 February 2020].

14. Losek JD, Olson LR, Dobson JV, Glaeser PW. Tracheal intubation practice and main- taining skill competency: survey of pediatric emergency department medical direc- tors. Pediatr Emerg Care. 2008;24(5):294-9 May.
15. Ames SG, Davis BS, Marin JR, et al. Emergency department pediatric readiness and mortality in critically ill children. Pediatrics. 2019;144(3) Sep.
16. Remick K, Gausche-Hill M, Joseph MM, et al. Pediatric readiness in the emergency department. Ann Emerg Med. 2018;72(6):e123-36 Dec.
17. Remick K, Gausche-Hill M, Joseph MM, et al. Pediatric readiness in the emergency department. Pediatrics. 2018;142(5) Nov.
18. Remick K, Gausche-Hill M, Joseph MM, et al. Pediatric readiness in the emergency department. J Emerg Nurs. 2019;45(1):e3-18 Jan.
19. EMS for children innovation and improvement center national pediatric readiness project. http://www.pediatricreadiness.org. [Accessed 29 January 2020].
20. Ericsson KA. Deliberate practice and the acquisition and maintenance of expert per- formance in medicine and related domains. Acad Med. 2004;79(10 Suppl):S70-81 Oct.
21. McLaughlin C, Barry W, Barin E, et al. Multidisciplinary simulation-based team train- ing for trauma resuscitation: a scoping review. J Surg Educ. 2019;76(6):1669-80 Nov – Dec.
22. Lin Y, Cheng A. The role of simulation in teaching pediatric resuscitation: current perspectives. Adv Med Educ Pract. 2015;6:239-48.
23. McLaughlin CM, Wieck MM, Barin EN, et al. Impact of simulation-based training on perceived provider confidence in acute multidisciplinary pediatric trauma resuscita- tion. Pediatr Surg Int. 2018;34(12):1353-62 Dec.
24. Ebbinghaus H. Memory: a contribution to experimental psychology. Ann Neurosci. 2013;20(4):155-6 Oct.
25. Anderson R, Sebaldt A, Lin Y, Cheng A. Optimal training frequency for acquisition and retention of high-quality CPR skills: a randomized trial. Resuscitation. 2019; 135:153-61 Feb.
26. Mittiga MR, FitzGerald MR, Kerrey BT. A survey assessment of perceived importance and methods of maintenance of critical procedural skills in pediatric emergency medicine. Pediatr Emerg Care. 2019;35(8):552-7 Aug.
27. Craig SS, Auerbach M, Cheek JA, et al. Preferred learning modalities and practice for critical skills: a global survey of paediatric emergency medicine clinicians. Emerg Med J. 2019;36(5):273-80 May.