Original Contribution

Quality of emergency care provided by physician assistants and nurse practitioners in acute asthma^?,??

Chu-Lin Tsai MD, ScD ^a,?, Ashley F. Sullivan MS, MPH ^a,

Adit A. Ginde MD, MPH ^b, Carlos A. Camargo Jr. MD, DrPH ^a

^aDepartment of Emergency Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA

^bDepartment of Emergency Medicine, University of Colorado Denver School of Medicine, Aurora, CO 80045, USA

Received 15 December 2008; revised 30 January 2009; accepted 30 January 2009

Abstract

Objective: The aim of this study was to evaluate the quality of care provided by physician assistants or nurse practitioners (ie, midlevel providers [MLPs]) in acute asthma, as compared with that provided by physicians.

Methods: We performed a secondary analysis of the asthma component of the National Emergency Department Safety Study. We identified emergency department (ED) visits for acute asthma in 63 urban EDs in 23 US states between 2003 and 2006. Quality of care was evaluated based on 12 guideline-recommended process-of-care measures, a composite guideline concordance score, and 2 outcome-of-care measures (admission and ED length of stay).

Results: Of the 4029 patients included in this analysis, 3622 (90%) were seen by physicians only, 319 (8%) by MLPs supervised by physicians, and 88 (2%) by MLPs not supervised by physicians. After adjustment for patient mix, unsupervised MLPs were less likely to administer inhaled ?-agonists within 15 minutes of ED arrival (odds ratio [OR], 0.2; 95% confidence interval [CI], 0.1-0.7), less likely to prescribe Systemic corticosteroids in the ED (OR, 0.4; 95% CI, 0.2-0.9), and were more likely to prescribe inappropriate antibiotics at discharge (OR, 2.1; 95% CI, 1.1-4.1), as compared with physicians. Overall, their composite guideline concordance score was lower than that of physicians (-6 points; 95% CI, -9 to

-3 points). Supervised MLPs provided similar quality of care to that of physicians.

Conclusions: The MLPs were involved in 10% of ED patients with acute asthma and provided independent care for 2% of these patients. Compared with care provided by physicians or by supervised MLPs, there are opportunities for improvement in unsupervised MLP care.

(C) 2010

^? Funding: This study was supported by grant 5 R01 HS013099 from the Agency for Healthcare Research and Quality (Rockville, MD). Dr Camargo also was funded by grant HL084401 (Bethesda, MD).

^?? Conflict of Interest Statement: Dr Camargo has received financial support from a variety of groups for participation in conferences, consulting, and

medical research. Recent industry sponsors with an interest in asthma were AstraZeneca, Dey, GSK, Merck, Novartis, and Schering-Plough. Other authors have no conflicts of interest to disclose.

* Corresponding author. EMNet Coordinating Center, Department of Emergency Medicine, Massachusetts General Hospital, Boston, MA 02114. Tel.: +1

617 726 5276; fax: +1 617 724 4050.

E-mail address: [email protected] (C.-L. Tsai).

0735-6757/$ – see front matter (C) 2010 doi:10.1016/j.ajem.2009.01.041

Introduction

physician assistants and nurse practitioners have been increasingly used in US emergency departments (EDs) as a potential strategy for shortage of emergency physicians and rising Health care costs. According to a national survey, the involvement of these midlevel providers (MLPs) in emergency care has increased from 4% of ED visits in 1995 to 13% in 2005 [1]. Because of their expanding role in emergency care, it is important to understand how they perform to better inform health policy decisions. Yet, data on quality of care provided by MLPs in the ED are limited. Few single-center studies have examined emergency care of minor injures and found that MLPs provided equal quality of care to that of physicians [2,3]. These findings, however, may not be generalizable to other hospitals or to other emergency conditions.

Acute asthma is a common presentation to the ED, accounting for approximately 2 million ED visits each year [4]. To our knowledge, no studies have assessed the performance of MLPs in the management of acute asthma. Clinical practice guidelines for asthma are available, which outline evidence-based recommendations for the ED [5]. Assessment of concordance with these guideline recom- mendations can be used to evaluate quality of emergency care for asthma.

Using data from a large cohort study, we sought to evaluate the quality of care provided by MLPs in acute asthma compared with that provided by physicians.

Methods

Study design and setting

This retrospective cohort study was part of the National Emergency Department Safety Study (NEDSS), a large, multicenter study that sought to characterize organizational- and clinician-related factors associated with the occurrence of errors in EDs. Details of the study design and data collection have been published previously [6]. The NEDSS was coordinated by the Emergency Medicine Network (EMNet, www.emnet-usa.org). The NEDSS recruited EDs by inviting sites affiliated with the EMNet and invited EDs not affiliated with EMNet through postings on emergency medicine listservs and presentations at emergency medicine meetings. The NEDSS excluded military, Veterans Admin- istration hospitals, children’s hospitals, and hospitals in US territories. A total of 63 US EDs in 23 US states completed the asthma component of NEDSS. The Institutional Review Board at all participating hospitals approved the study.

Study population

Using a standardized data abstraction tool, trained research personnel at 63 US EDs abstracted data from

randomly selected ED visits for acute asthma during 2003 to 2006. The visits were identified by using International Classification of Diseases, Ninth Revision, Clinical Mod- ification codes 493.xx. Inclusion criteria were age 14 to 54 years and a history of asthma before the index visit. The following visits were excluded: repeat visits; transfer visits; patient visits with a history of chronic obstructive pulmonary disease, emphysema, or chronic bronchitis; or visits not prompted, in large part, by asthma exacerbation.

Measurements

Data abstracted included baseline patient characteristics, past asthma history, ED presentation, ED provider type, ED management, and ED disposition. Peak expiratory flow (PEF) was recorded in liters per minute and expressed as the absolute value; no percent predicted values are presented because of infrequent recording of the patient’s height in ED charts. Instead, severity of acute asthma was classified according to the initial PEF as follows: mild, 300+ for women and 400+ for men; moderate, 200 to 299 for women and 250 to 399 for men; severe, 120 to 199 for women and 150 to 249 for men; and very severe, less than 120 for women and less than 150 for men. These absolute PEF cutoffs represent approximately 70%, 40%, and 25% predicted, respectively, for a typical adult woman and man [7].

For the purpose of this analysis, we combined patients seen by PAs or NPs because of small numbers of patients in each category of MLP. Physicians were defined as attending or resident physicians. Patients were classified into 3 groups: patients seen by physicians alone were classified as “physician” group; patients seen by both physicians and MLPs, “supervised MLP” group; and patients seen by MLPs without evidence of physician involvement, “unsupervised MLP” group. We excluded patients whose provider type was unable to be determined or not documented.

We also distributed a key informant survey at each site to collect data on ED characteristics, such as number of beds in the ED and annual asthma visit volume. Geographic regions (northeast, south, midwest, and west) were defined according to Census Bureau boundaries [8]. Rural and urban distinc- tions were made according to the Office of Management and Budget’s designation of metropolitan statistical area [9]. Similar to the classification used at the patient level, sites at which all included patients were seen by physicians alone were classified as “physician sites”; sites at which patients were seen by either physicians or supervised MLPs, “supervised MLP sites”; sites with any involvement of unsupervised MLPs care, “unsupervised MLP sites.”

Concordance with guidelines: quality of care measures

• • 1. Process-of-care measures

On the basis of common recommendations contained in the 1997 and 2007 National Institutes of Health asthma

guidelines [5,10] and in the consensus view of the EMNet Steering Committee, we developed a priori a set of process- of-care measures, which included level A and level B evidence-basED treatments according to the National Institutes of Health guidelines. Level A and level B treatments both were supported by evidence from rando- mized controlled trials. The specifications of these mea- sures, including numerators and denominators, have been published previously [11] (available as online supplemental material). Specifically, to avoid unfairly penalizing MLPs for providing less care because they cared for patients with mild exacerbations, we restricted the denominators to patients eligible for the quality measures. We then summarized the 12 process measures by a patient composite guideline concordance score, which was calculated as the number of times a patient received the guideline-concordant care across quality measures divided by the patient’s total number of eligible opportunities [12]. These percentages were rescaled from 0 to 100, with a score of 100 indicating perfect concordance.

• • 1. Outcome-of-care measures

The ED outcomes included length-of-stay and disposi- tion. Hospital admission was defined as admission to an Inpatient unit, observation unit, or intensive care unit.

Statistical analysis

Summary statistics at both the patient and ED level are presented as proportions (with 95% confidence intervals [CIs]) or medians (with interquartile ranges [IQRs]). For categorical variables, the associations between provider type and other variables were examined using ?2 test or Fisher exact test, as appropriate. For continuous variables, the associations between provider type and other variables were examined using Kruskal-Wallis tests. To adjust for baseline patient and ED characteristics that may have confounded the relationships between provider type and quality measures,

Table 1 Baseline ED characteristics, according to provider type

we performed multivariable regression modeling. Model variables were selected a priori or were selected from variables associated with provider type or outcome at P b .10 in univariable analyses [13]. These variables included age, sex, race/ethnicity, ever admitted for asthma, transported to ED by ambulance, duration of symptoms less than 24 hours, recent upper respiratory infection, initial respiratory rate, initial oxygen saturation, initial PEF, and 1 ED characteristic (region). Variables with missing data (race/ethnicity and PEF) were dummy coded using the missing indicator method [14]. To account for the effects of clustering of patients within EDs, the multivariable analyses were performed by fitting 2-level mixed-effects models, with fixed effects for covariates adjusted and random intercepts for EDs [15]. Because the data of patient composite guideline Concordance scores and ED length of stay were skewed, the associated multivariable Linear regression models were bootstrapped 1000 times to obtain the bias-corrected CIs [16]. All odds ratios (ORs) and ? coefficients are presented with 95% CIs. All analyses were performed using Stata v10.0 software (StataCorp, College Station, TX). All P values are 2-sided, with P b .05 considered statistically significant.

Results

Emergency department characteristics

Of the 63 EDs participating in the NEDSS study, 76% were affiliated with an emergency medicine residency program (ie, were academic EDs). Participating EDs were all urban but located in different geographic regions of the country. Of the 63 EDs, 27 (43%) were physician sites, 21

(33%) were supervised MLPs sites, and 15 (24%) were unsupervised MLP sites (Table 1). Unsupervised MLP care tended to occur at sites that had a higher volume of ED visits (overall or asthma related); however, this trend was not

ED characteristic	Physician sites (n = 27)	MLP sites, supervised (n = 21)	MLP sites, unsupervised (n = 15)	P
No. of ED visits per year, median (IQR)	50 000 (43 000-70 000)	53 676 (40 000-72 533)	65 000 (55 000-81 244)	.19
No. of ED visits for asthma per year, median (IQR)	990 (511-1610)	804 (453-1500)	1632 (782-2000)	.37
No. of ED beds, median (IQR)	39 (24-51)	32 (26-40)	42 (31-50)	.20
No. of ED physicians, median (IQR), FTE	19 (13-26)	18 (13-24)	20 (16-25)	.60
No. of ED nurses, median (IQR), FTE	55 (43-70)	51 (34-65)	53 (45-64)	.87
Residency affiliated (%)	70	71	93	.23
Census region (%)				.049
Northeast	37	43	60
Midwest	41	19	0
South	11	19	7
West	11	19	33
Urban location (%)	100	100	100	–
FTE indicates full-time equivalent.

Table 3 Processes and outcomes of care among ED patients with acute asthma, according to provider type

Variable Physician Supervised MLP Unsupervised MLP

P

No. of patients eligible (% care given)

Table 2 Baseline patient characteristics, according to provider type

Patient characteristic	Physician (n = 3622)	Supervised MLP (n = 319)	Unsupervised MLP (n = 88)	P
demographic factors Age, median (IQR), y Female, % Race/Ethnicity, % ? White Black Hispanic Other Chronic asthma factors, % Ever admitted for asthma ED visit for asthma in past year Ever intubated or ventilated for asthma ED presentation Recent upper respiratory infection, % Duration of symptoms b24 h, % Transported to ED by ambulance, % Initial respiratory rate, median (IQR), breath/min Initial oxygen saturation, median (IQR) Initial PEF, median (IQR), L/min + Severity based on initial PEF, % + Mild Moderate Severe Very severe	34 (24-43)	35 (24-43)	33 (23-41)	.18
	64	70	59	.052
				b.001
	33	34	18
	47	38	47
	17	25	28
	3	2	7
	26	26	13	.02
	24	18	13	.003
	9	9	0	.002
	31	37	49	b.001
	36	33	22	.02
	20	20	10	.053
	20 (19-24)	20 (18-24)	20 (18-20)	b.001
	97 (95-99)	98 (96-99)	98 (96-99)	.04
	240 (160-300)	250 (180-330)	275 (200-325)	.01
				.02
	22	31	30
	39	39	50
	29	21	15
	10	10	5
* Available among a subset of patients (n = 2610). + Available among a subset of patients (n = 1863).

Processes of care (Level of evidence)
Prescribing inhaled ?-agonists in ED (A)	3622 (91)	319 (95)	88 (84)	.006
Prescribing inhaled anticholinergics in ED (A)	1147 (78)	105 (73)	9 (78)	.56
Not prescribing methylxanthines in ED (A)	3621 (99)	319 (99)	88 (100)	.06
Prescribing Systemic corticosteroids in ED (A)	1891 (78)	181 (78)	31 (48)	b.001
Prescribing oral corticosteroids at discharge (A)	1204 (66)	115 (72)	29 (45)	.02
Not prescribing antibiotics in ED (B)	3438 (92)	303 (90)	83 (93)	.40
Not prescribing oral antibiotics at discharge (B)	2804 (83)	224 (80)	83 (76)	.09
Assessment of PEF (B)	3247 (51)	299 (67)	87 (49)	b.001
Timeliness measures (level of evidence)
Initial PEF <=30 min of arrival (B)	1409 (48)	176 (43)	36 (28)	.02
Posttreatment PEF within 30-90 min (B)	3440 (21)	300 (30)	85 (25)	.004
Inhaled ?-agonists <=15 min of arrival	3092 (29)	285 (28)	70 (4)	b.001
Systemic corticosteroids <=75 min of arrival	1394 (62)	131 (62)	15 (33)	.07
	Physician	Supervised MLP	Unsupervised MLP	P
	(n = 3622)	(n = 319)	(n = 88)
Composite score
Composite guideline concordance score, median (IQR)	64 (57-75)	67 (57-82)	57 (50-67)	b.001
Outcomes of care
ED length of stay, median (IQR), min	175 (116-276)	165 (112-267)	151 (111-205)	.049
Hospital admission, % ?	19	21	2	b.001
* Included patients admitted to ward, observation unit, or intensive care unit.

statistically significant. Unsupervised MLP care was more likely to occur in northeastern and western EDs.

Patient characteristics

There were 4053 patients with acute asthma in the NEDSS database. Of these, 24 patients were excluded because their provider types were unable to be determined or not documented. Of the 4029 patients included in this analysis, 3622 (90%) were seen by physicians only, 319 (8%) by supervised MLPs, and 88 (2%) by unsupervised MLPs. Of the 3622 patients seen by physicians only, 3529 (97%) were covered by attending physicians. Of the 88 patients seen by unsupervised MLPs, 61 were seen by PAs and 27 by NPs.

Patients seen by supervised MLPs were similar to those seen by physicians alone. Patients seen by unsupervised MLPs, however, were more likely to be nonwhite and had less severe chronic asthma, as suggested by fewer hospita- lizations, fewer ED visits, and no intubations in the past (Table 2). Patients seen by unsupervised MLPs were more likely to have an upper respiratory infection, were less likely to present to the ED less than 24 hours of symptom onset, and had less severe exacerbations, as indicated by lower respiratory rates and higher PEFs at ED presentation.

Quality of emergency care

• • 1. Processes of care

Performance rates for supervised MLPs were generally similar to physicians’ rates; however, performance rates for unsupervised MLPs were generally lower than physicians’ rates (Table 3). For example, unsupervised MLPs were less likely to prescribe inhaled ?-agonists and systemic corticos- teroids in the ED and were less likely to prescribe systemic

Table 4 Multivariable associations between provider type and quality of care among ED patients with acute asthma Quality of care ? OR or ? coefficient (95% CI) ⁺

Physician Supervised MLP Unsupervised MLP

corticosteroids at discharge, compared with physicians or supervised MLPs. Unsupervised MLPs also were less likely to deliver timely care than physicians, as suggested by lower performance on several timeliness measures. Overall, their composite guideline concordance score was significantly lower than physicians’ or supervised MLPs’ score.

• • 1. Outcomes of care

Patients cared for by unsupervised MLPs had a shorter ED length of stay and were less likely to be admitted, as compared with patients cared for by physicians or supervised MLPs.

• • 1. Multivariable analyses

The lower performance for unsupervised MLPs was partially explained by patient and ED characteristics, but, even after adjustment, we found differences in quality of care between provider groups (Table 4). Compared with physicians, unsupervised MLPs were less likely to prescribe systemic corticosteroids in the ED, were more likely to prescribe inappropriate antibiotics at discharge (OR, 2.1; 95% CI, 1.1-4.1), and were less likely to administer inhaled ?-agonists within 15 minutes of arrival. The ED length of stay among patients cared for by unsupervised MLPs was approximately 1 hour shorter than those cared for by physicians.

Discussion

In this study, of 4029 patients presenting to 63 EDs with acute asthma, we found that MLPs were involved in 10% of the patient visits. In 2% of all visits, the MLPs provided unsupervised care without any documentation of physician involvement. Unsupervised MLPs saw lower-acuity asthmatic

Processes of care Prescribing systemic corticosteroids in ED 1.0 (referent) 1.1 (0.7-1.7) 0.4 (0.2-0.9) Not prescribing oral antibiotics at discharge 1.0 (referent) 0.9 (0.6-1.3) 0.5 (0.2-0.9) Timeliness measure Inhaled ?-agonists <=15 min of arrival 1.0 (referent) 0.7 (0.5-1.03) 0.2 (0.1-0.7) Composite score ? Composite guideline concordance score 0 (referent) -0.5 (-2 to 1) -6 (-9 to -3) Outcome of care ? ED length of stay, min 0 (referent) -6 (-33 to 20) -60 (-108 to -12)

* For the following 10 quality measures, there were no statistically significant differences by provider type in the multivariable model: prescribing inhaled ?-agonists in ED, prescribing inhaled anticholinergics in ED, not prescribing methylxanthines in ED, prescribing oral corticosteroids at discharge, not prescribing antibiotics in ED, assessment of PEF, initial PEF <=30 minutes of arrival, posttreatment PEF within 30 to 90 minutes, systemic corticosteroids <=75 minutes of arrival, and hospital admission. + Multivariable model was fit with ED random effects and adjusted for age, sex, race/ethnicity, ever admitted for asthma, transported to ED by ambulance, duration of symptoms b24 hours, recent upper respiratory infection, initial respiratory rate, initial oxygen saturation, initial PEF, plus 1 ED characteristic (region). ? Multivariable model was bootstrapped 1000 times to obtain the bias-corrected CI.

patients and provided less guideline-concordant care, as compared with physicians. Differences in patient mix explained some, but not all, of the differential quality of care observed between the provider groups.

We were struck by the consistently low performance rates for unsupervised MLPs on several quality measures–on both univariable and multivariable analyses. These findings are concerning because some would argue against further adjustment for patient mix since all patients who are eligible for a given treatment should receive the therapy. Never- theless, adjustment for patient mix only partially explained the worse performance of MLPs, and it did not eliminate the quality gap between provider groups.

In Primary care settings, MLPs have demonstrated their competence and accountability and have become a major part of the primary care workforce [17-19]. Why would MLPs perform less well in emergency care of asthma? Possible explanations may include lack of adequate physician backup and limited training and experience in caring for higher-acuity patients. We found that unsupervised MLPs cared for 2% of asthmatic patients. Among these patients, 10% arrived by ambulance, 70% had moderate-to- Severe asthma, and 2% were admitted. These data suggest that the role of MLPs in emergency care for asthma has expanded beyond minor exacerbations, a finding that is consistent with the national trend across all types of ED visits [1]. This raises concerns about limitations of MLP training and how independently MLPs can function in the ED, particularly for higher-acuity patients.

Indeed, a Canadian study found that the highest-ranked NP in a community hospital failed to achieve the same level of quality care as physicians, with the exception of follow-up-related complaints, simple lacerations, and isolated sore throats [20]. In the United States, a telephone survey of 250 EDs showed that PAs or NPs typically worked in fast-track areas to provide independent care for minor and nonemergent cases, particularly in high-volume, urban, and northeastern EDs [21]. These studies suggest that although unsupervised MLPs may not provide high- quality care to all types of ED patients, it may still be appropriate for them to provide independent care for a subset of ED patients. The identification of these conditions requires further study. The algorithmic nature of acute asthma care made it seem like an appropriate candidate for such autonomous care, but our data argue otherwise, in the absence of further MLP training.

This study examined mostly guideline concordance and did not include all potential outcome measures, such as wait times, patient satisfaction, and relapses. In several aspects of emergency care, MLPs have been shown to reduce wait times [3,22], reduce resident workload [22], and improve patient satisfaction in the ED [3,23,24] when working with emergency physicians. Perhaps for certain complex and high-acuity emergency conditions, MLPs may be used to supplement, rather than substitute for, overextended physi- cians. Moreover, MLPs may play more diverse roles,

depending on site-specific need, to maximize the benefit they bring to the emergency care team and to patients.

Limitations

This study has some potential limitations. First, because of the relatively small sample size of patients cared for by MLPs, our findings need to be replicated in future studies. Second, we did not have information on MLP identifiers, so the exact number of MLPs involved was not known. However, the key informant survey did collect information on “average number of MLPs during the past year” for each site. According to this survey, 126 and 127 MLPs were hired at the supervised and unsupervised MLP sites, respectively. The high number of hired MLPs, rotating shifts in the ED, and random sampling of visits did not support the argument that only a few MLPs provided care at the unsupervised sites. Third, the EDs that composed our sample are predominantly urban, academically affiliated hospitals. This may make our results less generalizable to community hospitals without academic affiliation. Fourth, although we have used restric- tions in defining quality metrics and multivariable adjust- ment technique to address confounding by severity, the association between unsupervised MLP care and suboptimal guideline concordance may be confounded by unmeasured factors. For example, we did not have information on wait times to see providers. Although we have attempted to control for this unmeasured factor by including several proxies (eg, arrival by ambulance) in the multivariable analyses, it is possible that patients cared for by MLPs had longer wait times, thereby affecting their timely use of inhaled ?-agonists. Fifth, because of the study design, we only included admission and ED length of stay as outcome measures in this study. Future study should include more important outcomes, such as return visits. Finally, the study relied on medical record review for quality assessment, and some of the apparent quality deficit may be due to underdocumentation. A study showed that underdocumenta- tion was more pronounced in physicians than that in NPs [2], and therefore, the quality gap would have been more significant had this been corrected.

Conclusions

In summary, PAs or NPs were involved in 10% of more than 4000 ED patients with acute asthma and provided independent care for approximately 2% of these patients. Compared with care provided by physicians or by supervised MLPs, there are opportunities for improvement in unsupervised MLP care. Our study suggests that quality of care provided by MLPs needs to be examined in individual conditions to better understand the scope of their practice in emergency settings, improve training, and inform health policy recommendations. At present, we suggest

MLPs work collaboratively with physicians in the care of acute asthma rather than substituting for the emergency physician providers.

Acknowledgments

The authors thank Dr David Blumenthal, Dr Paul D. Cleary, Dr James A. Gordon, Dr Edward Guadagnoli, Dr Rainu Kaushal, Dr David J. Magid, and Dr Sowmya R. Rao. We also thank the NEDSS site principal investigators and local chart abstractors; without their help, this study would not have been possible.

Appendix A. Supplementary data

Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j. ajem.2009.01.041.

References

1. Ginde AA, Espinola JA, Sullivan AF, Blum FC, Camargo Jr CA. Use of physician assistants and nurse practitioners in United States emergency departments, 1993-2005: implications for the emergency physician workforce. Am J Emerg Med 2008 [in press].
2. Sakr M, Angus J, Perrin J, Nixon C, Nicholl J, Wardrope J. Care of minor injuries by emergency nurse practitioners or junior doctors: a randomised controlled trial. Lancet 1999;354(9187):1321-6.
3. Buchanan L, Powers RD: Establishing an NP-staffed minor emergency

area. Nurse Pract 1997; 22(4): 175-8, 183, 187.

1. Ginde AA, Espinola JA, Camargo Jr CA. Improved overall trends but persistent racial disparities in emergency department visits for acute asthma, 1993-2005. J Allergy Clin Immunol 2008;122:313-8.
2. Expert Panel Report 3 (EPR-3). Guidelines for the diagnosis and management of asthma-summary report 2007. J Allergy Clin Immunol 2007;120(5 Suppl):S94-S138. Also available at http://www.nhlbi.nih. gov/guidelines/asthma/asthgdln.htm.
3. Sullivan AF, Camargo Jr CA, Cleary PD, et al. The National Emergency Department Safety Study: study rationale and design. Acad Emerg Med 2007;14(12):1182-9.
4. Radeos MS, Camargo Jr CA. Predicted peak expiratory flow: differences across formulae in the literature. Am J Emerg Med 2004;22(7):516-21.
5. United States Bureau of the Census. Census regions and Divisions of the United States. Available at http://www.census.gov/geo/www/ us_regdiv.pdf. Accessed November 16, 2008.
6. Office of Management and Budget. Standards for defining metropo- litan areas. Available at http://www.whitehouse.gov/omb/inforeg/ statpolicy/combined_msa_2007.pdf. Accessed November 16, 2008.
7. National Asthma Education and Prevention Program. Expert Panel Report II. Guidelines for the Diagnosis and Management of Asthma. Bethesda (Md): National Institutes of Health; 1997.
8. Tsai CL, Sullivan AF, Gordon JA, et al. Quality of care for acute asthma in 63 US emergency departments. J Allergy Clin Immunol 2009;123(2):354-61.
9. Brook RH, McGlynn EA, Cleary PD. Quality of health care. Part 2: measuring quality of care. N Engl J Med 1996;335(13):966-70.
10. Greenland S. Modeling and variable selection in epidemiologic analysis. Am J Public Health 1989;79(3):340-9.
11. Miettinen OS. Theoretical epidemiology: principles of occurrence research in medicine. New York: Wiley; 1985 [A Wiley medical publication].
12. Rabe-Hesketh S, Skrondal A. Multilevel and longitudinal modeling using Stata. College Station,(Tex): Stata Press; 2005.
13. Efron B, Tibshirani R. An introduction to the bootstrap. New York: Chapman & Hall; 1993 [Monographs on statistics and applied probability 57].
14. Horrocks S, Anderson E, Salisbury C. Systematic review of whether nurse practitioners working in primary care can provide equivalent care to doctors. BMJ 2002;324(7341):819-23.
15. Mundinger MO, Kane RL, Lenz ER, et al. Primary care outcomes in patients treated by nurse practitioners or physicians: a randomized trial. JAMA 2000;283(1):59-68.
16. Sackett DL, Spitzer WO, Gent M, Roberts RS. The Burlington randomized trial of the nurse practitioner: health outcomes of patients. Ann Intern Med 1974;80(2):137-42.
17. Steiner IP, Blitz S, Nichols DN, Harley DD, Sharma L, Stagg AP. Introducing a nurse practitioner into an urban Canadian emergency department. Cjem 2008;10(4):355-63.
18. Ellis GL, Brandt TE. Use of physician extenders and fast tracks in United States emergency departments. Am J Emerg Med 1997;15(3): 229-32.
19. Spisso J, O’Callaghan C, McKennan M, Holcroft JW. Improved quality of care and reduction of housestaff workload using trauma nurse practitioners. J Trauma 1990;30(6):660-3 [discussion 663-5].
20. Rhee KJ, Dermyer AL. Patient satisfaction with a nurse practitioner in a university emergency service. Ann Emerg Med 1995;26(2):130-2.
21. Nyberg SM, Waswick W, Wynn T, Keuter K. Midlevel providers in a level I trauma service: experience at Wesley Medical Center. J Trauma 2007;63(1):128-34.