Original Contribution

ED antibiotic use for acute respiratory illnesses since pneumonia performance measure inception^?,??

Christopher Fee MD ^a,?, Joshua P. Metlay MD, PhD ^b, Carlos A. Camargo Jr MD, DrPH ^c,

Judith H. Maselli MSPH ^d, Ralph Gonzales MD, MSPH ^e

^aDepartment of Emergency Medicine, University of California San Francisco, San Francisco, CA 94143, USA

^bDivision of General Internal Medicine, University of Pennsylvania School of Medicine and VA Medical Center,

Philadelphia, PA 19104, USA

^cDepartment of Emergency Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA

^dDepartment of Medicine, University of California San Francisco, San Francisco, CA 94118, USA ^eDivision of General Internal Medicine, Department of Medicine, University of California San Francisco, San Francisco, CA 94118, USA

Received 1 July 2008; revised 9 September 2008; accepted 10 September 2008

Abstract

Objective: The study aimed to determine if emergency department (ED)-administered antibiotics for patients discharged home with nonpneumonia acute respiratory tract infections (ARIs) have increased since national pneumonia performance measure implementation, including antibiotic administration within 4 hours of arrival.

Methods: Design: Time series analysis. Setting: Six university and 7 Veterans Administration EDs participating in the Improving Antibiotic Use for Acute Care Treatment (IMPAACT) trial (randomized educational intervention to reduce antibiotics for bronchitis). Participants: Randomly selected adult (age N18 years) ED visits for acute cough, diagnosed with nonpneumonia ARIs, discharged home during winters (November-February) of 2003 to 2007. Main outcome: Time trend in ED-administered antibiotics, adjusted for patient demographics, comorbidities, vital signs, ED length of stay, IMPAACT intervention status, geographic region, Veterans Administration/university setting, and site and provider level clustering.

Results: Six thousand four hundred seventy-six met study criteria. Three hundred ninety-four (6.1%) received ED-administered antibiotics. Emergency department-administered antibiotics did not increase across the study period among all IMPAACT sites (odds ratio [OR], 0.88; 95% confidence interval [CI], 0.76-1.01) after adjusting for age, congestive heart failure history, temperature higher than 100?F, heart

^? This study was funded by the Translating Research into Practice initiative, jointly sponsored by the Agency for Healthcare Research and Quality, Rockville, Maryland (1 R01 HS013915) and the Health Services research and development Office of the Department of Veterans Affairs, Washington, D.C. (AVA-03-239). The funding agencies had no role in the design and conduct of the study; collection management, analysis, and interpretation of the data; or preparation, review, or approval of the article.

^?? Dr Metlay has served as a scientific consultant or received unrestricted educational funds from Aventis Pharmaceuticals and Roche Pharmaceuticals. Dr

Gonzales served as a consultant for Abbott Laboratories, Inc, to study C-reactive protein levels as a potential diagnostic test for outpatients with community- acquired pneumonia. Dr Camargo has received financial support for participation in conferences, consulting, and medical research from the following industry sponsors with an interest in Respiratory infections: Abbott, Aventis, Aventis Pasteur, GlaxoSmithKline, MedImmune, and Merck.

* Corresponding author. Department of Emergency Medicine, UCSF Medical Center, Box 0208, San Francisco, CA 94143, USA. Tel.: +1 415 353 1634;

fax: +1 415 3531799.

E-mail address: [email protected] (C. Fee).

0735-6757/$ - see front matter (C) 2010 doi:10.1016/j.ajem.2008.09.023

rate more than 100, blood cultures obtained, diagnoses, and ED length of stay. The ED-administered antibiotic rate decreased at IMPAACT intervention (OR, 0.80; 95% CI, 0.69-0.93) but not nonintervention sites (OR, 1.04; 95% CI, 0.91-1.19). Adjusted proportions receiving ED-administered antibiotics were 6.1% (95% CI, 2.7%-13.2%) for 2003 to 2004; 4.8% (95% CI, 2.2%-10.0%) for 2004

to 2005; 4.6% (95% CI, 2.7%-7.8%) for 2005 to 2006; and 4.2% (95% CI, 2.2%-8.0%) for 2006

to 2007.

Conclusions: Emergency department-administered antibiotics did not increase for patients with acute cough discharged home with nonpneumonia ARIs since pneumonia antibiotic timing performance measure implementation in these academic EDs.

(C) 2010

Introduction

Background

In 2002, the Joint Commission and Centers for Medicare & Medicaid Services (CMS) released pneumonia perfor- mance measure PN-5b (“identify patients who received their initial antibiotic within 4 hours of hospital arrival”) as one of their initial “Core Measures of Quality Care” [1]. Individual hospital performance has been publicly reported since its inception and has subsequently been linked to pay for performance demonstration programs. Thus, hospitals have placed a significant focus on meeting this and other quality measures. This perceived pressure to make certain that admitted patients with pneumonia receive antibiotics within the quality time “envelope” creates an environment whereby emergency physicians may lower their threshold for ordering antibiotics (ie, before securing a diagnosis of pneumonia) to capture a larger percentage of those who may have a final diagnosis of pneumonia.

Soon after the measure was instituted, emergency physicians and others expressed concern regarding the research that served as the foundation for the measure, how the measure was applied, and the potential for unintended consequences (eg, inappropriate antibiotic administration to patients who are ultimately diagnosed with conditions not requiring antibiotics, increased costs, increasing antibiotic resistance, and side effects such as diarrhea, yeast infections, and Allergic reactions) [2-10].

In response to these criticisms, the Joint Commission and CMS amended this measure to track the rate of antibiotic administration within 6 hours of hospital arrival for patients admitted with pneumonia (measure PN-5c) and no longer publicly report rates of compliance with the 4-hour rule (effective March 2008). The national average compliance, through June 2007 (the most recent data available), with the 6- hour rule is 93%, with the top 10% of hospitals achieving 100% compliance [11]. Although the change from a 4-hour to the new 6-hour window for antibiotic administration may have led to a decreased perceived time pressure, continued pressure to achieve or maintain such high compliance may increase the likelihood of unintended consequences.

At the same time that emergency physicians are feeling pressure to administer antibiotics in a timely manner for

patients who have pneumonia, pressure is also mounting to minimize the unnecessary use of antibiotics for nonpneu- monia acute respiratory tract infections (ARIs) [12].

Importance

In this era of increasing pressure to improve quality of care (public reporting, pay-for-performance, etc), agencies responsible for establishing quality measures must be aware of the potential for unintended consequences. If it can be demonstrated that inappropriate emergency department (ED) antibiotic administration to patients with outpatient ARIs has increased since the inception of the 4-hour rule, policy makers may need to reassess the cost-benefit ratio of this measure as it is currently applied.

Goals of this investigation

Our aim was to determine if the proportion of ED patients with nonpneumonia ARI diagnoses receiving ED- administered antibiotics has increased since implementa- tion of the Joint Commission and CMS pneumonia core measures, particularly the requirement to report compli- ance with rate of antibiotic administration within 4 hours of hospital arrival. In addition, we examined whether the trend in ED-administered antibiotics during this period differed as a function of whether the ED was required to publicly report pneumonia core measures and whether the ED was exposed to an educational intervention to reduce unnecessary antibiotic use for nonpneumonia ARIs. We hypothesized that the proportion of nonpneumonia ARI visits with an inappropriate ED-administered antibiotic has increased in response to the pneumonia antibiotic timing measure.

Methods

Study design

We performed a time series analysis across a 4-year series of winter seasons (November 2003-February 2004 through November 2006-February 2007).

Setting

Data were collected as part of the Improving Antibiotic Use in Acute Care Treatment (IMPAACT) trial and included patients seen in 6 university and 7 Veterans Administration (VA) EDs representing geographically diverse regions within the United States. All sites have an affiliation with an academic medical center. All university medical center sites have emergency medicine training programs and are members of the Emergency Medicine Network (EMNet; http://www.emnet-usa.org) [13]. Four university and 4 VA EDs received a patient and physician educational interven- tion to reduce overuse of antibiotics for colds, upper respiratory tract infections, and bronchitis during the winters of 2004 to 2005 and 2005 to 2006 [14].

Selection of participants

A full description of the IMPAACT data collection and processing has been previously described [15]. In brief, at each site and in each study year, 200 adult patient (age >=18 years) visits with a primary discharge diagnosis for an ARI (based on the International Classification of Diseases, 9th Revision [ICD-9] codes corresponding to cold/upper respira- tory infection, bronchitis, pharyngitis, sinusitis, Otitis media, and pneumonia) were randomly identified for medical record abstraction. To ensure equal representation of visits through- out the winter period, approximately 50 visits were randomly selected per month for November through February, for a total of 200 visits sampled per site per year. For this study, we limited the database to those patients with cough as a presenting complaint and discharged to home with a nonpneumonia ARI diagnosis (based on ICD-9 codes).

Data collection and processing

Medical record review was performed on site by research coordinators using a validated data abstraction instrument. All data collection personnel attended a 2-day training meeting in proper chart abstraction and human subjects’ protection and engaged in ongoing quality assurance activities, including ongoing feedback based on data review, highlighting missing data, and out-of-range values.

The following clinical data were included in the present study: patient demographics (age, sex, race, ethnicity), ED arrival time, ED discharge time, discharge status, vital signs (temperature, pulse, blood pressure, oxygen saturation), comorbidities, diagnostic tests (chest radiograph, blood culture), final ED diagnosis, and documentation of ED- administered antibiotics and route of administration (oral, intramuscular, or intravenous). Administration of an anti- biotic in the ED was based on review of the triage note and both physician and nurse ED notes. Although ARI visits were sampled according to the primary ICD-9-coded discharge diagnosis, a final diagnosis for each visit was

assigned based on the diagnoses listed in the physician note for the visit. Emergency department length of stay (LOS) was calculated as the difference between ED discharge time and ED arrival time.

Primary data analysis

To identify potential confounders of the association between period and ED administration of antibiotics, we calculated the unadjusted odds ratios (ORs) (with 95% confidence intervals [CIs]) for the association between each covariate (patient demographics, comorbidities, vital signs, diagnostic tests, ED LOS, final ED diagnosis, geographic region, and VA/university setting) and ED administration of antibiotics. For this purpose, ED LOS was dichotomized to less than 4 hours and 4 hours or more because we hypothesized that ED physicians would be compelled to administer antibiotics to patients with ED LOS approaching 4 hours to comply with the pneumonia performance measure.

Those variables found to be associated with ED- administered antibiotics were included in our multivariable analysis. Because of our large sample size, we used a stricter threshold for inclusion in our models (P b .05 rather than .1 or .2). We used backward selection to remove nonsignificant variables to create a more parsimonious model but retained study period, patient age, presenting heart rate, and final ED diagnoses in the all models (regardless of significance) because we had an a priori hypothesis that ED physicians are more likely to administer antibiotics to patients with increasing age and heart rate and discharged with certain diagnoses (eg, Acute bronchitis) [14].

We used this final model to measure the association between the proportion of visits with an ED-administered antibiotic and period using multivariable alternating logistic regression models (PROC GENMOD), which allows one to adjust for clustering at both the site and provider levels. In our final model, 486 visits were excluded from the regression analysis because of missing data for provider ID and predictor variables included in the final model (60 missing provider ID, 24 missing temperature data, 6 missing heart rate data, 393 missing ED LOS data [8 were missing both ED LOS and provider ID], and 11 missing blood culture data).

Sensitivity analysis

We repeated the analysis without retaining variables that we hypothesized, a priori, would influence the ED physician’s decision to administer antibiotics, unless they were found to be significant. We compared these results to our initial analysis.

To test for a difference in time trends in ED-administered antibiotics between VA and university EDs, and between intervention and non-intervention sites, we performed

Table 1 Unadjusted association between visit characteristics and probability of ED-administered antibiotics

Visit characteristic (n) Given ED-administered antibiotics Unadjusted

95% CI

Respiratory rate >=24 (369)	29 (7.9%)	1.34	0.90-1.98
Temperature N100?F (651)	97 (14.9%)	3.26	2.54-4.16
Heart rate >=100 beats per minute (1411)	129 (9.1%)	1.82	1.46-2.26
Oxygen saturation <=94% (636)	52 (8.2%)	1.18	0.87-1.61
Oxygen saturation

n (%) OR

Sex
Female (2096) (reference)	153 (7.3%)	1.00
Male (4388)	241 (5.5%)	0.74	0.60-0.91
Age (y)
18-44 (2612) (reference)	177 (6.8%)	1.00
45-64 (2640)	142 (5.4%)	0.78	0.62-0.98
>=65 (1224)	75 (6.1%)	0.90	0.68-1.19
Race
White (2165) (reference)	106 (4.9%)	1.00
Black (1461))	78 (5.3%)	1.10	0.81-1.48
Other (245)	13 (5.3%)	1.09	0.60-1.97
Unable to determine (2605)	197 (7.6%)	1.59	1.24-2.03
Ethnicity
Non-Hispanic (2040) (reference)	142 (7.0%)	1.00
Hispanic (875)	54 (6.2%)	0.88	0.64-1.22
Unable to determine (3561) Comorbidities a	198 (5.6%)	0.79	0.63-0.98
Cancer (217)	13 (6.0%)	0.98	0.56-1.74
Liver disease (302)	29 (9.6%)	1.69	1.13-2.51
Congestive heart failure (218)	30 (13.8%)	2.58	1.73-3.84
Stroke (137)	6 (4.4%)	0.70	0.31-1.60
Renal disease (157)	14 (8.9%)	1.53	0.87-2.67
Chronic obstructive pulmonary disease (554)	40 (7.2%)	1.22	0.87-1.72
Vital sign abnormalities b

Not obtained (1669) (reference)	50 (3.0%)	1.00
Obtained (4807)	344 (7.2%)	2.50	1.84-3.38
Blood cultures c

Not obtained (6354) (reference)	358 (5.6%)	1.00
Obtained (109) Chest x-ray e	36 (33.0%)	8.26	5.46-12.49
Not obtained (3942) (reference)	161 (4.1%)	1.00
Obtained (2522) ED length of stay (h) d	233 (9.2%)	2.39	1.94-2.94
0-1 (1237) (reference)	24 (1.9%)	1.00
1-2 (1617)	72 (4.4%)	2.36	1.48-3.76
2-3 (1244)	88 (7.1%)	3.85	2.43-6.09
3-4 (837)	69 (8.2%)	4.54	2.83-7.29
4-5 (445)	50 (11.2%)	6.40	3.88-10.54
N5 (703) ED length of stay (h) d	84 (12.0%)	6.86	4.31-10.91
b4 (4915) (reference)	252 (5.1%)	1.00
>=4 (1168)	135 (11.6%)	2.42	1.94-3.01
Diagnosis
Upper respiratory tract infection (2640) (reference)	63 (2.4%)	1.00
Acute exacerbation of chronic bronchitis (476)	45 (9.4%)	4.27	2.87-6.34
Bronchitis (1993)	162 (8.1%)	3.62	2.69-4.87
Otitis media (50)	6 (12.0%)	5.58	2.29-13.57
Pharyngitis (518)	65 (12.6%)	5.87	4.09-8.42
Sinusitis (488)	27 (5.5%)	2.40	1.51-3.80
Other ARI (152)	15 (9.9%)	4.48	2.49-8.07
Other (159)	11 (6.9%)	3.04	1.57-5.89

Practice setting
VA ED (3687) (reference)	174 (4.7%)	1.00
University ED (2789)	220 (7.9%)	1.73	1.41-2.12
Geographic region
South (1741) (reference)	40 (2.3%)	1.00
West (1996)	88 (4.4%)	1.96	1.34-2.87
North (1800)	228 (12.7%)	6.17	4.38-8.69
Midwest (939)	38 (4.1%)	1.79	1.14-2.82
Winter period (November-February)
2003-2004 (1681) (reference)	124 (7.4%)	1.00
2004-2005 (1730)	104 (6.0%)	0.80	0.61-1.05
2005-2006 (1576)	87 (5.5%)	0.73	0.55-0.97
2006-2007 (1489)	79 (5.3%)	0.70	0.53-0.94
Non-intervention site (2543) (reference)	148 (5.8%)	1.00
Intervention site (3933)	246 (6.3%)	1.08	0.87-1.33
a Comorbidities missing data for 10 observations. b Respiratory rate missing for 38 observations; temperature missing for 24 observations; heart rate missing for 18 observations; oxygen saturation missing for 1669 observations. c Blood cultures missing for 13 observations. d ED length of stay missing for 393 observations. e Chest x-ray missing for 12 observations.

stratified multivariate analysis, stratifying groups by VA and university ED status, and by IMPAACT intervention and non-intervention site status. Adjusted ORs and 95% CIs were calculated and used to compare the association between period and ED administration of antibiotics across strata.

Table 1 (continued)

Visit characteristic (n)

Given ED-administered antibiotics

n (%)

Unadjusted OR

95% CI

To confirm that the key estimate of year effect was not biased as a result of excluding visits with missing data (eg, ED LOS), we also examined trends among visits with missing ED LOS. All statistical computations were performed with SAS statistical software program (9th version; SAS Institute, Cary, NC). All study sites obtained approval from their respective institutional review boards before initiating data collection, including institutions at which data analyses were performed.

Results

Characteristics of study subjects

Of the 10 449 ARI visits initially abstracted across the 13 hospitals for the 4-year study period, 6476 patient visits met the specific criteria for this study (ie, presenting symptom of cough and nonpneumonia ARI discharge diagnosis). Unadjusted associations between visit charac- teristics and the probability of receiving ED-administered antibiotics are shown in Table 1. Several characteristics were associated with an increased probability of receiving ED-administered antibiotics over the 4-year study period. These factors were entered into a backward stepwise

logistic regression analysis of the association between study period and the probability of ED administered antibiotics, with the final model including the following variables: a history of congestive heart failure, fever, having blood cultures obtained, final ED diagnosis, and ED LOS more than 4 hours from stepwise selection, and age and tachycardia that we forced into the model based on a priori hypotheses.

Main results

Emergency department-administered antibiotics were documented in 394 visits (6.1%). Of those who received ED-administered antibiotics, 272 (69%) were oral, 79

(20.0%) were intravenous, 38 (9.7%) were intramuscular, and the route of administration was unable to be determined in 5 (1.3%). After adjusting for the potential confounders listed above, there was no evidence of increased ED- administered antibiotics during the study period (adjusted OR, 0.88; 95% CI, 0.76-1.01) (Table 2). The adjusted proportion receiving ED-administered antibiotics were as follows: winter 2003 to 2004 (6.1%; 95% CI, 2.7%-13.2%);

winter 2004 to 2005 (4.8%; 95% CI, 2.2%-10.0%); winter

2005 to 2006 (4.6%; 95% CI, 2.7%-7.8%); winter 2006 to

2007 (4.2%; 95% CI, 2.2%-8.0%).

Sensitivity analysis

The results of our repeat analysis (without automatically retaining those variables that we hypothesized, a priori,

analysis with this subgroup. The unadjusted OR for the trend in ED-administered antibiotics during the study period was

Table 2 The independent association between visit charac- teristics and probability of ED-administered antibiotics
		ED-administered (n = 5990) a	antibiotics
		Adjusted OR	95% CI
	Study period b	0.88	0.76-1.01
	Age (y)
18-44 (reference)		1.00
45-64		1.00	0.82-1.20
N65		0.95	0.68-1.33
History of congestive heart failure		2.80	2.25-3.47
Vital sign abnormalities
Temperature N100?F		2.40	1.73-3.31
Heart rate >=100 beats		1.11	0.90-1.38
per minute
Blood culture obtained		5.08	2.33-11.05
Diagnosis
Upper respiratory tract		1.00
infection (reference)
Acute exacerbation of		5.80	3.89-8.64
chronic bronchitis
Bronchitis		3.44	2.06-5.76
Other c		4.88	2.53-9.39
ED length of stay >=4 h 1.84 1.38-2.46 a Four hundred eighty-six visits were excluded from the regression analysis because of missing data for provider ID or predictor variables included in the final model (60 missing provider identification, 24 missing temperature data, 6 missing heart rate data, 393 missing ED length of stay data, [8 were missing LOS and provider ID], and 11 were missing blood culture data). b Study period is reporting the association for each successive winter period (November through February) compared to the preceding winter period. c Other includes otitis, pharyngitis, sinusitis, other ARI, and other.

2.04 (95% CI, 0.49-8.52) for those missing ED LOS from the single non-intervention VA site and 1.00 (95% CI, 0.71-1.42) for those with ED LOS at all non-intervention VA sites. The unadjusted percentage of ED-administered antibiotics was 0.89% (95% CI, 0.18%-2.58%) at the single non-interven- tion VA site with most of the missing ED LOS cases compared with 2.00% (95% CI, 1.34%-2.96%) at the other non-intervention VAs.

Limitations

would influence the ED physician’s decision to administer antibiotics), showed no material difference for variables associated with the probability of receiving ED-administered antibiotics. The adjusted ORs and 95% CIs are nearly identical (results not shown).

After stratifying for the principal IMPAACT trial interven- tion (educational interventions aimed at reducing inappropriate antibiotic use for bronchitis), we found a statistically significant decrease in ED-administered antibiotics during the study period at the intervention sites (OR, 0.80; 95% CI, 0.69-0.93) but no change at the non-intervention sites (OR, 1.04; 95% CI, 0.91-1.19) (Table 3). The adjusted ORs for ED-administered antibiotics over the study period were similar for the VA ED (OR, 0.89; 95% CI, 0.72-1.11) and university ED cohorts (OR, 0.90; 95% CI, 0.76-1.08) (Table 4).

Because our final multivariable models excluded visits missing ED LOS data, we explored this group (n = 393) in a separate series of analyses. Of note, 337 (85.8%) of these visits were from a single non-intervention VA site. Because only 7 visits (1.8%) received an ED-administered antibiotic, it was not possible to perform multivariable regression

As part of the principal reason for the IMPAACT trial, 8 EDs included in our cohort were randomly selected to receive physician and patient educational interventions aimed at reducing inappropriate antibiotic use for bronchitis [14]. Indeed we found evidence to suggest this was the case when we stratified our analysis by intervention vs non- intervention sites. However, we found no evidence of either an increase or decrease in ED-administered antibiotics over time in the non-intervention sites in our stratified analysis, consistent with our primary analysis.

Our data come from VA and university EDs associated with academic medical centers (ie, no community ED or nonacademic representation). Therefore, our results apply only to these settings. Although the VA is not subject to the Joint Commission and CMS core measures reporting, a similar set of quality measures has been implemented, and ED physicians in both settings are therefore likely to face similar pressures to achieve higher compliance. Our study results seem to support this.

Even within academic medical center settings, our results may be further limited to sites that would be willing to participate in the IMPAACT trial-meaning that sites were willing to have their antibiotic Prescription rates measured, and all sites were provided feedback on their antibiotic prescription rates for colds/upper respiratory infections, bronchitis, sinusitis, pharyngitis, and pneumonia at the beginning of each winter period regardless of their interven- tion status. Trends in ED administration of antibiotics may have been different in EDs that were not undergoing measurement of unnecessary Antibiotic prescribing for nonpneumonia ARIs. Nonetheless, if we look only at trends among the non-intervention EDs in IMPAACT, our findings suggest that a simple strategy for minimizing unintended effects of the pneumonia core measures could be measure- ment and feedback of antibiotic prescription rates for non- ARI diagnoses.

The findings from our analysis apply only to a -acuity patient population (100% were discharged to home). We did not measure what effect, if any, the Joint Commission and CMS pneumonia core measures may have had on the rate of inappropriate antibiotic administration to patients with discharge diagnoses other than nonpneumonia ARIs

Study period a	0.80	0.69-0.93	1.04	0.91-1.19
Age (y)
18-44 (reference)	1.00		1.00
45-64	1.03	0.76-1.39	0.93	0.80-1.09
N65	0.85	0.55-1.33	1.21	1.08-1.36
History of congestive heart failure	2.83	2.06-3.89	3.21	2.43-4.24
Vital sign abnormalities
Temperature N100?F	2.84	1.80-4.50	2.15	1.55-2.99
Heart rate >=100 beats per minute	0.91	0.74-1.13	1.51	1.09-2.09
Blood cultures obtained	3.51	1.58-7.80	7.34	2.50-21.53
Diagnosis
Upper respiratory tract infection (reference)	1.00		1.00
Acute exacerbation of chronic bronchitis	5.02	4.02-6.28	10.07	4.56-22.25
Bronchitis	2.72	1.61-4.57	6.72	2.74-16.49
Other b	3.94	2.78-5.57	9.64	2.49-37.35
ED length of stay >=4 h	2.07	1.30-3.30	1.51	1.05-2.19
a Study period is reporting the association for each successive winter period (November through February) compared to the preceding winter period. b Other includes otitis, pharyngitis, sinusitis, other ARI, and other.

(eg, congestive heart failure) nor to those requiring hospital admission.

Table 3 Independent association between visit characteristics and probability of ED-administered antibiotics stratified by intervention and non-intervention sites

ED-administered antibiotics

Intervention (n = 3797) Non-intervention (n = 2193)

Adjusted OR

95% CI

Adjusted OR

95% CI

Finally, our analyses were limited to those ED visits with complete data for each of the predictor variables. Excluding those with missing data may have introduced bias into the analysis. We investigated this possibility by examining

Table 4 Independent association between visit characteristics and probability of ED-administered antibiotics stratified by VA and university ED sites

ED-administered antibiotics

VA ED (n = 3291) University ED (n = 2699)

Adjusted OR

95% CI

Adjusted OR

95% CI

trends in the key estimate of year effect among visits with and without ED LOS (the variable with the most missing data points). We did see a nonstatistically significant difference in the trend in ED-administered antibiotics when we compared the visits missing ED LOS from the one non- intervention VA site (that had 85.8% of the total number of

Study period a	0.89	0.72-1.11	0.90	0.76-1.08
Age (y)
18-44 (reference)	1.00	0.67-1.11	1.00	0.87-1.29
45-64	0.86	0.52-1.09	1.06	1.11-1.96
N65	0.75		1.47
History of congestive heart failure	2.76	2.14-3.55	3.31	2.32-4.71
Vital sign abnormalities
Temperature N100?F	2.23	1.31-3.77	2.50	1.66-3.77
Heart rate >=100 beats per minute	1.10	0.82-1.48	1.17	0.91-1.51
Blood cultures obtained	6.81	2.45-18.96	3.35	1.77-6.34
Diagnosis
Upper respiratory tract infection (reference)	1.00		1.00
Acute exacerbation of chronic bronchitis	4.28	3.07-5.97	7.28	3.89-13.63
Bronchitis	2.90	1.73-4.87	3.80	1.33-10.87
Other b	2.29	1.63-3.21	7.88	3.37-18.44
ED LOS >=4 h	1.76	1.05-2.94	1.94	1.37-2.74
a Study period is reporting the association for each successive winter period (November through February) compared to the preceding winter period. b Other includes otitis, pharyngitis, sinusitis, other ARI, and other.

those missing ED LOS) with visits containing ED LOS from the other non-intervention VA EDs. It is interesting to note that the site missing most of the ED LOS data had an overall lower rate of ED antibiotic administration despite the increasing trend.

Discussion

The Joint Commission and CMS pneumonia core measures include a requirement to publicly report compli- ance with antibiotic administration within 4 hours of hospital arrival. Many believe this time-sensitive measure has placed significant pressure on emergency physicians to administer antibiotics before securing a pneumonia diagnosis. In this study, we found no evidence to suggest this was the case. In fact, in the aggregate model, we found a trend toward a reduction in ED-administered antibiotics over time that appears to be primarily driven by those sites randomly selected for educational interventions aimed at reducing inappropriate antibiotic use for bronchitis (as shown in our sensitivity analysis). Perhaps more notable is the fact that the non-intervention sites demonstrated no change in proportion of patients receiving ED-administered antibiotics over time, despite pressure to improve performance on the pneumonia antibiotic timing measure.

In contrast, 2 recent studies have reported an association between the initiation of the pneumonia antibiotic timing core measure and an increase in misdiagnoses of pneumonia and subsequent inappropriate antibiotic administration in ED patients requiring hospital admission [9,10]. Kanwar et al reviewed charts of patients with pneumonia hospital admission diagnoses over 6-month periods before and after the Joint Commission and CMS pneumonia antibiotic timing core measure. They found that although more patients received antibiotics within 4 hours of arrival, a larger percentage had normal chest X-rays and a larger proportion did not have a final hospital discharge diagnosis of pneumonia after initiation of the pneumonia antibiotic timing measure [9]. Welker et al studied the accuracy of ED pneumonia admitting diagnoses before and after conversion of the pneumonia antibiotic timing performance measure change from 8 to 4 hours. They found that patients admitted after the change to a 4-hour time to first antibiotic dose performance measure were 39% less likely to meet predefined diagnostic criteria for pneumonia than those admitted when the benchmark was within 8 hours. Most of those patients misdiagnosed with pneumonia did not have illnesses responsive to antibiotic therapy and may have experienced complications related to inappropriate antibiotic administration (Adverse drug reactions) or delay in arriving at the correct diagnoses [10]. Our study differs from these in 2 important ways. First, we sampled ED visits from multiple sites throughout the country, whereas the Kanwar and Welker studies analyzed data from single sites. Secondly, we

examined a lower-acuity cohort (those being discharged to home as opposed to those requiring hospital admission).

Our results are encouraging in that they suggest that ED physician ability to distinguish appropriate vs inappropriate conditions for antibiotic treatment in a low-acuity population is not adversely effected by any perceived pressure to meet the pneumonia antibiotic timing performance measure. Although the pneumonia antibiotic timing measure has been shown to increase the rate of inappropriate antibiotic administration to certain patient populations (ie, those with higher illness acuity requiring admission) [9,10], policy makers will need to weigh the costs and benefits associated with its institution and modify or abolish it accordingly.

In retrospect

Because participation in the IMPAACT trial could have resulted in selection bias or a co-intervention effect with the EDs in our study sample, it would be interesting to see similar analyses performed in other academic medical centers, or community EDs not associated with residency training programs, to further examine if the pneumonia core measures have led to an increase in unnecessary ED- administered antibiotics.

The effect of implementing the pneumonia antibiotic timing performance measure in 2003 is difficult to assess using administrative or existing data sources. We would have liked to have performed a similar analysis on a broader national sample of ED patients such as in the National Hospital Ambulatory Medical Care Survey. Unfortunately, the National Hospital Ambulatory Medical Care Survey database did not distinguish between ED-administered and outpatient prescription of medications at ED discharge until 2005 [16].

In conclusion, using a multicenter sample of academic- affiliated EDs, we find that ED-administered antibiotics have not increased for patients receiving ED care for nonpneu- monia ARIs since the Joint Commission and CMS instituted the pneumonia antibiotic timing performance measure. The generalizability of these findings, and the potential role of educational interventions and reporting of appropriate antibiotic treatment rates for ARIs as a strategy to minimize unintended consequences of national quality improvement efforts deserve further study. These studies will help hospitals design the most optimal strategies for delivering safe, effective, and high-quality care for patients who seek ED care for ARIs, which account for 2.7% of all US adult ED visits (98.8% of whom are discharged home) [17].

References

1. A comprehensive review of development and testing for national implementation of hospital core measures. The Joint Commission web site. Available at: http://www.jointcommission.org/NR/rdonlyres/

48DFC95A-9C05-4A44-AB05-1769D5253014/0/AComprehensiveRe-

viewofDevelopmentforCoreMeasures.pdf [Accessed March 5, 2008].

1. Walls RM, Resnick J. The Joint Commission on Accreditation of Healthcare organizations and Center for Medicare and Medicaid Services community-acquired pneumonia initiative what went wrong? Ann Emerg Med 2005;46:409-11.
2. Fee C, Weber E, Sharpe BA, et al. JCAHO/CMS core measures for community-acquired pneumonia. Ann Emerg Med 2006;47(5):505.
3. Fee C, Weber EJ. Identification of 90% of patients ultimately diagnosed with community-acquired pneumonia within four hours of emergency department arrival may not be feasible. Ann Emerg Med 2007;49(5):553-9.
4. Pines JM. Measuring antibiotic timing for pneumonia in the emergency department: another nail in the coffin. Ann Emerg Med 2007;49:561-3.
5. Fee C, Weber EJ, Sharpe BA, Quon T. When is a scarlet letter really a red badge of courage?: the paradox of percentage of pneumonia patients receiving antibiotics within 4 hours in accordance with JCAHO and CMS core measures. Ann Emerg Med 2007;50(2):205-6.
6. Pines JM, Hollander JE, Lee H, Everett WW, et al. Emergency department operational changes in response to pay-for-performance and antibiotic timing in pneumonia. Acad Emerg Med 2007;6:545-8.
7. Thompson D. The pneumonia controversy: hospitals grapple with 4 hour benchmark (News and Perspective). Ann Emerg Med 2006;47:259-61.
8. Kanwar M, Brar N, Khatib R, Fakih MG. Misdiagnosis of community- acquired pneumonia and inappropriate Utilization of antibiotics: side effects of the 4-h antibiotic administration rule. Chest 2007;131:1865-9.
9. Welker JA, Huston M, McCue JD. Antibiotic timing and errors in diagnosing pneumonia. Arch Intern Med 2008;168:351-6.
10. United States Department of Health and Human Services Hospital Compare website. Available at: http://www.hospitalcompare.hhs.gov [Accessed April 14, 2008].
11. Stone S, Gonzales R, Maselli J, et al. Antibiotic prescribing for patients with colds, upper respiratory tract infections, and bronchitis: a national study of hospital-based emergency departments. Ann Emerg Med 2000;36:320-7.
12. Metlay JP, Camargo CA, Bos K, et al. Assessing the suitability of intervention sites for quality improvement studies in emergency departments. Acad Emerg Med 2005;12:667-70.
13. Metlay JP, Camargo Jr CA, MacKenzie T, et al. Cluster-randomized trial to improve antibiotic use for adults with Acute respiratory infections treated in emergency departments. Ann Emerg Med 2007;50:221-30.
14. Gonzales R, Camargo Jr CA, MacKenzie TD, et al. Antibiotic treatment of acute respiratory tract infections in Acute care settings. Acad Emerg Med 2006;13:288-94.
15. National Center for Health Statistics website. Available at: http://www. cdc.gov/nchs/about/major/ahcd/ahcd1.htm#Micro-data [Accessed March 31, 2008].
16. Merrill CT, Owens PL, Stocks C. Statistical brief #47. Emergency department visits for adults in community hospitals from selected states, 2005. Healthcare Cost and Utilization Project, February 2008. Agency for Healthcare Research and Quality website. Available at: http://www.hcup-us.ahrq.gov/reports/statbriefs/sb47.jsp [Accessed May 19, 2008].