a b s t r a c t

Study Objective: We investigated emergency physician knowledge of the Centers for Medicare & Medicaid Services (CMS) reimbursement for common tests ordered and procedures performed in the emergency department (ED), determined the relative accuracy of their estimation, and reported the impact of perceived costs on physicians’ ordering and prescribing behavior.

Methods: We distributed an online survey to 189 emergency physicians in 11 EDs across multiple institutions. The survey asked respondents to estimate reimbursement rates for a limited set of medical tests and procedures, rate their level of current cost knowledge, and determine the effect of Health expenditures on their medical decision making. We calculated relative accuracy of cost knowledge as a percent difference of participant estimation of cost from the CMS reimbursement rate.

Results: Ninety-seven physicians participated in the study. Most respondents (65%) perceived their knowledge of costs as inadequate, and 39.3% indicated that beliefs about cost impacted their ordering behavior. Eighty percent of physicians surveyed were unable to estimate 25% of the costs within +-25%, and no physicians estimated at least 50% of costs within 25% of the CMS reimbursement and only 17.3% of medical services were estimated correctly within +-25% by 1 or more physicians.

Conclusion: Most emergency physicians indicated they should consider cost in their decision making but have a limited knowledge of cost estimates used by CMS to calculate reimbursement rates. Interventions that are easily accessible and applicable in the ED setting are needed to educate physicians about costs, reimbursement, and charges associated with the care they deliver.

(C) 2014

Introduction

Background

Health care expenditure accounts for 17.9% of the US Gross domestic product, exceeding $2.6 trillion annually [1]. Americans pay $8402 per capita for health care, which is more than any other country in the world, yet have health outcomes that are far below those of other industrialized countries [1,2]. It is estimated that approximately $700 billion (5% of the gross domestic product) is

? Prior presentation: None.

?? Funding source/disclosures: None.

? Clinicaltrials.Gov Registration System, Identifier: NCT01883778.

?? Author contributions: C.B.H., T.E.M., C.A.P., and L.H.G. conceived the study and

designed the trial. C.B.H. and T.E.M. supervised the conduct of the trial and data collection.

C.B.H. managed the data, including quality control. C.B.H., L.H.G. and S.T.Y. provided statistical advice on study design and analyzed the data. C.B.H. drafted the manuscript, and all authors contributed substantially to its revision. C.B.H. takes responsibility for the manuscript as a whole.

* Corresponding author. 375 Chipeta Way, Suite A, Salt Lake City, UT 84018, USA. Tel.: +1 801 581 7234.

E-mail address: [email protected] (C. Broadwater-Hollifield).

spent annually on medical tests and procedures that may not improve health outcomes [3]. In the past decade, the mean expenditure for an emergency department (ED) visit preventing a hospital admission increased by 77%, and emergency care represents approximately 4% to 10% of all health care spending in the United States [4-6]. Physicians’ ability to make cost-effective decisions is limited by their education and understanding of Health care costs.[7].

The term health care “costs” is used in a variety of ways depending on the audience or individual using the term. Health care providers and administrators use the term costs to refer to the cost of production and money needed for resources for patient care [8]. policy makers use the term costs to refer to state and federal expenditure for health care [8]. Business owners use the term costs to illustrate costs of employee’s health insurance plans and premiums [8]. Finally, patients refer to costs as the amount of out-of-pocket money they pay at the time of service from a doctor or pharmacist and possibly their premiums [8]. Payment or reimbursement is “the amount paid to a provider for a medical service or supply. This amount is the shared responsibility of the health plan and beneficiary, and excludes amounts for non-covered services.” [9] The reimbursement rate is based on the cost of physicians’ services including the

http://dx.doi.org/10.1016/j.ajem.2014.01.044

0735-6757/(C) 2014

following: physicians work, practice expenses, and professional liability expenses [10,11].

Traditionally, emergency medicine (EM) physicians have been taught to consider a comprehensive differential diagnosis with little regard to cost or resource limitations [12]. The escalating costs of the US health care system have forced physicians into a dialogue about “cost consciousness” and appropriate resource allocation [12,13]. In today’s health care climate, physicians’ ability to provide high-value care is not only predicated on their clinical care but also taking into consideration the resources used and the Associated costs [14].

Efforts to modify physicians’ cost knowledge and shape clinical practice to provide more affordable care have been underway for more than 40 years [13,15,16]. Studies conducted over the last 3 1/2 decades suggest that physicians have poor knowledge about the cost of medical care [7,17-21], yet strategies to effectively change cost knowledge and improve cost-effective practice behaviors are difficult to implement and sustain over time [15,16]. To date, the number of studies published measuring emergency physician knowledge about the cost of health care is relatively small, and very few studies demonstrate how this knowledge translates into practice.

Importance

Physicians acknowledge that being aware of the costs associated with the care they provide is an important component of high-quality Health care delivery [22]. Simultaneously, physicians admit that this knowledge is deficient but concurrently report that erroneous cost estimates influence their medical decision making. Highlighting gaps in current knowledge will facilitate the development of targeted educational programs for emergency physicians, enabling them to deliver high-quality, high-value health care in the ED.

Goals of the investigation

This study sought to investigate emergency physician knowledge of the cost of common tests and procedures ordered in the ED, determine their relative accuracy in cost estimation, and report the impact of perceived costs on physician ordering and prescribing behavior.

Materials and methods

Study design

We conducted a cross-sectional study, using a convenience sample of emergency physicians from 1 academic medical center and 10 community hospitals within a single metropolitan area. The survey was administered between August and December 2012.

Setting and participants

Eligible participants consisted of 189 physicians: 51 emergency physicians (27 attending physicians, 24 residents) from the University of Utah Medical Center, which is an academic medical center, and 138 community emergency physicians from Emergency Physicians Inte- grated Care (EPIC). EPIC staffs 10 community hospitals in the major

Study instrument

The cross-sectional survey was designed to measure physician knowledge of cost of a limited set of medical tests and procedures and to assess how awareness of costs impacts their medical decision making. The survey consisted of the following: (1) questions measuring how much cost impacts physician decision making and test ordering, (2) a list of 26 items (laboratory tests, procedures, medication administration, professional fees for medical visits, and emergency medical service [EMS] transportation) for physicians to estimate the costs, and (3) a limited set of demographics about the physician. The novel survey was constructed based on existing literature, investigators’ hypotheses, interviews with emergency physicians, interviewer debriefing, FoCUS groups, and modified questions from existing surveys by Allan and Innes [7]. Innes et al [21], and Sehgal and Gorman [3]. Established methods to determine relevance of content were used in the survey development including expert review, pilot testing, and focus groups [23]. An expert panel consisting of 3 emergency physicians determined survey face and content validity. Five physicians and 7 medicine/research personnel who were not part of the research team completed 12 pilot surveys to determine relevance of content, language, and structure. The principal investigator (CBH) reviewed all pilot data. Individual review of content was conducted with the 5 physicians, and the principal investigator conducted a focus group with the 7 medical/research personnel. Changes were incorporated, and the revised survey was readministered to a subset of the original medical/research personnel. A second round of revisions was incorporated based on this feedback, and the finalized questionnaire was sent via e-mail to prospective participants with a link to a Web-based data collection instrument.

Data collection and processing

In the online, self-administered questionnaire, participants were asked to rate, on a 5-point Likert scale (1= strongly disagree to 5= strongly agree), the extent to which cost considerations impact their test ordering behavior, their self-perceived cost knowledge, their willingness to learn more about costs, and whether physicians should consider costs in the delivery of health care. A sample of survey questions is presented in Table 2. Next, participants were asked to “estimate the price reimbursed by Medicare” for several common tests, procedures, and medications delivered in the ED, as well as the reimbursement for an ED visit, General practice office visit, and

Table 1

Characteristics of physician respondents (n = 97)

Variables

Age (y), mean (SD) 43.6 (10.9)

Sex, no. (%)

Female 25 (25.8)

Male 71 (73.2)

Missing 1 (1.0)

Years in practice, no. (%)

b5-y experience 12 (12.4)

5- to b10-y experience 20 (20.6)

10- to b15-y experience 18 (18.6)

15- to b20-y experience 11 (11.3)

N 20-y experience 34 (35.1)

Missing 4 (2.1)

metroplitian areas of Utah. The 11 hospital EDs provide care for more

than 265 000 patients annually. The University of Utah institutional

Years since finishing training, no. (%) In training

8 (8.3)

review board approved the study. Ninety-seven participants (51.3% response rate) completed the survey, which represents 35.7% of emergency physicians in the urban catchment area. The study has been registered in the ClinicalTrials.gov Registration System, Identi- fier: NCT01883778.

b5-y experience 14 (14.4)

5- to b10-y experience 20 (20.6)

10- to b15-y experience 14 (14.4)

15- to b20-y experience 12 (12.4)

N 20-y experience 23 (23.7)

Missing 6 (6.2)

Table 2

Emergency physician agreement with cost impact on self-behavior (n = 97)

Value statement Proportion of emergency physicians reported level of agreement

	1 = Strongly disagree, no. (%)	2 = Disagree, no. (%)	3 = Neutral, no. (%)	4 = Agree, no. (%)	5 = Strongly agree, no. (%)
The costs of imaging or laboratory test influences my decision to order.	9 (9.3)	25 (25.8)	25 (25.8)	36 (37.1)	2 (2.1)
I have adequate knowledge of the charges associated with medical tests.	16 (16.5)	47 (48.5)	21 (21.7)	10 (10.3)	3 (3.1)
I would like to know more about the charges of specific medical tests.	1 (1.0)	3 (3.1)	12 (12.4)	53 (54.6)	28 (28.9)
Improved access to medical test charges would affect my ordering.	2 (2.1)	20 (20.6)	32 (33.0)	34 (35.1)	9 (9.3)
The patients’ insurance status affects my ordering of Diagnostic tests.(n = 96).	24 (24.6)	27 (27.8)	19 (19.8)	25 (26.0)	1 (1.0)
Physicians should consider cost-effectiveness when selecting which tests to	2 (2.1)	2 (2.1)	13 (13.4)	58 (59.8)	22 (22.7)

order for their patient’s.

transport to an ED. Finally, participants were asked to provide a limited amount of demographic information. Study data were collected and managed using Research Electronic Data Capture (REDCap) software hosted at the University of Utah [24]. A copy of the survey is available upon request.

Financial data

Financial data were abstracted from the 2012 and 2013 Centers for Medicare & Medicaid Services (CMS) Physician Fee Schedule, Diagnostic Laboratory Fee Schedule, and the Hospital Outpatient services Fee Schedule at the Current Procedural Terminology (CPT) code level [10]. For the analysis, the CMS reimbursement rate was used to calculate the estimation error using the formula [(CMS reimbursement) – participant estimated cost))/CMS reimbursement]

x 100 for each of the specific services. The CMS reimbursement rate

was used because these data are publicly available and used as a base for calculating payment rates across most payers for reimbursement for services provided. In addition, CMS has determined that the reimbursement ratio is related to the cost of care delivery and can be used as a surrogate for cost estimations [10] as described earlier. The term cost is used throughout the article for simplicity and all calculations used CMS reimbursement as the reference standard. Values presented are based on the national CMS payment structure for the professional and facility fees.

Analytical approach

Response data were summarized using standard descriptive statistics (mean, SD, median, interquartile range [IQR], and range) for continuous variables. Frequencies are reported for categorical variables such as number of years in practice, level of agreement with

Table 3

Accuracy in cost estimations among emergency physicians (n = 85)

Test/Procedure/Medical visit (CPT code)	Percent with correct estimation (within +-25% of cost)	Percent with correct estimation (within +-50% of cost)
Laboratories, no. (%)
Urinalysis, automated, with microscopy (81001)	8 (9.4)	13 (15.3)
Serum human chorionic gonadotropin 84703	26 (31.3)	40 (48.2)
CBC with differential (85025)	19 (22.4)	34 (40.0)
BMP (Na, CI, K, C02, BUN, Cr, Glu, Ca-80048)	9 (10.6)	24 (28.2)
Blood culture (87116)	9 (10.6)	11 (12.9)
Blood and urine toxicology screen-drug screen (G0434)	20 (20.8)	29 (34.5)
Laboratories (average percent correct), mean (SD)	18 (9.1)	29.9 (13.9)
Administration of drugs, no. (%)
IV morphine (96374)	7 (8.2)	28 (32.9)
IV zofran (96375)	7 (8.2)	15 (17.7)
Administration of fluids (99360)	25 (29.4)	35 (41.2)
IV antibiotics (all: below -96365)	13 (15.3)	31 (36.5)
IV vancomycin (96364)	15 (17.7)	29 (34.1)
IV cefuroxime (96365)	13 (15.3)	30 (35.3)
IV levofloxacin (96365)	15 (17.9)	29 (34.5)
Administration of drugs (average percent correct), mean (SD)	16 (7.2)	33.2 (7.3)
Imaging
Chest x-ray (71010)	18 (21.2)	45 (52.9)
Ankle x-ray (73600)	11 (12.9)	42 (49.4)
Knee x-ray (73560)	12 (14.1)	42 (49.4)
Head CT without contrast (70450)	16 (18.8)	44 (51.8)
Abdominal/Pelvic CT without contrast (74150)	13 (15.3)	30 (35.3)
MRI brain without contrast (70551)	17 (20.0)	38 (44.7)
Imaging (average percent correct), mean (SD)	17.1 (3.4%)	47.2 (6.5)
EMS transportation, no. (%) ground ambulance ride in the greater SLC area (A0427)a	13 (15.3)	43 (50.6)
AirMed flight ride in the greater SLC area (A0431)	19 (22.4)	28 (32.9)
EMS transportation (average percent correct, mean (SD)	18.8 (5.1)	41.8 (12.5)
Medical visit
General practitioner office visit (G0438)	21 (25.0)	51 (39.3)
Emergency department visit, level III (99283)	10 (11.8)	16 (18.8)
Medical visit(s) (average percent correct), mean (SD)	18.4 (9.4)	39.8 (29.6)
Average percent correct across all services, mean (SD)	17.3 (6.2)	37.3 (12.6)

CBC, complete blood count; BUN, blood nitrogen urea; BMP, basic metabolic panel; CT, computed tomography; MRI, magnetic resonance imaging; SLC, Salt Lake City.

^a Utah-Specific CMS ambulatory rate from the 2013 fee schedule.

the impact of cost on compliance and self-perceived level of competency in cost knowledge. We reported agreement with the cost impact statements as the proportion of physicians selecting each category of the 5-point Likert responses. Physician reimbursement estimates were compared with the CMS reimbursement as described above. When the calculated cost estimate was within +-25% of the actual reimbursement rate, it was reported as being accurate, a commonly used margin of error [3,14,19,21]. In addition, values are presented with a margin of error +-50% of the reimbursement rate to determine the proportion of individuals who were able to estimate costs with a broader margin of error. Each median error represents the median percent estimated above or below the actual CMS reimburse- ment using a positive or negative value, depending on the physicians’ over or under estimation of the value. The summary proportions across patient services and for all medical activities were calculated using absolute values. Based on the literature [3,7], it was anticipated that there would be a ~ 50% response rate from physicians. There were 51 EM physicians (27 attending, 24 residents) at the University of Utah and 138 emergency physician in EPIC. There were 189 possible provider subjects; therefore, a projected provider sample of n = 94 (about 50% of 189) was anticipated. Statistical analyses were performed using Stata 12.0 (StataCorp, College Station, TX).

Results

Table 1 illustrates respondent characteristics. Ninety-seven of the possible 189 participants (51.3% response rate) representing 11 EDs completed the survey. Because all responses were anonymous, we were unable to link the proportion of responses from each specific

facility or institution to determine any differences in responses based on service environment. The average (SD) age of respondents was

43.6 (10.9) years, 73.2% were male, 85.6% had been practicing for 5 or more years, and 8.3% were in residency training. Twelve individuals did not estimate more than 65% of the test and procedure reimbursement rates and therefore were not included in the reimbursement estimation analysis. There were no significant differences in demographics for those who estimated reimburse- ments and those who chose not to estimate the reimbursements of tests and procedures.

Physicians self-rated their behavior in considering costs when treating patients. Most physicians (65%) indicated that they do not have adequate knowledge of the cost of tests and procedures they frequently order. Less than half (39.2%) of respondents indicated that costs influence their decision to order imaging or laboratory tests. In addition, 82.5% of physicians indicated that physicians should consider costs-effectiveness when delivering care in the ED (Table 2). Finally, 29.8% (n = 29) of physicians who indicated they agreed or strongly agreed that the cost of imaging or a laboratory test influenced their decision to order it for a patient also indicated they were neutral (n = 8), disagreed (n = 19), or strongly disagreed (n = 2) they have adequate knowledge of costs.

We evaluated the overall accuracy of physician knowledge of reimbursements by calculating the number of medical service items physicians were able to estimate within a certain margin of error (Table 3). We first calculated the proportion of tests physicians estimated within +-25% of the reimbursement rate across all medical services (laboratories, drugs, imaging, EMS services, medical visits), and found that 17.3% (SD 6.5%) of estimates were within this range.

Table 4

Overestimation and underestimation of CMS reimbursement by greater than 50% among emergency physicians (n = 85)

Test/Procedure/Medical visit (CPT code)	CMS ratea	Underestimation by N 50%	Overestimation by N 50%
Laboratories, no. (%)
Urinalysis, automated, with microscopy (81001)	$8.83	4 (4.7)	72 (84.7)
Serum human chorionic gonadotropin 84703	$20.97	12 (14.1)	31 (36.5)
CBC with differential (85025)	$21.71	6 (7.1)	45 (52.9)
BMP (Na, CI, K, C02, BUN, Cr, Glu, Ca-80048)	$23.61	4 (4.7)	57 (67.1)
Blood culture (87116)	$30.15	4 (4.7)	70 (82.4)
Blood and urine toxicology screen-drug screen (G0434)	$122.98	27 (31.8)	28 (32.9)
Laboratories average percent, mean (SD)		11.2 (9.8)	59.4 (20.4)
Administration of drugs, no. (%)
IV morphine (96374)	$96.63	55 (64.7)	2 (2.4)
IV zofran (96375)	$114.66	69 (81.2)	2 (2.4)
Administration of fluids (99360)	$133.21	39 (45.9)	11 (12.9)
IV antibiotics (all: below 96365)	$221.77	42 (49.4)	12 (14.1)
IV vancomycin (96364)	$221.77	49 (57.7)	7 (8.2)
IV cefuroxime (96365)	$221.77	50 (58.8)	5 (5.9)
IV levofloxacin (96365)	$221.77	48 (56.5)	7 (8.2)
Administration of drugs (average percent), mean (SD)		59.2 (10.7)	7.7 (4.3)
Imaging, no. (%)
Chest x-ray (71010)	$69.77	5 (5.9)	35 (41.2)
Ankle x-ray (73600)	$75.89	10 (11.8)	33 (38.8)
Knee x-ray (73560)	$78.27	10 (11.8)	33 (38.8)
Head CT without contrast (70450)	$340.97	7 (8.2)	34 (40.0)
Abdominal/Pelvic CT without contrast (74150)	$353.56	2 (2.4)	53 (62.3)
MRI brain without contrast (70551)	$775.69	7 (8.2)	34 (40.0)
Imaging average percent, mean (SD)		8.1 (3.3)	43.5% (8.4)
EMS transportation Ground ambulance ride in the greater SLC area (A0427)b	$394.21	4 (4.7)	38 (44.7)
AirMed flight ride in the greater SLC area (A0431)	$3,267.70	26 (30.6)	31 (36.5)
EMS transportation average percent, mean (SD)		17.7 (13.0)	40.6 (4.1)
Medical visit
General practitioner office visit (G0438)	$169.43	24 (28.2)	9 (10.6)
Emergency department visit, level III (99283)	$203.24	1 (1.2)	68 (80.0)
Medical visit(s) average percent		14.7 (13.5)	45.3 (34.7)

CBC, complete blood count; BUN, blood nitrogen urea; BMP, basic metabolic panel; CT, computed tomography; MRI, magnetic resonance imaging; SLC, Salt Lake City.

^a “Cost” calculated based on CMS physicians reimbursement fees 2013 and costs for hospital outpatient services by Healthcare Common Procedure Coding system (HCPCS) code for CY 2013-CMS payment rate.

^b Utah-specific CMS ambulatory rate from the 2013 fee schedule.

Thus, fewer than 20% of all physician estimates were within +-25% of the reimbursement rate. We also calculated the percentage of tests that physicians estimated within +-50% of the reimbursement rate for all medical services queried and found that 37.3% of estimates were within this range (Table 3).

Results of the proportion of tests emergency physicians either overestimated or underestimated the CMS reimbursement rates by greater than 50% are show in Table 4. Overall, physicians tended to overestimate all tests, procedures, and medical visits by greater than 50%, with 59.4% of physicians overestimating laboratories, 43.5% overestimating imaging, 40.6% overestimating EMS transportation, and 45.3% overestimating a medical visit by greater than 50%. Conversely, 59.2% of physicians surveyed underestimated intravenous

(IV) drug administration and fluids by greater than 50%. The relative estimates of the costs of tests were largely inaccurate (Table 4).

Results of participant estimations of CMS reimbursement for medical services and the CMS reimbursement rates are shown in Tables 4 and 5. The range in reimbursement estimations was broad across the group. The median (SD) estimation error was 162.6% (130.5%) for a medical visit with a general practitioner or in the ED, 117.6% (89.9%) for laboratories, 56.9% (11.9%) for drug and drug

administration, 43.6% (24.8%) for imaging, and 17.5% (9.3%) for EMS services. For example, the median estimation for a blood culture (CPT 87116) was $100 (IQR, 50-200), but reimbursement by CMS is only

$30.15, which indicates a median overestimation error of 231.7% (IQR, 65.8-563.3; Table 5). Illustrations of the distribution of estimation errors for laboratories, administration of drugs, imaging, EMS transport, and medical visits are presented in Figs. 1 to 4. Physicians overestimated most laboratories, with the exception of a blood and urine toxicology screen (Fig. 1). Physicians underestimated all drugs and drug administration (Fig. 2) and overestimated the reimburse- ment rates of imaging studies (Fig. 3). The largest estimation errors

were for a level 3 ED visit (with a median error of a 293% overestimation in reimbursement [IQR, 96.8-638]; Fig. 4) and a urinalysis (with a median error of 239.8% [IQR, 126.5-466.3]).

Overall, only 17.3% of estimates were within +-25% of reimburse- ment values, and 37.3% were within +-50% of reimbursement values. There did not appear to be a specific type of medical service with better accuracy in estimation. Furthermore, this knowledge did not change based on numbers of years practicing medicine, nor was it better among those who had recently graduated from medical school or residency. In fact, the average estimate of the cost of a test or procedure was within +-25% of the actual cost 18.5% of the time for physicians who had more than 5 years in practice vs 13.9% of the time for physicians who had less than 5 years in practice or finished residency in 2008 or later.

We further analyzed the data by individual physician to determine a level of proficiency based on the proportion of correct estimations (an accurate response was defined as within +- 25% of the reimbursement rate) (Table 6). One physician accurately estimated 47.8% of all medical service reimbursements. Eleven (12.9%) physicians accurately estimated 29% to 37.5% of the reimbursement values for all medical services. Eighty-six percent of physicians surveyed did not accurately estimate at least 25% of the values, and 7 physicians (8.2%) did not correctly estimate a single reimbursement value. We then expanded the margin of error to +-50% of the reimbursement rate. One physician estimated 75% of the values within +- 50% of CMS reimbursement, and this was 1 of the 3 individuals who reported having received training on medical test pricing during medical school. Nine physicians (10.6%) accurately estimated 50% of the medical services within +- 50% of reimbursement rates, and 7 physicians (8.2%) estimated 25% of the medical services within +- 50% of reimbursement rates (Table 6).

Table 5

Median errors for cost estimations among emergency physicians (n = 85)

Test/Procedure/Medical visit (CPT code)	CMSa	Median estimation (IQR)	Estimation range	Median estimation error, % (IQR)b
Laboratories
Urinalysis, automated, with microscopy (81001)	$8.83	$30 (20-50)	$5-200	239.8 (126.5 to 466.3)
Serum human chorionic gonadotropin 84703	$20.97	$25 (16-50)	$5-150	19.2 (-23.7 to 138.4)
CBC with differential (85025)	$21.71	$40 (22-70)	$2-200	84.24 (1.3 to 222.4)
BMP (Na, CI, K, CO2, BUN, Cr, Glu, Ca-80048)	$23.61	$50 (30-100)	$5-300	111.8 (27.1 to 323.5)
Blood culture (87116)	$30.15	$100 (50-200)	$5-800	231.7 (65.8 to 563.3)
Blood and urine toxicology screen-drug screen (G0434)	$122.98	$100 (50-200)	$5-400	-18.7 (-59.3 to 62.6)
Administration of drugs
IV morphine (96374)	$96.63	$25 (15-50)	$5-200	-74.1 (-84.5 to -48.3)
IV zofran (96375)	$114.66	$30 (15-50)	$5-180	-73.8 (-86.9 to -56.4)
Administration of fluids (99360)	$133.21	$ 80 (40-100)	$0-300	-39.9 (-70 to -24.9)
IV antibiotics (all: below -96365)	$221.77	$120 (75-250)	$10-700	-45.9 (-66.2 to 12.7)
IV vancomycin (96364)	$221.77	$100 (50-200)	$5-500	-54.9 (-77.5 to -9.8)
IV cefuroxime (96365)	$221.77	$100 (50-175)	$5-500	-54.9 (-77.5 to -21.1)
IV levofloxacin (96365)	$221.77	$100 (60-200)	$10-700	-54.9 (-73 to -9.8)
Imaging Chest x-ray (71010)	$69.77	$100 (55-150)	$20-300	43.3 (-21.2 to 115)
Ankle x-ray (73600)	$75.89	$100 (50-200)	$20-350	31.8 (-34.1 to 163.5)
Knee x-ray (73560)	$78.27	$100 (50-175)	$20-350	27.8 (-36.1 to 123.6)
Head CT without contrast (70450)	$340.97	$450 (250-750)	$50-1500	32 (-26.7 to 120)
Abdominal/Pelvic CT without contrast (74150)	$353.56	$700 (400-1200)	$50-3000	98 (13.1 to 239.4)
MRI brain without contrast (70551)	$775.69	$1000 (700-2000)	$100-8000	28.9 (-9.8 to 157.8)
EMS transportation Local ground ambulance (A0427) c	$394.21	$500 (400-800)	$50-3000	26.8 (1.5 to 102.9)
Local AirMed flight (A0431)	$3,267.70	$3000 (1500-6000)	$0-20,000	-8.2 (-54.1 to 83.6)
Medical visit General practitioner office visit (G0438)	$169.43	$115 (80-200)	$20-2000	-32.1 (-52.8 to 18)
Emergency department visit, level III (99283)	$203.24	$800 (400-1500)	$100-5000	293 (96.8 to 638)

CBC, complete blood count; BUN, blood nitrogen urea; BMP, basic metabolic panel; CT, computed tomography; MRI, magnetic resonance imaging; SLC, Salt Lake City.

^a CMS reimbursement rate–calculated based on CMS physicians reimbursement fees 2013 and costs for hospital outpatient services by Healthcare Common Procedure Coding System (HCPCS) code for CY 2013-CMS national payment rate.

^b [(CMS reimbursement) – participant estimated cost))/CMS reimbursement] x 100.

^c Utah-specific CMS ambulatory rate from the 2013 fee schedule.

Fig. 1. Group estimation errors of laboratory values for the 6 laboratory tests commonly ordered in the ED in order of actual values. The horizontal axis is the percent of estimation error. For each laboratory test on the vertical axis, there is a box plot illustrating the distribution of estimation errors for the physician group. Boxes indicate the IQR; dots are medians, and there is a line at zero; whiskers extend to the highest and lowest values (excluding outliers).

Limitations

There are a number of limitations associated with this study. First, the data were self-reported, and therefore, the reliability of the data is dependent on the integrity and completeness of individual responses [25]. In addition, the quality of the responses relies on how well subjects understand the survey items. There may be a response bias due to the 51.3% response rate of physicians and possible differences between responders and nonresponders. It is not clear if the results can be generalized to other parts of the country or to other populations [7]. Owing to the cross-sectional study design, there could be alternative explanations for the study results [26], although the findings are similar to studies that have been conducted in other institutional settings. There are a number of methods for discussing and calculating costs including direct, indirect, fixed, variable, charges, reimbursement, payment, and allowed costs, all of which contribute to complexity and confusion; consequently, we cannot be sure how respondents interpreted the questions when asked about costs and reimbursements. Overall cost is calculated based on a number of variables including the following: (1) direct costs related to specific activity such as staff salaries, equipment, fringe benefits, and so on; (2) indirect costs or overhead are the central administrative expenses such as executive staff and central support

services, which cannot be allocated to a specific activity; (3) future costs include wage increases, escalations in cost of living, insurance, and so on; (4) capital costs are the costs of purchasing and developing tangible property such as durable goods, buildings, construction, and debt services (interest paid on funds borrowed) [27,28]. Charge is “The dollar amount a provider charges/asks for medical services rendered, which can be different from the allowed costs or amount paid.” [9] Payment/ Reimbursement or Allowed Costs is “the amount paid to a provider for a medical service or supply after provider discounts. Also defined as negotiated rates paid by a health plan to a provider for a medical service or supply that qualifies as a covered expense. This amount is the shared responsibility of the health plan and beneficiary, and excludes amounts for non-covered services” [9]. Finally, this study was not designed as an intervention; therefore, educating emergency physicians about the difference between cost, charge, and reimbursement was outside the scope of the study.

Discussion

This study analyzed emergency physician knowledge of CMS reimbursement for a limited set of tests and procedures that are commonly used in the ED. Physician awareness of reimbursement for

Fig. 2. Group estimation errors of 4 common drugs administered in the ED and administration of fluids in order of actual values. The horizontal axis is percent of estimation error. For each drug on the vertical axis, there is a box plot illustrating the distribution of estimates for the physician group. Boxes indicate the IQR; dots are medians, and there is a line at zero; whiskers extend to the highest and lowest values (excluding outliers).

Fig. 3. Group estimates of imaging commonly ordered in the ED in order of actual values. The horizontal axis is percent of estimation error. For each imaging test on the vertical axis, there is a box plot illustrating the distribution of estimate error for the physician group. Boxes indicate the IQR; circles are medians, and there is a line at zero; whiskers extend to the highest and lowest values (excluding outliers).

all medical services studied was poor. Our study found that 65% of emergency physicians indicated they have inadequate knowledge of costs associated with medical test, whereas 39.2% simultaneously reported that the costs of laboratory testing and imaging impact their decision to order these for patients. This discrepancy in reported perceptions highlights a need for further education in associated costs of common tests and procedures delivered in the ED.

Although cost has been at the center of the debate around the Patient Protection and Affordable Care Act, overall cost knowledge among emergency physicians remains low. According to Shrank et al [29], the lack of cost knowledge for emergency physicians may largely be due to their practice environment and that they have “relatively brief interactions with patients, manage patients from a variety of health care plans, and typically expect patients to follow-up with their primary care physicians to care for chronic conditions.” Moreover, EM residents are taught protocols to follow in emergency situations to rule out life- or limb-threatening conditions. Some of the costly laboratory tests and imaging ordered have an understandably low probability of discovering disease or influencing decision making, but physicians perceive that these tests might lower their legal risk [30]. Our study findings are consistent with previously published studies conducted specifically with emergency physicians by Mader and Playe

[31], Kraska et al [32], Hernu et al [33], and Gervais et al [34]. These studies measured the accuracy of estimations and awareness and cost knowledge, but none of the studies were conducted in the United States after 2004. In 2007, the Accreditation Council for Graduate Medical Education required all residency programs to “incorporate consider- ations of cost-awareness” into the curriculum [14,30,35,36]. In 2008, the American College of Emergency Physicians changed the EM Physician Rights and Responsibilities policy statement to include the following: “gaining knowledge of the basic principles of…reimbursement and practicing expense costs” [37]. Measuring possible changes in cost knowledge and accuracy of estimates allowed us to determine whether changes had occurred within the physician population since 2007.

A recent study published by Caldwell et al [38] found that efforts to create “cost transparency” through the use of pricing indexes are underway for a number of inpatient procedures. Similar efforts, however, are not common for ED outpatient treatments due to a lack of availability of these data [38]. In 2010, a single article described educational strategies targeting EM residents, whereas 27 studies targeted internal medicine residents. A limited number of studies [39,40] have shown that providing physicians with computerized drug and diagnostics cost information alters short-term behavior. The effects, however, are not sustained in long-term follow up, nor did these studies measure

Fig. 4. Group estimates of EMS transportation, a level III ED visit, and a General Practice Annual visit. The horizontal axis is percent of estimation error. For each item on the vertical axis, there is a box plot illustrating the distribution of estimate error for the physician group. Boxes indicate the IQR; circles are medians, and there is a line at zero; whiskers extend to the highest and lowest values (excluding outliers).

Table 6

Proportion of individuals physicians and level of proficiency in estimating reimburse- ment values for all medical services (n = 85)

Proportion of test accurate within	Physician, n (%)	Proportion of test accurate within	Physician, n (%)
+-25% (%)		+-50% (%)
0.0	7 (8.2)	8.3	2 (2.4)
4.1	3 (3.5)	12.5	5 (5.9)
8.3	10 (11.8)	16.6	4 (4.7)
12.5	10 (11.8)	20.8	2 (2.4)
13.0	3 (3.5)	25.0	7 (8.2)
16.6	14 (16.5)	26.1	1 (1.2)
17.3	1 (1.2)	29.1	4 (4.7)
20.8	17 (20.0)	33.3	12 (14.1)
25.0	8 (9.4)	37.5	10 (11.8)
29.1	5 (5.9)	39.1	2 (2.6)
33.3	5 (5.9)	41.6	9 (10.6)
47.8	1 (1.2)	45.8	7 (8.2)
		47.8	1 (1.2)
		50.0	9 (10.6)
		54.1	4 (4.7)
		58.3	2 (2.4)
		62.5	1 (1.2)
		66.6	1 (1.2)
		69.5	1 (1.2)
		75.0	1 (1.2)

correlations in perceived self-efficacy or attitudes of physicians [41]. A study conducted by Tierney et al [39] found that providing physicians with charge data electronically for diagnostic tests during ordering significantly reduced test ordering by 14% and reduced costs by reducing the number of more expensive tests by 13%. This intervention, however, did not increase overall physician knowledge, nor was the effect of the intervention Sustained AFter discontinuation. Hampers et al [40] found that providing real-time cost information in the electronic medical record reduced associated test charges by 27%. Schilling [42] reported that the distribution of a price list of common tests and procedures ordered in the ED resulted in a decrease in “investigation costs,” for example, imaging and laboratory testing, due to a reduction in the frequency of ordering. However, it is unclear if providing physicians with cost information will decrease test ordering if they already overestimate the cost of a test or procedure. In fact, providing physicians with variable cost information about individual tests and procedures may increase the likelihood of them ordering tests that are less expensive without regards for the cumulative impact it may have on patient charges.

To meet the challenges facing the medical community in building a value-based health care system that is sustainable, emergency physicians require more training in medical economics. Effective modalities for educating physicians have been described in the literature and include multifactorial approaches with interactive didactic sessions, individual feedback, and audit [43], with additional computerized tools including clinical reminders with cost information incorporated. Providing physicians with computer-generated prompts doubles compliance with best practice protocols, increases awareness of costs, and decrease unnecessary test ordering and admissions [44]. These may be some of the best strategies for teaching cost-effective practice in graduate medical education [45].

In summary, most emergency physicians indicated that they should consider cost in their decision making but demonstrated limited knowledge of reimbursement rates, as shown by low accuracy in estimations and wide variation in estimations compared with actual values. Interventions that are easily accessible and applicable in the ED setting are needed to educate physicians about costs, reimbursement, and charges associated with the care they deliver.

Acknowledgments

We are grateful to Matthew Broadwater-Hollifield, MD; Krista Viau, MS; and Kajsa Vlasic for their thoughtful review of an earlier

version of this manuscript. Study data were collected and managed using REDCap electronic data capture tools hosted at the University of Utah. REDCap is a secure, Web-based application designed to support data capture for research studies, providing (1) an intuitive interface for validated data entry, (2) audit trails for tracking data manipulation and export procedures, (3) automated export procedures for seamless data downloads to common statistical packages, and (4) procedures for importing data from external sources. REDCap is supported by National Center for Advancing Translational Sciences/National In- stitutes of Health (8UL1TR000105 [formerly UL1RR025764]) at the University of Utah, College of Nursing.

References

1. CDC. US Health Expenditures. Fast Stats Helath Expenditures in the US 2013. http://www.cdc.gov/nchs/fastats/hexpense.htm. Accessed July 23, 2013.
2. Dalen JE. We can reduce US health care costs. Am J Med 2010;123(3):193-4.
3. Sehgal RT, Gorman P. Internal medicine physicians’ knowledge of health care charges. J Graduate Med Educ 2011;3(2):182-7.
4. AHRQ. Emergency Room Services-Mean and Median Expenses per Person With Expense and Distribution of Expenses by Source of Payment: United States, 2010. Medical Expenditure Panel Survey Household Component Data.: Agency for Healthcare Research and Quality;2011.
5. National Center for Health Statistics. Health, United States, 2012 with special features on emergency care. Health and Human Services-(HHS). Hayettsville, MD: DHHS; 2012.
6. Lee MH, Schuur JD, Zink BJ. Owning the cost of emergency medicine: beyond 2%. Ann Emerg Med 2013;62(5):498 e493-505 e493.
7. Allan GM, Innes GD. Do family physicians know the costs of medical care? Survey in British Columbia. Can Fam Physician 2004;50:263-70.
8. Brownlee S, Colucci J, Walsh T. What “health care costs” really means. The Atlantic. Washington DC: Atlantic Media Company; 2012.
9. Health Care Cost Institute. Cost and utilization report glossary. Health Care Cost Utilization Report 2011. http://www.healthcostinstitute.org/glossary. Accessed March 10, 2013.
10. CMS. Center for Medicare and Medicaid Services-Physician Fee Schedule. In: Services DHH, ed. Baltimore, MD 2012.
11. MedPAC. In: (MedPAC) MPAC, editor. Medicare payment basics: physician and other health professionals payment system (October 2013). Washington DC: Medicare Payment Advisory Commission (MedPAC); 2013.
12. Rosenbaum L, Lamas D. Cents and sensitivity–teaching physicians to think about costs. N Engl J Med 2012;367(2):99-101.
13. Cooke M. Cost consciousness in patient care. What is medical education’s responsibility? N Engl J Med 2010;362(14):1253-5.
14. Rock TA, Xiao R, Fieldston E. General pediatric attending physicians’ and residents’ knowledge of inpatient Hospital finances. Pediatrics Jun 2013;131(6):1072-80.
15. Varkey P, Murad MH, Braun C, Grall KJ, Saoji V. A review of cost-effectiveness, cost- containment and economics curricula in graduate medical education. J Eval Clin Pract 2010;16(6):1055-62.
16. Eisenberg JM, Williams SV. Cost containment and changing physicians; practice behavior: can the fox learn to guard the chicken coop? JAMA 1981;246(19):2195-201.
17. Dresnick SJ, Roth WI, Linn BS, Pratt TC, Blum A. The physician’s role in the cost- containment problem. JAMA 1979;241(15):1606-9.
18. Allan GM, Lexchin J. Physician awareness of diagnostic and nondrug therapeutic costs: a systematic review. Int J Technol Assess Health Care 2008;24(2):158-65.
19. Allan GM, Lexchin J, Wiebe N. Physician awareness of drug cost: a systematic review. PLoS Med 2007;4(9):e283.
20. McGuire C, King S, Roche-Nagle G, Barry MC. Doctors’ attitudes about prescribing and knowledge of the costs of common medications. Ir J Med Sci 2009;178(3):277-80.
21. Innes G, Grafstein E, McGrogan J. Do emergency physicians know the costs of medical care? CJEM 2000;2(2):95-102.
22. Schoen C, Osborn R, Doty MM, Squires D, Peugh J, Applebaum S. A survey of primary care physicians in eleven countries, 2009: perspectives on care, costs, and experiences. Health Aff (Millwood) 2009;28(6):w1171-83.
23. Czaja R, Blair J. Designing surveys: a guide to decisions and procedures. Thousand Oaks, CA: Sage Publications, Incorporated; 2004.
24. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research Electronic Data Capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42(2): 377-81.
25. Passmore C, Dobbie AE, Parchman M, Tysinger J. Guidelines for constructing a survey. Fam Med 2002;34(4):281-6.
26. Carlson MD, Morrison RS. Study design, precision, and validity in observational studies. J Palliat Med 2009;12(1):77-82.
27. Cost benefit Knowledge Bank for Criminal Justice. Sensitivity analysis. Introduc- tion to the cost benefit knowledge bank for criminal justice 2013. http://cbkb.org/ toolkit/sensitivity-analysis/. Accessed March 10, 2013.
28. Finkler SA, Ward DM. Cost accounting for health care organizations: concepts and applications. Gaithersburg, MD: Aspen Pub; 1999.
29. Shrank WH, Asch SM, Joseph GJ, et al. Physicians’ Perceived knowledge of and responsibility for managing patients’ out-of-pocket costs for prescription drugs. Ann Pharmacother 2006;40(9):1534-40.
30. Okie S. Teaching Physicians the Price of Care. Kaiser Health News in collaboration with The New York Times May 4, 2010, 2010; Health.
31. Mader TJ, Playe SJ. Academic emergency physicians’ perception of patient charges resulting from routine ED care. Am J Emerg Med 1999;17(7):663-7.
32. Kraska EM, Miner JR, McGill JW, Sterner S. Providing cost information as a tool to change physician behavior. Ann Emerg Med 2004;44(4, Supplement):S75.
33. Hernu R, Cour M, Causse G, Robert D, Argaud L. Connaissance du cout des prescriptions aux urgences: enquete multicentrique aupres des prescripteurs. La Presse Medicale. 2013.
34. Gervais N, Veronique G, Jacques J, Dominique V. How aware are Belgian permanent and resident emergency physicians of common Medical costs and radiation doses? Eur J Emerg Med. 2011;18(6):344-350 310.1097/MEJ.1090b1013e3283466eea.
35. Accrediation Council for Graduate Medical Education. Educational Program Curriculum Components ACGME Competencies Systems-based Practice. Educa- tion ACfGM; 2008.
36. Moriates C, Soni K, Lai A, Ranji S. The value in the evidence: teaching residents to

    “choose wisely”. JAMA Intern Med 2013;173(4):308-10.

    American-College-of-Emergency-Physicians. EMTALA. Clinical & Practice Man- agement–ACEP Fact Sheet 2013. http://www.acep.org/content.aspx?id=25936 . Accessed November 20, 2013.

37. Caldwell N, Srebotnjak T, Wang T, Hsia R. “How much will I get charged for this?” Patient charges for top ten diagnoses in the emergency department. PLoS One 2013;8(2):e55491.
38. Tierney WM, Miller ME, McDonald CJ. The effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med 1990;322(21): 1499-504.
39. Hampers LC, Cha S, Gutglass DJ, Krug SE, Binns HJ. The effect of price information on test-ordering behavior and patient outcomes in a pediatric emergency department. Pediatrics 1999;103(4 Pt 2):877-82.
40. Beilby JJ, Silagy CA. Trials of providing costing information to general practi- tioners: a systematic review. Med J Aust 1997;167(2):89-92.
41. Schilling UM. Cutting costs: the impact of price lists on the cost development at the emergency department. Eur J Emerg Med 2010;17(6):337-9.
42. Grimshaw JM, Thomas RE, MacLennan G, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004;8(6):iii-iv, 1-72.
43. McDonald CJ, Hui SL, Smith DM, et al. Reminders to physicians from an introspective computer medical record. Ann Intern Med 1984;100(1):130.
44. Mazmanian PE, Davis DA. Continuing medical education and the physician as a learner: guide to the evidence. JAMA 2002;288(9):1057-60.