a b s t r a c t

Background: The United States continues to face a significant issue with Opioid misuse, overprescribing, depen- dency, and overdose. electronic health record interventions have shown to be an effective tool to modify opioid prescribing behaviors. This Quality improvement project describes an EHR intervention to reduce daily dosing in Opioid prescriptions in 11 emergency departments (ED) across the largest safety net health system in the US.

Measures: The primary outcome measure was the rates of oxycodone-acetaminophen 5-325 mg prescriptions exceeding 50 morphine milligram equivalents per day (MMED) pre- vs. post-intervention; and stratified by individual hospitals and provider type.

Intervention: The defaults for dose and frequency were uniformly changed to ‘every 6 hours as needed’ and ‘1 tablet’, respectively, across 11 EDs.

Outcomes: The percentage of prescriptions greater than or equal to 50 MMED decreased from 46.0% (1624 of 3530 prescriptions) to 1.6% (52 of 3165 prescriptions) (96.4% relative reduction; p < 0.001). All 11 hospitals had a significant reduction in prescriptions exceeding 50 MMED. nurse practitioners had the highest relative reduction of prescriptions exceeding 50 MMED at 100% (p < 0.001), and the attendings/fellows had the lowest relative reduction at 95.6% (p < 0.001).

Conclusions/Lessons Learned: Default nudges are a simple yet powerful intervention that can strongly influence opioid prescribing patterns.

(C) 2023

1. Introduction

The opioid epidemic in the United States has led to negative socio- economic and health impacts such as increased number of overdose deaths [1], Opioid use disorder [2], Hepatitis C and HIV infections [3], and neonatal Opioid abstinence syndrome [4]. According to the Centers for Disease Control and Prevention (CDC), there were over 93,000 Drug overdoses in 2020, with opioids involved in over two- thirds of those deaths [5]. Over-prescription of opioids is an important

Abbreviations: CDC, the Centers for Disease Control and Prevention; MMED, morphine milligram equivalents per day; EHR, electronic health record; ED, Emergency Departments; NYC H + H, New York City Health + Hospitals; PDMP, prescription drug monitoring programs.

* Corresponding author at: 50 Water Street, 1627 New York, NY, USA.

E-mail address: kroussm@nychhc.org (M. Krouss).

contributor to the opioid crisis, given its association with diversion, misuse, and overdose [6].

The CDC recommends clinicians use caution when prescribing higher doses of opioids, specifically, avoiding doses >90 morphine milligram equivalents per day (MMED) unless the benefits outweigh the risks [5]. Furthermore, recommendations advise against 30 MMED or higher in opioid naive patients and 50 MMED or higher in patients at increased risk of opioid-related harms. The New York City Emergency Department Discharge Opioid Prescribing Guidelines does not include any specifics on MMED, but recommends prescribing no more than a 3-day course of opioid medication for acute pain in most patients [8].

Opioid stewardship programs involving a range of electronic health record (EHR) strategies, including education and feedback, prescribing defaults, and Prescription Drug Monitoring Programs (PDMPs) can be effective in shaping prescription behaviors by physicians [9-11]. Of these, altering the opioid default dispense parameters is a simple,

https://doi.org/10.1016/j.ajem.2023.06.005

0735-6757/(C) 2023

low-cost, yet highly effective nudge. Studies have shown that changing the default quantity, duration, and number of refills can influence opioid Analgesic prescribing in academic Emergency Departments (ED) [12- 21].

To date, there is sparse literature on the effectiveness of EHR interventions in reducing total daily dose of opioids on discharge, or targeting MMED. Bursua et al. [22] changed the dosing defaults for hydrocodone-acetaminophen products and achieved a 5.8% absolute reduction of the number of prescriptions with >50 MMED in a large ac- ademic clinical enterprise (inpatient, outpatient, ED). In a retrospective urban academic multicenter study, Smalley et al. [23] achieved a signif- icant 4% absolute reduction of all ED Opioid prescriptions exceeding 30 MEDD utilizing 4-tiered, EHR-based interventions.

Most previous EHR interventions were done in single center, high- resourced settings such as academic institutions. The effectiveness and the variation in multi-hospital settings, particularly in resource- limited safety net systems deserve further study. Many medically underserved patients who experience a high prevalence of substance use and barriers to substance use treatment seek care through safety net facilities, which deliver health-care services regardless of patient’s ability to pay [24]. Here we describe an EHR intervention to reduce opi- oid prescriptions exceeding recommended MMED across the largest safety net health system in the US.

1. Methods
   1. Project setting

This quality improvement initiative was implemented at New York City Health + Hospitals (NYC H + H), the largest municipal health sys- tem in the US, including 11 EDs. All hospitals are metropolitan, teaching centers with 6 of the 11 hospitals serving as trauma centers. Our project was deemed a quality improvement project by the NYC H + H Central Research Office, and thus an Institutional Review Board submission was not required. The intervention was led and designed by the System

High Value Care Council with input from subject matter experts from patient safety, emergency medicine, internal medicine, and pharmacy.

• 1. Intervention

NYC H + H uses a single EHR throughout the system, Epic (Verona, Wisconsin). Medications may be ordered using a prebuilt Preference List that is uniformly accessed by all 11 NYC H + H EDs. Physicians may input parameters such as dose, route, frequency, dispense quantity, and number of refills (Fig. 1).

Prior to our intervention, we reviewed the most commonly

prescribed opioids in the ED. Oxycodone-acetaminophen was pre- scribed the most (63.9%), therefore, chosen for intervention. The prefer- ence list for oxycodone-acetaminophen 5-325 mg products had the following defaults: dose of ‘1-2 tablet’, frequency of ‘every 4-6 hours as needed’, refill of 0, duration of 3 days, and dispense quantity of 10 tablets. In our intervention, the defaults for dose and frequency were changed to ‘every 6 hours as needed’ and ‘1 tablet’, respectively. The new dosing defaults were selected to follow the CDC guideline of pre- scribing no >30 MMED for opioid naive patients. The pre-intervention refill, duration, and dispense quantity were unchanged and remained accordant to the guidelines. The intervention was implemented on March 29, 2022 across all 11 EDs.

• 1. Measures and statistical analysis

The pre-intervention period was March 30, 2021 to March 28, 2022 (52 weeks). The post-intervention period was March 29, 2022 to Febru- ary 13, 2023 (46 weeks).

The primary outcome measure was the proportion of prescriptions exceeding 50 MMED. This target was chosen based on the CDC recom- mendation against prescribing >50 MME per day of opioids except in certain circumstances such as active cancer treatment, palliative care, and end-of-life care [7]. This threshold aimed to account for the

Fig. 1. Dosing default modifications on EHR.

potential inclusion of patients who were not opioid-naive in the ED population and to capture the impact of the intervention on reducing dosages associated with increased risks. The primary outcome measure was also stratified by individual hospitals and clinician type. Of note, Jacobi and North Central Bronx are two different hospitals under the same operating certificate and a singular data entity. Pre- vs. post- intervention proportions prescriptions exceeding 50 MMED were compared using a two-sample test for equality of proportions.

Additionally, the system-wide pre- vs. post-intervention weekly proportions of prescriptions exceeding 50 MMED were compared via a linear regression to measure changes over time. Intercepts at the intervention date were compared to detect an immediate change in Prescription rates (level difference). The temporal slopes were also compared to detect changes in prescription rates over time (slope difference).

Secondary outcome measures included the maximum MMED, total quantity, minimum days supplied, and total MME. The average number of prescriptions per week was calculated by dividing the total number of prescriptions by the total number of weeks for each intervention group. These were all compared pre- vs. post-intervention via two-sample t-test assuming unequal variance (Welch test). Indications for prescrip- tions were also compared pre- vs. post-intervention via a two-sample test for equality of proportions.

Demographic data including age, sex, race, and ethnicity were compared pre- vs. post-intervention to measure changes in the patient population. Age was compared via a Welch test. Other demographic data was compared via a two-sample test for equality of proportions.

As a balance measure, we compared the number of ED revisits during the 7-day and 30-day period after the index prescription pre- and post-intervention. We used this measure as a surrogate marker to identify potential inadequate pain management.

Data were abstracted through SQL query. All analyses were performed with version 4.0.3 of the R programming language (R Core Team, 2020).

1. Results

The pre-intervention average age was 45.3 years and 45.5 years post-intervention (p = 0.65) (Table 1). The pre-intervention included 3530 prescriptions, and the post-intervention period included 3165 prescriptions. There was no significant difference in the weekly prescription rates (67.9 pre-intervention vs 68.8 post-intervention; p = 0.63) and frequency of all indications for the opioid prescriptions (Table 2).

Compared to the pre-intervention period, the average maximum MMED across all prescriptions in the post-intervention period decreased from 52.4 to 29.3 MMED (44.1% relative reduction, p < 0.001; dispense quantity decreased from 10 to 9.7 tablets (2.7% rel- ative reduction; p < 0.05); and minimum number of days supplied

Table 1

Demographics.

Table 2

Indications for opioid prescription.

Total Count % Total

Indication	Pre	Post		Pre	Post	Difference	p-value*
Acute pain	3366	3018		95.4%	95.4%	0.0%	1.00
Chronic pain	88	92		2.5%	2.9%	0.4%	0.33
cancer care	70	48		2.0%	1.5%	-0.5%	0.18
Other	5	5		0.1%	0.2%	0.0%	1.00
Palliative care	1	2		0.0%	0.1%	0.0%	0.92

* All p-values from 2-sample test for equality of proportions with continuity correction.

increased from 1.7 to 2.5 days (53.0% relative increase; p < 0.001) (Table 3).

Across all EDs, the percentage of prescriptions greater than or equal to 50 MMED decreased from 46.0% (1624 of 3530 prescriptions) to 1.6% (52 of 3165 prescriptions) (96.4% relative reduction; p < 0.001) (Table 4). Using linear regression to compare pre-intervention to post- intervention (Fig. 2), the level difference was -45.1% (47.3% to 2.2%,

p < 0.001) and slope difference was -0.07% (0.05% to -0.03%, p = 0.25). A comparison of prescriptions exceeding 50 MMED reductions in individual hospitals is shown in Table 4. All eleven hospitals had a sig- nificant reduction in prescriptions exceeding 50 MMED. Jacobi/NCB Hospital had the highest relative reduction of 99.0% (p < 0.001).

Among different types of clinicians (Table 5), nurse practitioners had the highest relative reduction of prescriptions exceeding 50 MMED at 100% (p < 0.001), followed by physician assistants at 98.4% (p < 0.001) and resident physicians at 97.3% (p < 0.001). Attendings/ fellows had the lowest relative reduction at 95.6% (p < 0.001) but the highest absolute reduction at 44.4% (46 to 1.6; p < 0.001).

For the balancing measure, the rate of ED re-visit during the 7-day and 30-day period was 12.9% and 23.0% pre-intervention, respectively, and 12.4% (p = 0.56) and 22.8% (p = 0.87) post-intervention, respectively.

1. Discussion

Our initiative successfully reduced the total daily dose of opioids prescribed across 11 EDs in the largest safety net system in the US. This expands on previous work in using defaults nudges in the EHR to achieve change [12-21], but focuses specifically on MMED [22,23]. This is a crucial outcome measure given that opioid dosages above 50 MMED are associated with minimal pain improvement without any functional benefit, while the incidence of serious harms increases, such as misuse, overdose, and death [25,26].

Prior studies targeting the daily dosing of opioid prescriptions have been implemented successfully in different settings with varying levels of success. Similar to our study, Bursua et al [22] modified the dosing defaults for hydrocodone-acetaminophen in an academic multisite enterprise (inpatient, outpatient, ED). However, the study achieved a lower reduction (15.6% to 9.56%) of the opioid prescriptions exceeding

Table 3

Pre- and Post-intervention system-wide comparison of oxycodone-acetaminophen 5-325 prescribing behaviors.

Average Across Prescriptions

Variable	Pre Average (SD)	Post Average (SD)	Difference	p-value
Age	45.3 (15.1)	45.5 (15.7)	0.2	0.65
Female (%)	48.2%	47.8%	-0.4%	0.77
Black or African
American (%)	41.3%	40.1%	-1.3%	0.3
Hispanic/Latinx (%)	34.1%	36.7%	2.6%	< 0.05

Difference

American/Alaskan

Two or more Races/Other (%)	10.6%	10.0%	-0.6%	0.48	Outcome	Pre	Post		Absolute (Post – Pre)	Relative (Post % Pre)	p-value?
White (%)	10.2%	9.6%	-0.6%	0.44	MME/Day	52.4	29.3		-23.1	-44.1%	<0.001
Asian/Native Hawaiian/-					Total Quantity	10.0	9.7		-0.3	-2.7%	<0.05
Pacific Islander (%)	3.6%	3.4%	-0.2%	0.73	Min Days Supplied	1.7	2.5	0.9		53.0%	<0.001
Native					Total MME	74.7	72.7	-2.0		-2.7%	<0.05

Native (%) 0.3% 0.2% 0.0% 0.97

* All p-values from an unpaired 2-sample t-test assuming unequal variance (Welch test).

Pre- and Post-intervention Comparison of Oxycodone-Acetaminophen 5-325 mg orders exceeding 50 MMED within in 11 Emergency Departments.

Above50MMED/Total % Total Above50MMED % Difference

Location	Pre	Post		Pre	Post		Absolute (Post- Pre)	Relative (Post % Pre)	p-value?
System Wide	1624/3530	52/3165		46.0%	1.6%		-44.4%	-96.4%	< 0.001
BELLEVUE	39/104	3/89		37.5%	3.4%		-34.1%	-91.0%	< 0.001
CONEY ISLAND	123/307	2/258		40.1%	0.8%		-39.3%	-98.1%	< 0.001
ELMHURST	98/262	2/289		37.4%	0.7%		-36.7%	-98.1%	< 0.001
HARLEM	237/399	3/365		59.4%	0.8%		-58.6%	-98.6%	< 0.001
JACOBI/NCB	235/562	2/489		41.8%	0.4%		-41.4%	-99.0%	< 0.001
KINGS COUNTY	271/670	7/587		40.4%	1.2%		-39.3%	-97.1%	< 0.001
LINCOLN	241/477	6/390		50.5%	1.5%		-49.0%	-97.0%	< 0.001
METROPOLITAN	80/162	11/188		49.4%	5.9%		-43.5%	-88.2%	< 0.001
QUEENS	215/369	13/317		58.3%	4.1%		-54.2%	-93.0%	< 0.001
WOODHULL	85/218	3/193		39.0%	1.6%		-37.4%	-96.0%	< 0.001

* All p-values from 2-sample test for equality of proportions with continuity correction.

50 MMED compared to our study. In a retrospective study, Smalley et al.

[23] utilized 4-tiered EHR interventions that consisted of deleting clinician preference lists, defaulting dose, frequency, and quantity, standardizing formulary, and creating dashboards. This multi-faceted approach achieved a large relative reduction (92.9%) of opioid prescrip- tions >=30 MMED across 14 EDs, however our targeted nudge on just the defaults achieved a similar outcome, demonstrating efficacy of the targeted approach.

Implementation in safety net settings requires overcoming unique challenges. Understaffing, and high turnover rates are described in nearly all implementation efforts in this setting, which was further compounded by the COVID-19 pandemic [27]. Additionally, there is self-siloing, or a culture that is resistant to adopting external innova- tions and change [27]. The underserved patient population also presents complex concerns, including pain control and opioid use disorder [24].

While comprehensive, multifaceted approaches such as a combination of education, risk assessment and monitoring, PDMPs, and clinical path- ways are effective strategies to improve physician opioid prescribing behaviors, this may not be practical in resource limited settings such as ours [28-31].. This study adds to the body of literature that focuses on reducing overuse with low-effort, high impact interventions in safety net settings.

Surprisingly, low variability was seen when comparing the individ- ual hospitals. The relative reduction ranged from 88.2% to 99.0%. Similarly, there was low variability of the relative reduction among clinician types – ranging from 95.6% by attendings/fellows and 100% by nurse practitioners. Previous studies have shown high variability in their primary outcomes in the same setting: one intervention led to a 59.2% relative increase of the outcome measure at one hospital and a 58% relative reduction of the same outcome measure at another hospital

Fig. 2. Linear regression analysis of the frequency of Oxycodone-Acetaminophen 5-325 mg orders exceeding 50 MMED across 14 EDs.

Table 5

Pre- and post-intervention comparison of oxycodone-acetaminophen 5-325 mg orders exceeding 50 MMED stratified by clinician type.

Above50MMED/Total % Total Above50MMED

% Difference

Clinician Type	Pre	Post		Pre	Post		Absolute (Post- Pre)	Relative (Post % Pre)	p-value
System Wide	1624/3530	52/3165		46.0%	1.6%		-44.4%	-96.4%	< 0.001
Attending Physician/Fellow	1007/2057	39/1799		49.0%	2.2%		-46.8%	-95.6%	< 0.001
Nurse Practitioner	16/82	0/83		19.5%	0.0%		-19.5%	-100.0%	< 0.001
Physician Assistant	187/489	3/476		38.2%	0.6%		-37.6%	-98.4%	< 0.001
Resident Physician	414/902	10/807		45.9%	1.2%		-44.7%	-97.3%	< 0.001

in the same system. Such patterns can be partially attributed to the differences in the culture across sites and clinician types [32-37]. The unique finding in this study may be explained by the comprehensive education and awareness campaigns stemming from the national opi- oid epidemic, combined with the seamless fit of the intervention that makes it easier for physicians to prescribe.

There were several limitations in this study. First, this quality im- provement initiative was not randomized and lacked a control group, thus, we are not able to say our intervention directly caused the decrease in MMED. Second, this study did not include chart reviews for appropriateness of prescribed opioids. Third, because our study did not follow patients after the ED visit, we do not have information on whether prescriptions were filled or whether the medications were taken as prescribed. Fourth, we observed an increase in minimum num- ber of days prescribed, but it was consistent with our default changes and remained under 3 days, accordant with the CDC and ED guidelines [5,8]. However, the statistically significant reduction in “total quantity” and “total MME” cannot be solely attributed to our intervention, as fac- tors such as patient preferences and prescriber practices could have contributed to the observed changes. Finally, the balance measure used in this study is a surrogate marker and did not include follow-up visits to ambulatory centers or re-prescriptions from other clinicians. The 7-day revisits to the ED in our health system may not accurately in- dicate patient-oriented outcomes, such as pain, functioning, or quality of life.

This initiative successfully reduced opioid prescribing over >50 MMED across 11 EDs in the safety net setting. Variation among sites and clinician types were low. Future study is needed in assessing appro- priateness and patient follow up.

Funding

None.

Declaration of Competing Interest

None.

References

1. Mattson CL, Tanz LJ, Quinn K, et al. Trends and geographic patterns in drug and synthetic opioid overdose deaths – United States, 2013-2019. MMWR Morb Mortal Wkly Rep. 2021;70(6):202-7. Published 2021 Feb 12. https://doi.org/10.15585/ mmwr.mm7006a4.
2. Strang J, Volkow ND, Degenhardt L, et al. Opioid use disorder. Nat Rev Dis Primers. 2020;6(1):3. Published 2020 Jan 9. https://doi.org/10.1038/s41572-019-0137-5.
3. Schranz AJ, Barrett J, Hurt CB, Malvestutto C, Miller WC. Challenges facing a rural opioid epidemic: treatment and prevention of HIV and hepatitis C. Curr HIV/AIDS Rep. 2018;15(3):245-54. https://doi.org/10.1007/s11904-018-0393-0.
4. Jansson LM, Patrick SW. Neonatal abstinence syndrome. Pediatr Clin N Am. 2019;66 (2):353-67. https://doi.org/10.1016/j.pcl.2018.12.006.
5. Centers for Disease Control and Prevention. Drug Overdose. Available from: https:// www.cdc.gov/drugoverdose/deaths/index.html; 2022. Accessed March 27, 2023.
6. Kolodny A, Courtwright DT, Hwang CS, et al. The prescription opioid and heroin cri- sis: a public health approach to an epidemic of addiction. Annu Rev Public Health. 2015;36:559-74. https://doi.org/10.1146/annurev-publhealth-031914-122957.
7. Dowell D, Ragan KR, Jones CM, Baldwin GT, Chou R. CDC clinical practice guideline for prescribing opioids for pain – United States, 2022. MMWR Recomm Rep. 2022; 71(3):1-95. Published 2022 Nov 4. https://doi.org/10.15585/mmwr.rr7103a1.
8. New York City Department of Health. Opioid Prescribing Resources for Emergency Departments. Available from: https://www.nyc.gov/site/doh/providers/health- topics/opioid-prescribing-resources-for-emergency-departments.page; 2013.

Accessed March 27, 2023.

1. Lin EJD, Schroeder M, Huang Y, et al. Digital health for the opioid crisis: A historical analysis of NIH funding from 2013 to 2017. In: Linwood SL, editor. Digital Health [In- ternet]. Brisbane (AU): Exon Publications; 2022 Apr 29 https://doi.org/10.36255/ exon-publications-digital-health-opioid-crisis. Chapter 3. Available from: https:// www-ncbi-nlm-nih-gov.eresources.mssm.edu/books/NBK580625/.
2. Gazelka HM, Clements CM, Cunningham JL, et al. An institutional approach to man- aging the opioid crisis. Mayo Clin Proc. 2020;95(5):968-81. https://doi.org/10.1016/ j.mayocp.2019.11.019.
3. Rhodes E, Wilson M, Robinson A, Hayden JA, Asbridge M. The effectiveness of prescription drug monitoring programs at reducing opioid-related harms and con- sequences: a systematic review. BMC Health Serv Res. 2019;19(1):784. Published 2019 Nov 1. https://doi.org/10.1186/s12913-019-4642-8.
4. Bachhuber MA, Nash D, Southern WN, et al. Effect of changing electronic health re- cord opioid analgesic dispense quantity defaults on the quantity prescribed: A clus- ter randomized clinical trial. JAMA Netw Open. 2021;4(4):e217481. Published 2021 Apr 1. https://doi.org/10.1001/jamanetworkopen.2021.7481.
5. Montoy JCC, Coralic Z, Herring AA, Clattenburg EJ, Raven MC. Association of Default Electronic Medical Record Settings with Health Care Professional Patterns of opioid prescribing in emergency departments: a randomized quality improvement study. JAMA Intern Med. 2020;180(4):487-93. https://doi.org/10.1001/jamainternmed. 2019.6544.
6. Santistevan JR, Sharp BR, Hamedani AG, Fruhan S, Lee AW, Patterson BW. By default: the effect of prepopulated prescription quantities on opioid prescribing in the emer- gency department. West J Emerg Med. 2018;19(2):392-7. https://doi.org/10.5811/ westjem.2017.10.33798.
7. Zwank MD, Kennedy SM, Stuck LH, Gordon BD. Removing default dispense quantity from opioid prescriptions in the electronic medical record. Am J Emerg Med. 2017; 35(10):1567-9. https://doi.org/10.1016/j.ajem.2017.04.002.
8. Villwock JA, Villwock MR, New J, Ator GA. EMR quantity autopopulation removal on hospital discharge prescribing patterns: implications for opioid stewardship. J Clin Pharm Ther. 2020;45(1):160-8. https://doi.org/10.1111/jcpt.13049.
9. Delgado MK, Shofer FS, Patel MS, et al. Association between electronic medical re- cord implementation of default opioid prescription quantities and prescribing be- havior in two emergency departments. J Gen Intern Med. 2018;33(4):409-11. https://doi.org/10.1007/s11606-017-4286-5.
10. Chiu AS, Jean RA, Hoag JR, Freedman-Weiss M, Healy JM, Pei KY. Association of Low- ering Default Pill Counts in electronic medical record systems with postoperative opioid prescribing. JAMA Surg. 2018;153(11):1012-9. https://doi.org/10.1001/ jamasurg.2018.2083.
11. Beauchamp GA, Rosentel J, Yazdanyar A, et al. Implementation of an emergency department discharge opioid taper protocol. Am J Emerg Med. 2021;41:247-50. https://doi.org/10.1016/j.ajem.2020.05.102.
12. Carlson A, Nelson ME, Patel H. Longitudinal impact of a pre-populated default quan- tity on emergency department opioid prescriptions. J Am Coll Emerg Physicians Open. 2020;2(1):e12337. Published 2020 Dec 19. https://doi.org/10.1002/ emp2.12337.
13. Schwartz GD, Harding AM, Donaldson SR, Greene SL. Modifying emergency depart- ment electronic prescribing for outpatient Opioid analgesia. Emerg Med Austral. 2019;31(3):417-22. https://doi.org/10.1111/1742-6723.13192.
14. Bursua A, Mudreac A, Koppen L, Larson C, Park YS, Sreedhar R. Effect of default order standardization on opioid prescribing patterns. Jt Comm J Qual Patient Saf. 2021;47

(7):431-7. https://doi.org/10.1016/j.jcjq.2021.03.005.

1. Smalley CM, Willner MA, Muir MR, et al. Electronic medical record-based interven- tions to encourage opioid prescribing best practices in the emergency department. Am J Emerg Med. 2020;38(8):1647-51. https://doi.org/10.1016/j.ajem.2019. 158500.
2. Hurstak EE, Kushel M, Chang J, et al. The risks of opioid treatment: perspectives of primary care practitioners and patients from safety-net clinics. Subst Abus. 2017; 38(2):213-21. https://doi.org/10.1080/08897077.2017.1296524.
3. Chou R, Hartung D, Turner J, et al. Opioid treatments for chronic pain. Rockville (MD): Agency for Healthcare Research and Quality (US); April 2020.
4. Coyle DT, Pratt CY, Ocran-Appiah J, Secora A, Kornegay C, Staffa J. Opioid Analgesic dose and the risk of misuse, overdose, and death: a narrative review.

Pharmacoepidemiol Drug Saf. 2018;27(5):464-72. https://doi.org/10.1002/pds. 4366.

1. Lyles CR, Handley MA, Ackerman SL, et al. Innovative implementation studies con- ducted in US safety net health care settings: a systematic review. Am J Med Qual. 2019;34(3):293-306. https://doi.org/10.1177/1062860618798469.
2. Sud A, Molska GR, Salamanca-Buentello F. Evaluations of continuing health provider education focused on opioid prescribing: a scoping review. Acad Med. 2022;97(2): 286-99. https://doi.org/10.1097/ACM.0000000000004186.
3. Webster LR, Grabois M. Current regulations related to opioid prescribing. PM R. 2015;7(11 Suppl):S236-47. https://doi.org/10.1016/j.pmrj.2015.08.011.
4. Puac-Polanco V, Chihuri S, Fink DS, Cerda M, Keyes KM, Li G. Prescription drug mon- itoring programs and prescription opioid-related outcomes in the United States. Epidemiol Rev. 2020;42(1):134-53. https://doi.org/10.1093/epirev/mxaa002.
5. Howard R, Vu J, Lee J, Brummett C, Englesbe M, Waljee J. A pathway for developing postoperative opioid prescribing best practices. Ann Surg. 2020;271(1):86-93. https://doi.org/10.1097/SLA.0000000000003434.
6. Krouss M, Israilov S, Alaiev D, et al. Free the T3: implementation of best practice ad- visory to reduce unnecessary orders. Am J Med. 2022;135(12):1437-42. https://doi. org/10.1016/j.amjmed.2022.07.018.
7. Shin D, Krouss M, Alaiev D, et al. Reducing unnecessary routine laboratory testing for noncritically ill patients with COVID-19. J Hosp Med. 2022;17(12):961-6. https:// doi.org/10.1002/jhm.12993.
8. Krouss M, Israilov S, Alaiev D, et al. SEE the DIFFerence: Reducing unnecessary

C. difficile orders through Clinical decision support in a large, urban safety-net sys- tem [published online ahead of print, 2022 Nov 10]. Am J Infect Control. 2022. https://doi.org/10.1016/j.ajic.2022.11.003. S0196-6553(22)00783-0.

1. Krouss M, Israilov S, Alaiev D, et al. Tell-a provider about tele: reducing overuse of telemetry across 10 hospitals in a safety net system. J Hosp Med. 2023;18(2): 147-53. https://doi.org/10.1002/jhm.13030.
2. Krouss M, Alaiev D, Shin DW, et al. choosing wisely initiative for reducing urine cul- tures for asymptomatic bacteriuria and catheter-associated asymptomatic bacteri- uria in an 11-hospital safety net system [published online ahead of print, 2023 Jan 20]. Am J Infect Control. 2023. https://doi.org/10.1016/j.ajic.2023.01.005. S0196-6553(23)00006-8.
3. Krouss M, Tsega S, Alaiev D, et al. Reduction of intravenous antihypertensives through clinical decision support in a large safety net system [published online ahead of print, 2023 march 8]. Jt Comm J Qual Patient Saf. 2023. https://doi.org/ 10.1016/j.jcjq.2023.03.001.