a b s t r a c t

Objectives: We sought to compare National Institutes of Health (NIH) funding received by Emergency Medicine (EM) to the specialties of Family Medicine, Neurology, Orthopedics, Pediatrics and Psychiatry over the 10-year period from 2008 to 2017.

Methods: The NIH database of both submitted and funded NIH applications were queried and crossed with the departmental affiliation of the principal investigator. Research Grants were defined by the fol- lowing activity codes: R, P, M, S, K, U (excluding UC6), DP1, DP2, DP3, DP4, DP5, D42 and G12. Derived data were further analyzed using information from the Association of American Medical Colleges to determine the relationship between the number of awards and the size of respective teaching and research faculty.

Results: From 2008 to 2017, there were a total of 14,676 funded grants across included specialties with total monetary support of $6.002 billion. Of these funded grants, 250 (1.7%) were from EM principal investigators which corresponded to total support of $89,453,635 (1.5% of overall dollars). There was an increase in total support after 2012 in EM, however when compared to the other specialties, EM inves- tigators submitted relatively fewer grants and awarded grants were funded by a wider distribution of NIH Institutes and Centers (ICs).

Conclusions: Compared to other select specialties, EM investigators accounted for a small proportion of grants submitted and funded over the past decade. Though findings illustrate promising trends, to foster success, more submitted grant applications are needed from within EM along with systematic approaches to support faculty members in their pursuit of NIH funding.

(C) 2018

Introduction

In 2003, the National Academies of Science Engineering and Medicine (NASEM) convened to evaluate the state of emergency care systems [1-3], and a key recommendation was to ”examine the gaps and opportunities in emergency and trauma care research.. .” One of the recognizable issues facing Emergency Med- icine (EM) is the relative underfunding in research. When com- pared to many other specialties, Emergency Medicine received fewer total National Institutes of Health (NIH) awards and funding dollars overall and per active faculty [4]. In response to the NASEM reports, the NIH formed a NIH Task Force on Research in Emer- gency Medicine to address the challenges as a growing research

* Corresponding author at: John Morgan Building, 3620 Hamilton Walk, Philadel- phia, PA 19104, United States of America.

E-mail address: David.Jang@Uphs.upenn.edu (D.H. Jang).

specialty. There were 3 roundtable discussions to identify chal- lenges facing emergency care research with strong recommenda- tions to identify these barriers and develop potential solutions [5-7]. To better address these barriers and to also help improve health outcomes, a new Office of Emergency Care Research (OECR) was created in 2012 within NIH to serve as the focal point for emergency care research and training across NIH.

Some of the leading challenges facing emergency care research include the lack of EM-specific senior investigators with indepen- dent funding, the absence of a dedicated institute at the NIH and the lack of training mechanisms for EM trainees. A one-year cross-sectional study reported about 40 EM faculty with (R) or

(U) funding where both the absolute and proportionally (grants per board certified physicians) number of grants were smaller when compared to other specialties such as internal medicine [4]. This may be related in part to the relatively small number of

https://doi.org/10.1016/j.ajem.2018.12.045

0735-6757/(C) 2018

EM applicants who have successfully secured career development awards who often have mentors outside the specialty of EM [8].

Despite these recognized challenges, there have been changes over the past decade to promote emergency care research. In 2010, the NIH National Heart, Lung and Blood Institute (NHLBI) funded six institutional (K12) research career development pro- grams specifically to address the lack of few federally-funded EM investigators, and this program was renewed in 2016. Other critical developments also include funding various ED based networks including the Neurological Emergency Treatment Trials (NETT) and The Strategies to Innovate Emergency Care Clinical Trials Network (SIREN).

To better understand how NIH funding for EM researchers was progressed over a ten-year period, we sought to describe NIH research grant funding amounts and success rates for principal investigators from EM and five comparator specialties over the past 10 years. We analyzed non-public NIH grant application data.

Methods

Study design and data source

We performed a retrospective analysis of the federal Query/ View/Report (QVR) database. The QVR database is designed for use across the department of Health and Human Services, and access is limited to staff with log-in credentials. Unlike the publicly available RePORT database that describes NIH supported grants, QVR includes data about all submitted applications, including those that were not funded. Analysis was limited to grants submit- ted to the NIH, and did not include those supported by the Agency for Healthcare Research and Quality (AHRQ).

The QVR database was searched from 2008 to 2017 for all sub- missions in which the PI was from one of the following six spe- cialties: EM, Family Medicine (FM), Neurology, Orthopedics, Pediatrics, and Psychiatry. The five comparator specialties were chosen based on [1] similar numbers of board-certified practition- ers, and [2] relatively homogenous clinical content (e.g. in contrast to internal medicine or surgery, which many have multiple divi- sions). Three of the comparator specialties have corresponding NIH Institutes (Neurology- National Institute of Neurological Disorders and Stroke; Pediatrics- National Institute of Child Health and Development; Psychiatry- National Institute of Mental Health). Similar to EM, FM and Orthopedics do not have a corresponding NIH institute.

Duplicate submissions or duplicate funded applications were removed. We excluded those applications that were withdrawn by an applicant prior to review, or returned or administratively withdrawn by either the NIH Center for Scientific Review, or an NIH institute or center, and not peer reviewed by an Initial Review Group. This study does not meet the definition of human subjects’ research and was exempt from NIH IRB review and permission was given for publication of this study.

Data measures and analyses

We followed the official NIH reporting definitions, and analyzed grants that are reported as research grants (rather than research project grants). Research grants are defined as extramural awards made for Research Centers, Research Projects, Small Business Inno- vation Research/Small Business Technology Transfer (SBIR/STTR) Grants, and Other Research Grants. Research Grants are defined by the following activity codes: R, P, M, S, K, U (excluding UC6), DP1, DP2, DP3, DP4, DP5, D42 and G12. Research projects were first coded to NLM in fiscal year 2007. The RL5 activity was formerly

classified as a Research Project Grant but was reclassified as Other Research in fiscal year 2015. The P42 activity was formerly classi- fied as a Research Project Grant but was reclassified as Research Centers in fiscal year 2017. Not all of these activities may be in use by NIH every year.

Data are presented as summary statistics, using either percent- age or frequency, as applicable. To determine differences in distri- bution of funded vs submitted grants for EM compared to the other 5 specialties, as well as by type of grant funded, the chi-square test was used. To examine trends over time, number of funded grants subsequent to 2008 were standardized to 2008 by calculating the percent changed from 2008. Using the standardized values, a 2- factor analysis of variance (year & specialty) was performed. To adjust for multiple post-hoc pairwise comparisons, Tukey-Kramer tests were used. To examine average change over the 10-year per- iod by grant mechanism, the 2-sample t-test was used. All analyses were performed using SAS statistical software (version 9.4, SAS Institute, Cary NC).

Results

From 2008 to 2017, there were a total of 99,273 grants sub- mitted to NIH for the six included specialties. Of these, 7650 (8%) were grants types not included in the NIH definition of a research grant. This left 91,623 research grants for analysis, of which 58,699 (64%) were new applications. Of these new applica- tions, 14,676 (25%) were funded. Of funded grants, 69% of grants and total support were awarded to Pediatrics and Psychiatry (Table 1). In contrast, EM was the lowest for funded grants (1.7%) and total support (1.5%). Additionally, EM had the second-lowest amount of funding support per active physician ($2261/active physician). However, of grants submitted by each specialty, percent funded in EM was similar to other specialties (EM 22%, range for all specialties: 20%-28%). The total financial support across ten years was $6,002,750,255 for all grant types. The total amount of financial support for EM was $89,493,635 or 1.5% of the total support when compared to Pediatrics and Psy- chiatry with a total support of $ 1,993,243,097 and $ 2,148,292,897 or 33% and 36%, respectively. The top two NIH Institutes and Centers (IC) supporters for EM was NHLBI (28%) and NINDS (20%) accounting for 48% of all money support for EM, whereas other NIH funded grants by institution for EM showed distribution of ICs similar to FM when compared to Neu- rology and Psychiatry which had a majority of their funded grants through NINDS and NIMH, respectively (data not shown).

The top four grant mechanisms common to EM and all other

specialties were R01s, R21s, U01s and K23 career development awards (Table 2). Compared to other specialties, EM was similar with regard to R01s (30% vs 31%) but significantly higher for K23s (15% vs 3%) and K12s (4% vs 0.4%, p < 0.0001).

Compared to the other five specialties, EM total number of funded grants was the lowest number over the 10-year period (Fig. 1a). However, EM (87%) demonstrated the highest average increase in funding over the 9 year period since 2008 compared to Orthopedics (65%, p = 0.53), Neurology (22%, p = 0.002), Psychi-

atry (–3.6%, p < 0.0001), Pediatrics (15%, p < 0.001), and FM

(–5.9%, p < 0.0001) (Fig. 1b).

When examining the average change over the 9 years since 2008 for six of the major funding mechanisms for all specialties, R01s, R21s, K23s were relatively flat (average of a 4%, 4%, & 5% change, respectively), declined for K08 (9%) and R03 (22%) as well as a decline of 22% for U01 since 2014 (Fig. 2a). Conversely for EM, there was a 73% increase in R01s (p = 0.006 compared to all spe- cialties) and for K23 an increase since 2012 (40% vs –1% for all spe- cialties, p = 0.04).

Table 1

Total support by specialty and number of active physicians from 2008 to 2017.

^*Source: AAMC 2016 Physician Specialty Data Report.

Specialty	Funded grants 2008-2017 N %			Total support Dollars $	%		All grants submitted 2008-2017 N % funded in specialty		Active physicians*	Size compared with EM	Support ($) per active physician
Emergency Medicine	250	1.70%	89,493,635		1.5%	1129		22.1%	39,579	1.0	2261
Family Medicine	549	3.74%	173,233,133		2.9%	2652		20.7%	111,295	2.8	1557
Orthopedics	569	3.88%	159,467,133		2.7%	2845		20.0%	19,145	0.5	8329
Neurology	3158	21.52%	1,439,020,360		24.0%	11,405		27.7%	13,392	0.3	107,454
Pediatrics	4669	31.81%	1,993,243,097		33.2%	19,183		24.3%	57,543	1.5	34,639
Psychiatry	5481	37.35%	2,148,292,897		35.8%	21,485		25.5%	37,736	1.0	56,930
Grand total	14,676	100%	6,002,750,255		100%	58,699		25.0%

Discussion

This study describes the total number of NIH grants and funding dollars by total active physicians for six medical specialties using the QVR database that describes both funded and unfunded grants over a ten-year period. Our study quantifies NIH grants using the QVR database, providing a very comprehensive examination of NIH funding in EM when compared to other specialties. This differs from a prior paper using QVR that looks at emergency care research that was defined by NIH algorithm of content type, included grants by non-EM PI potentially missing grants by EM PIs, rather than funded emergency medicine PI research [9]. Con- sistent with previous studies, we determined that over the ten- year period, EM ranked the lowest in total NIH funded grants and total support per active physician. As a specialty, EM repre- sents a small percent of NIH research grant funding with success rates compared to other specialties varying by mechanism. While there is an increase in number of funded grants and support in EM, the overall number is much less when compared to psychiatry, neurology and pediatrics.

The NIH is the single largest source of research funding in the United States constituting a critical element in the development and maintenance of the research workforce [10,11]. It was recog- nized in the early 2000s there were many barriers and challenges facing EM as a growing research specialty. Subsequent reports have pointed to the fact that EM has consistently ranked among the low- est in terms of funded awards and support when compared to many other specialties [4]. There are many Potential causes as to why EM receives fewer grants and support in comparison to the other listed specialties. Some causes discussed in prior work include the lack of EM faculty with (R) or (U) level of funding; financial incentive for clinical work, and the lack of a research cul- ture at many academic institutions [8]. It is clear that the primary

Table 2

Ten most common NIH grant mechanism for all specialties and emergency medicine from 2008 to 2017.

Grant type All specialties Emergency medicine N Percent N Percent

R01	6683	31.4%	83	29.6%
R21	1914	9.0%	20	7.1%
U01	747	3.5%	22	7.9%
K23	617	2.9%	43	15.4%
R03	505	2.4%	10	3.6%
K08	395	1.9%	7	2.5%
U10	312	1.5%	12	4.3%
K01	291	1.4%	9	3.2%
R34	249	1.2%	6	2.1%
P50	238	1.1%	0	0.0%
K12	83	0.4%	11	3.9%
Other	9240	43.4%	57	20.4%
Total	21,274		280

issue facing EM research is that as a specialty, EM submits fewer applications.

Perhaps one of the most important barriers is the lack of poten- tial young investigators in the ”pipeline” for potential independent funding. One of the recommendations was to improve both the recruitment and training opportunities of young investigators in emergency care research [8]. There are various post-residency research training opportunities that include institutional T32s and K12s available for growing physician scientists, however, it is noted that applications to these programs may be at a low rate in the field of EM [8].

One of the corner stepping stones in becoming an independent investigator is obtaining an individual mentored career develop- ment award from NIH. Also known as ”K awards,” the goal of these 3-5 year awards is to provide both protected time and a modest research budget under the guidance of an established mentor or mentorship team to develop the required research skills to apply for independent funding such as an R01 [12]. There are three men- tored career development awards that include the K01, K08 and K23 mechanisms. Based on our study, the K23 mechanism is the avenue of funding for young EM investigators who obtain a K award While a (K) award provides many benefits to the individual PI, obtaining a (K) award also has other benefits in terms of obtain- ing future funding. One NIH report concluded that previous K awardees (Prior K01, K08 and K23) not only were more likely to apply for subsequent NIH awards but they had a significant higher R01 award success rate compared to individuals with no prior K awards [13].

Despite these benefits of obtaining a (K) award, this has not translated to increased K grantees in EM. One study examined the number of (K) awards obtained by individuals in EM over six years in comparison to six other specialties. While the success rate for EM investigators obtaining (K) awards was comparable, there were significantly fewer applications for (K) awards submitted which was the second lowest of the seven specialties examined. In our study we found similar success rates but a low number of funded K08 and K23 grants in EM [9]. In response to the prior study, there was an article describing solutions to the low number of EM applications for (K) awards. The various proposed solutions include availability of mentors and applicants, creation of a culture change where research is valued as highly as the clinical and edu- cational mission of academic institutions and finally increase awareness of young EM investigators to various post-residency training mechanisms such as T32s [8].

In recognition that there are very few federally funded EM investigators, NHLBI funded a total of 6 institutional (K12) research career development programs focused in emergency care research. This five-year commitment represented an investment of over 20 million dollars for investigators to obtain advanced research degrees with the goal to secure an individual K awards or indepen- dent NIH research funding. This program ended in 2016 and

Fig. 1. a. Illustrates the number of grants funded per year by specialty from 2008 to 2017. b. Illustrate percent change in funded grants from 2008 to 2017.

multiple individual K awards were obtained by trainees accepted into these K12 program. Specifically, 40 out of the 43 K12 scholars submitted a career development award or research project grant application during the K12 program. As of August 2017, 26 K12 scholars have secured independent grant funding and nineteen of the 43 scholars have received a career development award with 8 have received a federal research project grant [14]. This is sup- ported by our data as seen in Fig. 2b where there is an upward

trend in both K23 and R01 awarded which may be attributable in part to NHLBI K12s and formation of the OECR. While this may be a possible factor for the increased trend, it is important to note there were other EM-specific initiatives that may have also contributed such as SIREN and NETT as these were special oppor- tunities that received many applications. Based on this success, a new Trans NIH K12 Program in Emergency Care Research was funded in late July 2016 by NHLBI, NIMH and NINR. While these

Fig. 2. a. Illustrate the number of common funded grant types for all specialties. b. Illustrate number of funded grant types for Emergency Medicine.

new initiatives represent a commitment to the growth of research in EM, it is too early to evaluate the Long-term effects on EM research investigators successfully competing for Federal funding. There have also been other initiations from EM societies such as the Society for Academic Emergency Medicine (SAEM). SAEM recently developed the Advanced Research Methodology Evalua- tion and Design (ARMED). ARMED is targeted to assist junior fac- ulty who are interested in pursuing a career in research to teach basic research skills and provide networking opportunities with successful EM researchers. The American College of Emergency Physicians has developed the Emergency Medicine Basic Research Skills (EMBRS) Workshop. The purpose of EMBRS is also to better train potential young EM investigators with the combination of a broad overview of research methodology and grant funding to sup-

port EM-focused research.

Another interesting finding in our study was the distribution of funded grants in EM. The scope of emergency care research is broad and encompasses a diverse patient population such as acute care illnesses and various chronic diseases. While this diversity can be considered a strength, it remains the case that EM does not have a specific IC at NIH unlike Neurology (NINDS) or Psychiatry (NIMH). Not only does EM have a substantially lower number of funded grants but is also spread across several ICs. This can be seen as a disadvantage in competing for grants with another specialty that has a specific IC as these specialties may have research agen- das more aligned with that specific IC. Another possible explana- tion for the limited number of funded grants, in particular, with

(K) awards is the emphasis that NIH places on basic science research. At least with (K) awards, in our study there were only 6 funded K08 applications in EM when compared to 39 funded K23 awards over our study period. This finding was similar to the pre- vious study that examined individual mentored career develop- ment grants in EM [9].

Limitations

There are a few limitations to consider in the interpretation of this study. The first limitation is that this study only examines NIH-funded grants. We did not study other funding sources that provide potential support for research grants and training awards such as Agency for Healthcare Research and Quality and the United States Department of Defense. Another limitation is the choice of comparison specialties to EM. Prior studies have used different specialties to highlight the differences in awarded funds [4,9]. In our study we highlight how some specialties (like Neurology, Orthopedics and Psychiatry) that have a natural home or IC from where most of their support is found. FM is like EM and has no pri- mary home, and but its largest support comes from NCI support. The outlier is Pediatrics, which only gets 18% of its support from NICHD. Another limitation is differences within each specialty of basic versus clinical research as well as PhD versus MD researchers although it is not clear the importance this has on obtaining federal funding. Finally, we did not account for change in departmental status for EM as they may have been a division under another department like medicine or surgery during this time period that could impact results. It is possible an EM investigator may have a

primary appointment in another department or a duel appoint- ment in which instance it is not clear which department would get the credit.

Conclusions

Among the six specialties reviewed over ten years, EM had the lowest number of awarded grants and total support. Unlike some of the other compared specialties, funded grants for EM were spread across different ICs illustrating the challenges that emer- gency care does not have its own IC. Our study illustrates that improving the training mechanism and mentoring within EM may be required so that more high quality and impactful grants are submitted and funded.

Funding

1. K08HL136858 from the National Heart, Lung, and Blood Institute (DJ).

The views expressed in this manuscript are those of the authors and do not necessarily reflect the official views of the National Institutes of Health or the Department of Health and Human Services.

References

1. Institute Of M. IOM report: the future of emergency care in the United States health system. Acad Emerg Med 2006;13(10):1081-5.
2. Spaite DW. The future of emergency care in the United States: the Institute of Medicine Subcommittee on Prehospital Emergency Medical Services. Ann Emerg Med 2006;48(2):126-30.
3. States CotFoECitU, System. H. Future of emergency care–emergency care for children, growing pains, Washington, DC: National Academies Press; 2007.
4. Bessman SC, Agada NO, Ding R, Chiang W, Bernstein SL, McCarthy ML. Comparing National Institutes of Health funding of emergency medicine to four medical specialties. Acad Emerg Med 2011;18(9):1001-4.
5. Cairns CB, Maier RV, Adeoye O, et al. NIH roundtable on emergency trauma research. Ann Emerg Med 2010;56(5):538-50.
6. D’Onofrio G, Jauch E, Jagoda A, et al. NIH roundtable on opportunities to advance research on neurologic and psychiatric emergencies. Ann Emerg Med 2010;56(5):551-64.
7. Kaji AH, Lewis RJ, Beavers-May T, et al. Summary of NIH medical-surgical emergency research roundtable held on April 30 to May 1, 2009. Ann Emerg Med 2010;56(5):522-37.
8. Ranney ML, Limkakeng Jr AT, Carr B, Zink B, Kaji AH, Committees ASR. Improving the emergency care research investigator pipeline: SAEM/ACEP recommendations. Acad Emerg Med 2015;22(7):849-51.
9. Brown J. National Institutes of Health support for individual mentored career development grants in emergency medicine. Acad Emerg Med 2014;21 (11):1269-73.
10. Dorsey ER, de Roulet J, Thompson JP, et al. Funding of US biomedical research, 2003-2008. JAMA 2010;303(2):137-43.
11. Hensley P, LaMattina JL. A plan for sustainable funding for US biomedical research. Nat Rev Drug Discov 2015;14(2):147.
12. National Institutes of Health. NIH career development (K) grants. reportnihgov/ displayreportaspx?rid=551. Accessed Aug 8, 2018.
13. National Institutes of Health. Individual mentored career development awards program Available at: http://grantsnihgov/training/K_Awards_Evaluation_ FinalReport_20110901pdf.
14. Newgard CD, Morris CD, Smith L, et al. The first national institutes of health institutional training program in emergency care research: productivity and outcomes. Ann Emerg Med 2018;72(6):679-90. https://doi.org/10.1016/j. annemergmed.2018.06.018.