a b s t r a c t

Objective: The extent of intervention reporting in emergency medicine journals remains unclear. The primary ob- jective is to assess overall completion of the Template for intervention description and Replication (TIDieR) checklist described in emergency medicine randomized clinical trials (RCTs). The secondary outcomes were to

(1) compare reporting before and after TIDieR publication; (2) evaluate factors associated with intervention reporting.

Methods: Our cross-sectional study used Google Scholar‘s metrics to identify seven emergency medicine journals; of which, we randomly sampled 300 articles. Using two PubMed searches, we extracted 150 RCTs before and after publications of TIDieR. Two investigators independently extracted data. The primary analysis to measure overall completion included descriptive statistics for each checklist item. Our secondary analysis used an interrupted Time series analysis and generalized estimating equations to determine the effect of TIDieR publica- tion on intervention reporting.

Results: Our initial search yielded 635 articles; from which, we randomly sampled 300 articles. We excluded 67 articles, leaving 233 for analysis. The mean number of TIDieR items reported was 5.4 (standard deviation = 1.18). Of the 233 trials, 42.9% provided information about materials, 67% provided intervention procedures, and 99.1% provided intervention delivery. The least reported items were intervention modifications (2.6%), intervention ad- herence assessment methods (3.4%), and intervention adherence assessment outcomes (2.2%).

Conclusions: The completeness of intervention reporting is suboptimal in emergency medicine journals, necessi- tating improvement. The current state of adherence could be improved through the combined efforts of journal editors, major editorial organizations, and authors.

(C) 2020

Introduction

Randomized clinical trials (RCTs) are the gold standard for evidence- based medicine and guide clinical practice. For example, when creating

20 current clinical policies for emergency medicine practice, the American College of Emergency Physicians (ACEP) considered RCTs as Level 1 evidence, the strongest form of supporting evidence. However, several studies have shown that the majority of clinical policies are not created with Level 1 evidence. For example, Venkatesh et al. found that of 19 ACEP clinical policies, only 8% used Level 1 evidence, 32% used Level 2 evidence, and 60% used Level 3 evidence [1]. Despite the in- herent rigor of RCTs, these studies may still be susceptible to bias [2,3]. Additionally, studies have found that poor methodological reporting

* Corresponding author at: Oklahoma State University Center for Health Sciences, 1111 W 17th St., Tulsa, OK 74107, USA.

E-mail address: [email protected] (I.A. Fladie).

increases the likelihood of inappropriate interpretation of results [4]. Given the benefits and limitations of RCTs in the development of clinical practice guidelines, more robust reporting guidelines would benefit the field of emergency medicine.

Since the establishment of the Consolidated Standards of Reporting Trials (CONSORT), many RCTs have improved the completeness of their reporting [5]; however, trial reporting still has limitations. For in- stance, to help mitigate bias in RCTs, CONSORT requires trial registration [6,7]. However, we previously investigated the author instructions for 27 emergency medicine journals and found that more than half did not mention clinical trial registration, and only 55.6% recommended the use of the CONSORT [8]. Evidence suggests that adherence to CONSORT varies, with particular checklist items in need of improve- ment. Montane et al. found that approximately three-fourths of RCTs met 12 or less of the CONSORT items [9]. Yao et al. reported similar find- ings in ophthalmic surgery journals, with a median CONSORT score of

8.9 out of 23 items, and only 3.1% adherence to experimental treatment

https://doi.org/10.1016/j.ajem.2019.12.030

0735-6757/(C) 2020

reporting [10]. Given the low adherence rate to CONSORT in interven- tion reporting, further investigation is needed to evaluate the extent of transparent, thorough reporting of interventions, which would enhance the study’s reproducibility and enable clinical adoption of the intervention.

Since the establishment of CONSORT, many areas of clinical research have improved; however, limited reporting of clinical interventions has led to the development of the Template for Intervention Description and Replication (TIDieR) checklist. Previous attempts to improve inter- vention reporting have been made in the past [11]. TIDieR checklist was developed as an extension to item 5 of CONSORT through a literature re- view, Delphi survey collaboration with 125 international experts, and panel meetings test the checklist [12]. A recent study by Chauvin et al. found that of 68 simulation-based medical education trials, only 4% re- ported all TIDieR checklist items [13]. Our study focused on emergency medicine literature more broadly. While several limitations in RCT study design have been exposed through the use of CONSORT, little is known about intervention reporting in emergency medicine using TI- DieR. The primary objective of our study is to evaluate the completeness of intervention reporting, by using the TIDieR checklist, in emergency medicine RCTs.

Material and methods

Our cross-sectional study was not subject to institutional review board oversight because it did not meet the definition of human subject research. When considering the definition of human subject research, we referred to the US Department of Health and Human Services’ Code of Federal Regulations policy 45 CFR 46.102(d) and (f) [14]. All study materials, protocols, analysis scripts, and raw data are available on Open Science Framework [15].

Journal inclusion criteria

The following journals were included: Annals of Emergency Medicine, Resuscitation, The Journal of Trauma and Acute Care Surgery, Academic Emergency Medicine, Emergency Medicine Journal, The Journal of Emer- gency Medicine, and The American Journal of Emergency Medicine. These journals were selected based on their ranking in the 2019 Google h5- index. We selected Google Scholar Metrics for journal identification, since the platform is open access and not restricted by a paywall like other journal ranking platforms.

Study selection

Two investigators searched PubMed for RCTs published in emer- gency medicine journals on July 25, 2019. To accomplish our search, we used the Clinical Trial filter (Clinical Trial[ptyp]), searched for humans as a MeSH term (Humans [MeSH Terms]), and limited the returns by publication date. We chose PubMed because it has been used in similar methodological studies and because it is an internation- ally recognized database that includes the MEDLINE collection. We searched PubMed for RCTs from January 1, 2011 to December 31, 2013 and January 1, 2016 to December 31, 2018. These dates were se- lected to compare studies before and after TIDieR publication (March 2014). The exact search strings that were used are included in our pro- tocol. The period between TIDieR publication and the start date of the second search permitted 22 months for TIDieR to take effect. Following the execution of these two searches, we used Microsoft Excel’s random number function to generate a random sample of 150 RCTs from each search, for a total of 300 RCTs to screen. Our original search returns and a detailed list of randomly selected PubMed records are available on Open Science Framework (OSF).

Training

A training module was developed by SJ and MV for all investigators in the study. Each investigator responsible for data extraction partici- pated in the training module prior to study commencement. The train- ing session included an overview of the study, review of Hoffman et al.¹², and a review of all 12 checklist items. Following the training ses- sion, data extraction for one RCT was performed collectively. Results were discussed and the two investigators extracted data from one fur- ther RCT in an independent, blinded fashion. The extracted data were compared, and discrepancies were discussed to calibrate responses. Next, both investigators extracted data from three additional RCTs fol- lowing the same procedure, the results were discussed, and discrepan- cies resolved.

Screening and study inclusion criteria

Two investigators used Rayyan [16], a systematic review application, to screen PubMed records for eligible studies. This task was performed in an independent, blinded manner using the “blind” option in Rayyan. Results were discussed and discrepancies were resolved. The principle investigator was available for third party adjudication but were not needed. A list of inclusion criteria, included articles, and excluded arti- cles can be found on OSF.

Data extraction

The same two investigators who completed the initial Rayyan screening performed data extraction in an independent, blinded man- ner. Both investigators used a pilot-tested Google form for data extrac- tion. The form contained all 12 checklist items as described by Hoffman et al.¹² as well as title of the study, the name of the journal, month and year of publication, type of study, type of intervention, hy- pothesis, type of blinding used, trial location (US or International), funding source, endorsement of CONSORT or TIDieR by authors, data source (primary paper, website, published protocol, previously pub- lished paper, or online supplementary material), and whether the trial was registered (including the registry name and number). Following data extraction, results were discussed, and discrepancies were re- solved. Principal investigator mediation was available but not needed. We chose these variables based on their prior, reported relatedness to reporting completeness using CONSORT guidelines [17-22]. The Google form used for data extraction is included on OSF.

Primary and secondary outcomes

The primary outcome is to assess overall completion of the TIDieR checklist in emergency medicine RCTs [12]. The secondary outcomes were to (1) compare the completeness of intervention reporting before and after TIDieR publication and (2) evaluate factors associated with in- tervention reporting.

Data analysis

For each trial, we assigned a point value to each checklist item in ac- cordance with the completeness of the intervention reporting (as de- fined by the TIDieR checklist). Our point assignments were as follows: one point was assigned for complete compliance with an item, 0.5 was assigned for partial compliance, and 0 was assigned for noncompli- ance. The primary analysis consisted of summarizing each checklist item with descriptive statistics (frequencies and percentages) with 95% confidence intervals [CIs]. We generated a histogram of the distri- bution of the number of completely reported TIDieR items from each study (excluding partially completed items; Fig. 2). The decision to present a histogram of our data was made post hoc. We also summarized

the data for each item to evaluate which items were well reported or underreported.

Using Stata’s itsa command, we conducted an interrupted time se- ries analysis to evaluate whether publication of the TIDieR checklist im- proved intervention reporting in emergency medicine RCTs because this method has been used in previous studies to evaluate the effect of reporting guidelines on the completeness of reporting [23]. For this analysis, the TIDieR checklist completion was evaluated for each RCT. Given that this analysis requires only one data point per time period, we calculated the mean adherence for all trials published in the same month and year. Standard errors were estimated using the Newey West method [24].

For our co-secondary outcome, generalized estimating equations (GEE) were used to evaluate whether trial characteristics were associ- ated with intervention reporting in emergency medicine RCTs [25]. GEE was used to account for potential clustering between trials pub- lished in the same journals because they would adhere to the same ed- itorial policies and reporting guidelines. GEE requires the specification of the type of distribution, link function, and correlation structure. For this analysis, we specified a Poisson distribution, a log link function, and an exchangeable correlation structure. Due to the small sample size of some trial characteristics, we clustered characteristics into groups when appropriate. We reported the results of the GEE analyses as Incidence rate ratios [IRR] with CIs and p-values. This analysis employed Stata’s xtgee command.

For all analyses, Type I error (alpha) was set at 0.05. We used Matlab

[26] to prepare data for final analysis and Stata 15.1 for all analyses. All data used for our analysis can be found on OSF.

Fig. 1. Flow diagram for analyzed Emergency Medicine articles.

Results

Our PubMed search returned 635 records; from which, 150 records each were randomly sampled from the 2011-2013 and 2015-2018 co- horts. After screening the titles and abstracts of the 300 randomly se- lected trials, 45 were excluded because they were not RCTs. Upon reviewing the full-text publications, 22 additional trials were excluded because they were not published within our included time frame (n = 14) and because they were not randomized (n = 8). Our final anal- ysis totaled 233 articles: 107 during 2011-2013 and 126 during

2015-2018 (Fig. 1).

Sample characteristics

The American Journal of Emergency Medicine published the largest percentage of trials (29.6%, 69/233), followed by Resuscitation (24.9%, 58/233). Devices (30.5%, 71/233) and drugs (27.9%, 65/233) were the most common intervention types. Additionally, 90% of trial hypotheses were superiority (210/233), and 68.2% were parallel arm design (159/ 233). Studies failed to report protocol availability in 97% of the studies (226/233), CONSORT adherence in 70.8% of the studies (165/233), and TIDieR adherence in 100% of the studies (233/233). A total of 62.2% of trials had a clinical trial registration number (145/233). Additional sam- ple characteristics can be found in Table 1.

TIDieR checklist characteristics (primary outcome)

The 12 TIDieR checklist items were used to analyze the overall inter- vention reporting of RCTs in our sample (Table 2). Fig. 2 presents the frequency of TIDieR items reported across trials. The mean number of TIDieR items reported was 5.4 (standard deviation = 1.18). All of the 233 trials reported a brief name of the intervention and provided the reasoning for why the intervention was necessary. Less than half of the trials provided information about intervention materials (42.9%, 100/233), intervention procedures (67%, 156/233), or intervention de- livery (99.1%, 231/233). The least reported items were intervention modifications (2.6%, 6/233), intervention adherence assessment methods (3.4%, 8/233), and intervention adherence assessment out- comes (2.2%, 5/233). To see a complete breakdown of responses to the TIDieR checklist, see Supplementary Table 1.

Secondary analyses

Fig. 3 displays the mean percent completion by month for trials pub- lished before and after TIDieR publication. Results suggest that the pub- lication of the TIDieR checklist did not result in an increased percent completion of the TIDieR checklist items (p = .31). Our co-secondary analysis measured variables associated with intervention reporting, with our GEE model identifying three statistically significant character- istics (p b .05). Trials that mentioned CONSORT (IRR = 1.05, 95% CI: 1.01-1.09, p = .005) and trials that were registered (IRR = 1.05, 95% CI: 1.00-1.11, p = .035) were associated with improved TIDieR adher- ence. Trials that used a non-drug or non-device intervention (IRR = 0.97, 95% CI: 0.96-0.99, p = .001) were associated with poorer TIDieR adherence (Table 3).

Discussion

Our analysis of RCTs reveals that TIDieR adherence for intervention reporting in emergency medicine is limited and in need of improve- ment. Approximately 18% of trials met at least seven of the 12 TIDieR checklist items. No trials met all 12 TIDieR checklist items. Given that the majority of the analyzed trials did not meet at least seven of the TI- DieR checklist items, the claim can be made that intervention reporting in emergency medicine is not conforming. Similarly, a recent study found that only 4% of simulation-based studies in emergency medicine

Table 1

Sample characteristics of analyzed Emergency Medicine articles.

Characteristic Variables

No. (%)

Table 1 (continued)

Characteristic Variables

No. (%) 145

Annals of Emergency 36

Was a clinical trial registry number given?

Yes

(62.23)

Medicine

American Journal of Emer-

(15.45)

69

(n = 233)

No 88

(37.77)

gency Medicine

(29.61)

Abbreviations: RCT, randomized control trial, CONSORT, Consolidated Standards of

Resuscitation 58

(24.90)

Reporting Trials.

^a University (n = 19), nonprofit (n = 2).

Journal of publication (n = 233)

Academic Emergency Medicine

Journal of Trauma and Acute Care Surgery Journal of Emergency Medicine

Emergency Medicine

Journal

21

(9.01)

16

(6.87)

19

(8.15)

14

(6.01)

^b Unclear (n = 2).

^c Vaccine (n = 0), education (n = 19), lab test (n = 1).

reported all TIDieR checklist items [13]. Additionally, they found that less than half of their studies reported materials used for the interven- tion, intervention location, and the planned and actual adherence for

Funding type (n = 233)

Country where RCT was conducted (n = 233)

Hypothesis of RCT (n = 233)

Type of intervention (n = 233)

Study design of RCT (n = 233)

RCT blinding (n = 233)

Did the authors mention following CONSORT guidelines? (n = 233)

Did the authors mention following TIDieR guidelines? (n = 233)

Hospital 6 (2.58)

Industry 6 (2.58)

Private 23

(9.87)

Public 31

(13.30)

Mixed 35

(15.02)

No funding 55

(23.61)

Not mentioned 56

(24.03)

Other^a 21

(9.01)

US 81

(34.76)

Outside of US 149

(63.95)

Both 3 (1.29)

Superiority 210

(90.13)

Equivalence 10

(4.29)

Noninferiority 11

(4.72)

Other^b 2 (0.86)

Device 71

(30.47)

Drug 65

(27.90)

Mixed 14

(6.01)

Protocol 43

(18.45)

Procedure 20

(8.58)

Other^c 20

(8.58)

Crossover 62

(26.61)

Cluster 10

(4.29)

Factorial 2 (0.86)

Parallel arm 159

(68.24)

No blinding 133

(57.08)

Single-blind 43

(18.45)

Double-blind 57

(24.46)

Yes 26

(11.16)

No 165

(70.82)

Chart statement 42

(18.03)

Yes 0 (0.0)

No 233

(100.0)

how well the intervention was conducted according to plan [13]. These findings are similar to our study. Because reporting in emergency medicine is limited, we feel it would be challenging to reproduce the findings of an emergency medicine RCT.

The methodology of a well-documented and reproducible study could be disseminated among other medical specialties. However, our study found that reporting of materials (item 3), who provided the in- tervention (item 5), where the intervention was performed (item 7), modifications (item 10), and adherence (item 11 and 12), were not well reported. The TIDieR checklist was created as a minimum standard of reporting to ensure a trial intervention could be described with enough information to be reproduced [12]. However, without adher- ence to what materials were used for an intervention (item 3), who pro- vided the intervention (item 5), or where the intervention took place (item 7), it would be challenging to reproduce the trial findings at a dif- ferent location. Study reproduction ensures that trial findings are reli- able, verified, and accurate [27]. For example, a study within our sample found that patients who had longer wait times for massive transfusion (MT) intervention had increased mortality [28]. Given that this study failed to report the materials, location, or who provided the intervention, the results of this study may not remain valid if con- ducted elsewhere. Furthermore, the study reported that every minute past the time of MT protocol (MTP) initiation increased patient mortal- ity by 5% [28]. Without providing the location of the trial, hospitals that lack the infrastructure needed to initiate MTP may lead to increasED wait time and therefore could express increased patient mortality rates. In a field that requires physicians to make critical decisions, it is

Fig. 2. Histogram of TIDieR item frequency among emergency medicine RCTs.

Table 2

Intervention reporting before and after TIDieR checklist publication.

Item^a Group year 2011-2013, N = 107 Group year 2016-2018, N = 126 Total 2011-2018, N = 233

	N (%)	[95% CI]		N (%)	[95% CI]		N (%)	[95% CI]
1. Brief Name	107 (100)	[100.0-100.0]		126 (100)	[100.0-100.0]		233 (100)	[100.0-100.0]
2. why	107 (100)	[100.0-100.0]		126 (100)	[100.0-100.0]		233 (100)	[100.0-100.0]
3. What (materials)	45 (42.06)	[35.72-48.39]		55 (43.65)	[37.29-50.02]		100 (42.92)	[36.56-49.27]
4. What (procedures)	69 (64.49)	[58.34-70.63]		87 (69.05)	[63.11-74.98]		156 (66.95)	[60.91-72.99]
5. Who provided	38 (35.51)	[29.40-41.66]		45 (35.71)	[29.56-41.87]		83 (35.62)	[29.47-41.77]
6. How (mode of delivery)	106 (99.07)	[97.83-100.0]		125 (99.21)	[98.07-100.0]		231 (99.14)	[97.96-100.0]
7. Where	29 (27.10)	[21.40-32.81]		51 (40.48)	[34.17-46.78]		80 (34.33)	[28.24-40.43]
8. When and how much	45 (42.06)	[35.72-48.39]		70 (55.56)	[49.18-61.94]		115 (49.36)	[42.94-55.78]
9. Tailoring	5 (4.67)	[1.96-7.38]		4 (3.17)	[0.92-5.43]		9 (3.86)	[1.39-6.33]
10. Modifications	1 (0.93)	[0.00-2.17]		5 (3.97)	[1.46-6.47]		6 (2.58)	[0.54-4.61]
11. How well (assessment)	4 (3.74)	[1.30-6.17]		4 (3.17)	[0.92-5.43]		8 (3.43)	[1.10-5.77]
12. How well are outcomes reported	2 (1.87)	0.13-3.61		3 (2.38)	[0.42-4.34]		5 (2.15)	[0.29-4.01]

^a Items included if “Yes, complete”, item options “No” and “N/A” were not included.

imperative to completely report interventions that could influence clin- ical practice.

With intervention reporting being suboptimal in emergency medi- cine RCTs, studies that are used to dictate clinical practice could be un- reliable. Furthermore, given that physicians use their expert opinion in conjunction with evidence from RCTs to make decisions for clinical practice, this may cause unforeseen effects on patient care if interven- tions are not reproducible. By conducting and replicating studies, re- searchers have found that current standards of care may not remain as significant for patient care when used in clinical application. For in- stance, Medical reversals – a more robust trial with significant findings that contradict clinical practice – have gained recognition due to their ability to expose limitations in current practices [29]. A study by Prasad et al. found that of 363 articles addressing standard of care practices, ap- proximately 40% reversed current practices [30]. Such reversals are con- tingent upon the ability of an intervention to be replicated. Furthermore, the reversal of a practice implies that patients being

treated with old standard of care practice were at risk of harm or re- ceived inferior treatment [31]. However, it is important to understand that patients cannot be treated with interventions that have not yet been developed. Therefore, it is important to emphasize the need for more robust trials that focus on the ability of the intervention to be reproduced. Guidelines serve this purpose by providing a standardized format for what to include within a manuscript. Given the abundance of published guidelines, we feel it is important to discuss how the TIDieR checklist can be more widely implemented and how it can be used to in- crease reproducibility and improve clinical practice.

We have highlighted the need for better intervention reporting; however, it is also important to discuss how the TIDieR checklist can gain recognition, which currently has only an Altmetric score of 269 and 2223 citations according to Google Scholar. Given the methodolog- ical development of TIDieR, created and modified with a Delphi survey of 125 authors of clinical guidelines, statisticians, journal editors, and clinicians, we feel it should be the checklist of choice when reporting

Fig. 3. interrupted time series analysis before and after TIDieR publication.

Table 3

Associated factors with intervention reporting

Characteristics (n = 233) Variables

IRR^a [95% CI] P value

Funding

descriptions. Item 5 of CONSORT only requests that “the interventions for each group [be described] with sufficient details to allow replication, including how and when they were actually administered”; however, such a broad statement does not adequately capture the details neces- sary for true reproduction [6]. Our study demonstrated no significant

trend in intervention reporting since the publication of TIDieR. How-

Industry + Private (n = 29)

1

[reference] 1.00

1

[reference]

ever, similar findings have been reported in reference to the use of CON- SORT; and yet, CONSORT is one of the major guidelines used in clinical

Public + mixed (n = 66) Not mentioned (n = 56)

No funding (n = 55)

Location

[0.94-1.07] 0.957

0.99

[0.93-1.06] 0.825

0.98

[0.92-1.05] 0.578

research. For example, Hopewell et al. conducted a time series analysis and found no change after CONSORT publication; however, they did find that journals who endorsed CONSORT guideline had a greater mean checklist adherence than those that did not endorse CONSORT guideline [23]. Results from Shamseer et al. suggest that CONSORT

gained recognition through the support of editorial organizations,

Outside of US (n = 149)

Inside of the US or both inside and outside of the US

1

[reference] 1.04

1

[reference]

such as the International Committee of medical journals Editors (ICMJE) [32]. For this reason, we encourage major editorial organiza-

(n = 84)

Hypothesis

[0.98-1.09] 0.183

tions to increase awareness of the importance of intervention reporting, as described above. This can be accomplished by having leaders of high

Superiority (n = 210)

1

[reference] 0.95

1

[reference]

impact medical journals write an open access, collaborative editorial

highlighting the importance and implications of quality intervention reporting.

All other hypotheses (n = 23)

Study design

[0.89-1.02] 0.187

Our findings also highlight the need for trial registration. Trials that were registered were more likely to have better intervention reporting.

Parallel arm (n = 159)

1

[reference] 0.97

1

[reference]

Studies regarding CONSORT have found similar results. Even in our study, we found that trials mentioning CONSORT had better interven-

All other study designs (n = 74)

Intervention

[0.91-1.02] 0.201

tion reporting. Given that CONSORT and ICMJE both require trial regis-

tration, we recommend a similar form of registering information [6,7].

Drug (n = 65)

1

[reference] 0.96

1

[reference]

For example, authors could include any additional information that is pertinent to conduct the intervention as supplemental material when submitting their manuscript. Additionally, authors could also upload a

Device (n = 71)

[0.92-1.00] 0.056

detailed intervention description to an online repository (such as

0.97

b Open Science framework) [11,33]. Given that an amendment needs to

All other interventions (n = 97)

Blinding

[0.96-0.99] 0.001

be made to change a study, a new document highlighting these changes could be uploaded to the same platform. An added benefit of our recom-

No blinding (n = 133)

1

[reference] 0.98

1

[reference]

mendation is that it would help to avoid word count limitations and provided additional information that may allow for replication of the in-

Blinding (single or double)

Group assignment

[0.93-1.03] 0.392

tervention. Furthermore, poor RCT adherence may be influenced by

manuscript length and word count limitations [34]. Given that TIDieR

Double arm (n = 188)

1

[reference] 1.01

1

[reference]

is an extension of the CONSORT 2010 (item 5) [12] statement, increased exposure and the implementation of our recommendations may permit TIDieR to have similar effects on checklist adherence for intervention

2+ arm (n = 45)

Protocol availability

[0.96-1.07] 0.577

reporting as CONSORT.

No (n = 226)

1

[reference] 1.05

1

[reference]

Conclusions

In summary, our study evaluating the quality of intervention

Yes (n = 7)

CONSORT endorsement

[0.98-1.13] 0.176

reporting in emergency medicine RCTs found that reporting warrants improvement. Few studies mentioned the use of TIDieR or CONSORT

No (n = 26)

1

[reference] 1.05

1

[reference]

in their manuscript. Furthermore, no study met all 12 of the TIDieR checklist items. Lack of quality intervention reporting can make it chal-

Yes/Chart (n = 207) TIDieR endorsement

No (n = 233)

[1.02-1.09] 0.004^b

1 1

[reference] [reference]

lenging to reproduce trial findings. Additionally, poor reporting can in- fluence patient care. Better intervention reporting can be accomplished by raising awareness of the implications of intervention reporting.

Yes (n = 0) – –

Trial registry

Limitations

None (n = 88)

1

[reference] 1.05

1

[reference]

b

To our knowledge, this will be the first study to review the overall quality of intervention reporting of RCTs in emergency medicine

Yes, listed (n = 145)

^a IRR = incidence rate ratio.

^b Significant p-value b0.05.

[1.00-1.11] 0.035

journals. We feel that our methodology for this study was robust and conveys the limitations of current intervention reporting. However, we recognize the limitations present in our own study as well. Due to the recent publication of TIDieR, a similar analysis conducted at a later

clinical trial interventions. Furthermore, TIDieR checklist was created as an extension of item 5 of the CONSORT statement based on an identified need by experts that CONSORT fails to capture necessary intervention

date may yield different results. Because our study focused on emer- gency medicine journals, it is possible that our findings may not be gen- eralizable to other medical specialties. Lastly, we realize that it takes

time to implement change, and given that TIDieR is a relatively new guideline, we recommend a 5-year follow-up study to assess the effects of TIDieR adherence.

Supplementary data to this article can be found online at https://doi. org/10.1016/j.ajem.2019.12.030.

Funding

This investigation was funded by an Oklahoma State University Pres- idential Research Grant. Matt Vassar is supported by the Oklahoma Cen- ter for the Advancement of Science and Technology. Lehana Thabane, Gavin Gardner, and Kelly Murray do not have extramural funding.

Declaration of competing interest

None.

Acknowledgements

MV and SJ conceived the study. MV, IF, SJ, and LT designed the study protocol. MV, LT, KM, and GG supervised the study. MV received re- search funding. DR and JP extracted data. IF, SJ, and MV conducted data analysis, organization, and interpretation. IF, JP, DR, and DT drafted the manuscript and all authors helped with critical revision of the manuscript.

References

1. Venkatesh AK, Savage D, Sandefur B, Bernard KR, Rothenberg C, Schuur JD. System- atic review of emergency medicine clinical practice guidelines: implications for re- search and policy. PLOS ONE 2017;12(6):e0178456. https://doi.org/10.1371/ journal.pone.0178456.
2. Mansournia MA, Higgins JPT, Sterne JAC, Hernan MA. Biases in randomized trials: a conversation between trialists and epidemiologists. Epidemiology 2017;28(1):54-9. https://doi.org/10.1097/EDE.0000000000000564.
3. Brown J, Lane A, Cooper C, Vassar M. The results of randomized controlled trials in emergency medicine are frequently fragile. Ann Emerg Med 2019;73(6):565-76. https://doi.org/10.1016/j.annemergmed.2018.10.037.
4. Moher D, Pham B, Jones A, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998;352 (9128):609-13. https://doi.org/10.1016/S0140-6736(98)01085-X.
5. Turner L, Shamseer L, Altman DG, Schulz KF, Moher D. Does use of the CONSORT statement impact the completeness of reporting of Randomised controlled trials published in medical journals? A Cochrane review a. Syst Rev 2012;1(1):60. https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/2046-4053- 1-60.
6. Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 statement: up- dated guidelines for reporting parallel group randomised trials. Int J Surg 2011;9

   (8):672-7. https://doi.org/10.1016/j.ijsu.2011.09.004.

   De Angelis C, Drazen JM, Frizelle FA, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. Croat Med J 2004;45(5): 531-2. https://www.ncbi.nlm.nih.gov/pubmed/15906482.

7. Sims MT, Henning NM, Cole Wayant C, Vassar M. Do emergency medicine journals promote trial registration and adherence to reporting guidelines? A survey of “in- structions for authors”. Scand J Trauma Resusc Emerg Med 2016;24(1):137. https://doi.org/10.1186/s13049-016-0331-3.
8. Montane E, Vallano A, Vidal X, Aguilera C, Laporte J-R. Reporting randomised clinical trials of analgesics after traumatic or orthopaedic surgery is inadequate: a systematic review. BMC Clin Pharmacol 2010;10(2). https://doi.org/10.1186/1472-6904-10-2.
9. Yao AC, Khajuria A, Camm CF, Edison E, Agha R. The reporting quality of parallel randomised controlled trials in ophthalmic surgery in 2011: a systematic review. Eye 2014;28(11):1341-9. https://doi.org/10.1038/eye.2014.206.
10. Schroter S, Glasziou P, Heneghan C. Quality of descriptions of treatments: a review of published randomised controlled trials. BMJ Open 2012;2(6). https://doi.org/10. 1136/bmjopen-2012-001978.
11. Hoffmann TC, Glasziou PP, Boutron I, et al. Better reporting of interventions: tem- plate for intervention description and replication (TIDieR) checklist and guide. BMJ 2014;348:g1687. https://doi.org/10.1136/bmj.g1687.
12. Chauvin A, Truchot J, Bafeta A, Pateron D, Plaisance P, Yordanov Y. Randomized con- trolled trials of simulation-based interventions in emergency medicine: a methodo- logical review. Intern Emerg Med 2018;13(3):433-44. https://doi.org/10.1007/ s11739-017-1770-1.
13. Office for Human Research Protections (OHRP). 45 CFR 46. HHS.gov. https://www. hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/index.html. Published February 16, 2016. Accessed July 25, 2019.
14. Jellison S. TIDieR emergency medicine; 2019. https://doi.org/10.17605/OSF.IO/ B853M.
15. Da Q Rayyan. QCRI, the systematic reviews web app. https://rayyan.qcri.org, Accessed date: 26 July 2019.
16. Lai R, Chu R, Fraumeni M, Thabane L. Quality of randomized controlled trials reporting in the primary treatment of brain tumors. J Clin Oncol 2006;24(7): 1136-44. https://doi.org/10.1200/JCO.2005.03.1179.
17. Thabane L, Chu R, Cuddy K, Douketis J. What is the quality of reporting in weight loss Intervention studies? A systematic review of randomized controlled trials. Int J Obes (Lond) 2007;31(10):1554-9. https://doi.org/10.1038/sj.ijo.0803640.
18. Ghimire S, Kyung E, Kang W, Kim E. Assessment of adherence to the CONSORT state- ment for quality of reports on randomized controlled trial abstracts from four high- impact general medical journals. Trials 2012;13(1). https://doi.org/10.1186/1745- 6215-13-77.
19. Checketts JX, Sims MT, Detweiler B, Middlemist K, Jones J, Vassar M. An evaluation of reporting guidelines and clinical trial registry requirements among Orthopaedic sur- gery journals. J Bone Joint Surg Am 2018;100(3):e15. https://doi.org/10.2106/JBJS. 17.00529.
20. Samaan Z, Mbuagbaw L, Kosa D, et al. A systematic scoping review of adherence to reporting guidelines in health care literature. J Multidiscip Healthc 2013;6:169-88. https://doi.org/10.2147/JMDH.S43952.
21. Turner L, Shamseer L, Altman DG, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev 2012;11: MR000030. https://doi.org/10.1002/14651858.MR000030.pub2.
22. Hopewell S, Ravaud P, Baron G, Boutron I. Effect of editors’ implementation of CONSORT guidelines on the reporting of abstracts in high impact medical journals: interrupted time series analysis. BMJ 2012;344:e4178. https://doi.org/10.1136/bmj. e4178.
23. Linden A. Conducting interrupted time-series analysis for single- and multiple- group comparisons. Stata J 2015;15(2):480-500. https://doi.org/10.1177/ 1536867X1501500208.
24. Hanley JA. Statistical analysis of correlated data using generalized estimating equa- tions: an orientation. Am J Epidemiol 2003;157(4):364-75. https://doi.org/10. 1093/aje/kwf215.
25. Hughes BT. TIDieR Data Analysis; 2019. https://doi.org/10.17605/OSF.IO/3A46J.
26. Klein O, Hardwicke TE, Aust F, et al. A practical guide for transparency in psycholog- ical science. Collabra: Psychology 2018;4(1). https://www.collabra.org/articles/10. 1525/collabra.158/?utm_source=TrendMD&utm_medium=cpc&utm_campaign= Collabra%253A_Psychology_TrendMD_0.
27. Meyer DE, Vincent LE, Fox EE, et al. Every minute counts: time to delivery of initial massive transfusion cooler and its impact on mortality. J Trauma Acute Care Surg 2017;83(1):19-24. https://doi.org/10.1097/TA.0000000000001531.
28. Prasad V, Gall V, Cifu A. The frequency of medical reversal. Arch Intern Med 2011; 171(18):1675-6. https://doi.org/10.1001/archinternmed.2011.295.
29. Prasad V, Vandross A, Toomey C, et al. A decade of reversal: an analysis of 146 contradicted medical practices. Mayo Clin Proc 2013;88(8):790-8. https://doi.org/ 10.1016/j.mayocp.2013.05.012.
30. Prasad V, Cifu A. Medical reversal: why we must raise the bar before adopting new technologies. Yale J Biol Med 2011;84(4):471-8. https://www.ncbi.nlm.nih.gov/ pubmed/22180684.
31. Shamseer L, Hopewell S, Altman DG, Moher D, Schulz KF. Update on the endorse- ment of CONSORT by high Impact Factor journals: a survey of journal “instructions to authors” in 2014. Trials 2016;17(1):301. https://doi.org/10.1186/s13063-016- 1408-z.
32. OSF|Sign In. https://osf.io/dashboard, Accessed date: 29 July 2019.
33. Han C, Kwak K-P, Marks DM, et al. The impact of the CONSORT statement on reporting of randomized clinical trials in psychiatry. Contemp Clin Trials 2009;30 (2):116-22. https://doi.org/10.1016/j.cct.2008.11.004.