a b s t r a c t

Objective: The LIMIT clinical decision instrument (CDI) was published in 2021 to safely reduce neuroimaging in patients with recurrent seizures. The LIMIT CDI had a sensitivity of 90%, negative predictive value of >99.9%, and reduced neuroimaging by 13.3%. However, the design of the original LIMIT CDI made it cumbersome to use. The goal of this study was to validate the streamlined LIMIT-NeuroImaging (LIMIT-NI) CDI and compare its performance to the original LIMIT CDI. Methods: This was an observational study of patients presenting to three emergency departments with recurrent seizures. The LIMIT-NI CDI was applied to all patients. We calculated the test characteristics of the LIMIT-NI CDI and compared it to unstructured clinical judgement.

Results: 3401 patients were screened, and 2125 patients were included in the final analysis. 16 patients (0.75%) had positive CTs; Both the LIMIT-NI CDI and clinician judgement identified all 16 patients with a sensitivity of 100.0% and a negative predictive value of 100.0%. Using unstructured clinical judgement, emergency providers ordered 835 brain CTs, while only 499 brain CTs would have been ordered using the CDI, a reduction of 15.8% (rel- ative reduction 40.2%).

Conclusion: The LIMIT-NI CDI demonstrated greater ease of application and improved test characteristics com- pared to the original LIMIT CDI. Compared to unstructured clinician judgement, the LIMIT-NI CDI reduced neuro- imaging by 15.8% (relative reduction 40.2%) in recurrent seizure patients. The LIMIT-NI CDI can be used by physicians along with clinical judgement to reduce neuroimaging in the recurrent seizure patient.

(C) 2022

1. Introduction
   1. Background

Computed tomography (CT) usage during emergency department visits increased by approximately 60% between 2005 and 2013 [1]. This Rapid increase in the use of CT has prompted investigation into risks of overutilization of CT imaging: for example, radiation induced malignancy [2-5]. The average effective radiation dose of a single head CT scan is roughly equivalent to an entire year’s worth of background ra- diation [2]. While a single CT scan does not significantly increase cancer risk, radiation associated malignancy can become a public health con- cern if large numbers of people undergo CT Screening procedures of un- certain benefit. In an effort to decrease unnecessary CT imaging, clinical

* Corresponding author.

E-mail address: [email protected] (M. Gunchenko).

¹ Presented at: Pennsylvania Academy of Emergency Physicians Scientific Assembly, Pocono Manor, PA on 04/01/2022.

Decision instruments (CDI) have been designed to assist physicians in optimizing diagnostic and therapeutic decisions [6].

• 1. Importance

Although American Academy of Neurology (AAN) guidelines sug- gest emergent neuroimaging for adults with first time seizure, there is no recommendation for the use of emergent neuroimaging in patients with recurrent seizure [7]. Isenberg, et al. derived and prospectively validated a CDI to determine which patients with recurrent seizures require emergent neuroimaging [8,9]. The LIMIT (Let’s Image Malignancy, ICH, and Trauma) CDI consists of three criteria: active ma- lignancy, history of Intracranial Hemorrhage , and trauma. The LIMIT CDI yielded a sensitivity of 90% and a negative predictive value of >99%. Isenberg, et al. subsequently demonstrated that the LIMIT CDI reduced neuroimaging in recurrent seizure patients by 13.3% as compared to unstructured clinician judgement [10]. In an effort to streamline the application of this CDI, the LIMIT CDI was updated to LIMIT-NeuroImaging CDI (LIMIT-NI), consisting of five criteria: (active)

https://doi.org/10.1016/j.ajem.2022.09.029

0735-6757/(C) 2022

malignancy, (history of) Intracranial hemorrhage , trauma, (focal) neurologic deficit, and intraVentricular shunt.

• 1. Goals of this investigation

The goal of this study was to validate the streamlined LIMIT-NI and compare it to the test characteristics of the original LIMIT CDI.

1. Materials and methods
   1. Study design and setting

This was an observational study of consecutive patients with chief complaint of seizure presenting to three urban hospitals in Philadelphia, Pennsylvania with a combined 170,000 annual emergency department

• 1. Outcome

Fig. 1. Abnormal/Positive Findings on Neuroimaging.

visits. All three sites are staffed by board certified emergency physicians (EPs) and are training sites for a PGY 1-3 emergency medicine residency.

• 1. Selection of participants

Patients >=18 years who presented to the three EDs between May 13, 2019 and August 8, 2021 with a chief complaint of seizure were eligible for this study. Patients were excluded if the emergency clinician deter- mined that the patient did not have a seizure, the patient left prior to completing treatment, or was transferred from an outside hospital. Patients were also excluded for first lifetime seizure.

The LIMIT-NI CDI (active malignancy, history of intracranial hemor-

rhage, traumatic injury/mechanism of injury, focal neurologic deficit, and intraventricular shunt) was applied to all patients via chart review; a history of an active malignancy, intraventricular shunt, and intracra- nial hemorrhage was extracted from the medical record. A focal neurol- ogic deficit was defined as any acute unilateral abnormality in cranial nerve, motor system, or sensation regardless of their level of conscious- ness. A Traumatic head injury/mechanism of injury was defined as any injury above the clavicles with or without external signs of injury, such as abrasions, contusions, or lacerations. A traumatic mechanism of injury included falls from heights greater than ground level.

Three research associates, undergraduate students with standard- ized research training, determined whether the patients were positive or negative for each criterion of the LIMIT-NI CDI. 2509 charts were dis- tributed equally and randomly for review amongst the three research associates who were blinded to the Study objectives and hypothesis. A fourth research associate extracted CT results and patient follow up from the EMR in order to maintain blinding. The data collection forms completed by the abstractors were periodically monitored by the prin- cipal investigator and compared to the actual medical record charts to ensure consistent accuracy of recording. Any questions about patient outcomes and/or discrepancies in the medical record were adjudicated by the principal investigator (PI). The PI was blinded to the elements of the clinical decision instrument when adjudicating patient outcomes.

Findings that constituted an abNormal CT scan, such as intracranial hemorrhage, mass, or acute infarction were defined a priori in Fig. 1. Table 2 summarizes the patients who presented with Abnormal CT findings.

The ordering of neuroimaging was used as a proxy for unstructured clinician judgement.

Follow up for patients who were discharged without neuroimaging was determined by reviewing the EMR for any medical visits in the year following the index ED visit. If a patient demonstrated new neurol- ogic findings on follow up visit and/or abnormalities on follow up neu- roimaging that could be attributed to the index ED visit, then he or she was recorded as a “missed” patient. Patients who did not receive neuro- imaging in the ED and were lost to follow up were excluded from final analysis.

The outcome of interest were the test characteristics of the LIMIT-NI CDI compared to unstructured clinical judgement. We also compared the LIMIT-NI CDI to the original LIMIT CDI.

• 1. Analysis

Study data were collected and managed using REDCap electronic data capture tools [11,12]. Calculations were done using the R statistical software package (https://www.r-project.org/). 95% confidence inter- vals were calculated using the Clopper Pearson CI method. For a rule with a 100% sensitivity, 95% CI ranging from 98.5% to 100%, and a 3% prevalence of positive brain CTs, we estimated a sample size of 1988 patients.

The research was approved by the institutional review board at Temple University. The authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

1. Results
   1. Characteristics of study subjects

3401 patients with a chief complaint of seizure presented during the study period. A patient enrollment diagram is listed in Fig. 2. 892 pa- tients were excluded as they were ineligible for the study. An additional 384 patients were excluded prior to analysis: 254 had first time seizures and 64 were lost to follow up. 2125 patients were included in the final analysis (Fig. 2). The mean age of patients in the cohort was 42, and 804 (37.8%) were female (Table 1). Of the 2125 patients analyzed, 16 patients (0.75%) had abnormal CTs.

• 1. Main results

As seen in Table 3, 16 of 16 patients were identified by the LIMIT-NI CDI resulting in a sensitivity of 100.0% (95% CI 79.4%-100.0%), a specific- ity of 77.1% (95% CI 75.2%-78.9%), and a negative predictive value (NPV) of 100.0% (95% CI 99.8%-100.0%). Similarly, 16 of 16 patients with ab- normal CT scans were identified by unstructured clinician judgement resulting in a sensitivity of 100.0% (95% CI 79.4%-100.0%), a specificity of 61.2% (95% CI 59.0%-63.3%), and a NPV of 100.0% (95% CI 99.7%-

100.0%). Both the LIMIT-NI CDI and unstructured clinician judgement identified all abnormal CT scans.

Using unstructured clinician judgement, clinicians ordered a total of 835 Brain CT scans (39.3%). Utilizing the CDI, clinicians would have only ordered 499 brain CT scans (23.5%). The LIMIT-NI CDI would have re- duced the number of CT scans ordered by 15.8% (relative reduction 40.2%).

Fig. 2. Enrollment Diagram.

1. Limitations

Clinician judgement is closely linked to clinician experience. For any given patient, one physician may have elected to obtain neuroimaging while another physician may have found this to be unnecessary.

Table 1

Demographics.

Age, mean (range) 42 (18-99)

Female, n (%) 804 (37.8%)

Seizure Etiology Number of Patients (%)

Medication non-adherence 1054 (49.6%)

Breakthrough seizure 416 (19.6%)

Unknown 196 (9.2%)

Other 195 (9.2%)

Drug withdrawal 134 (6.3%)

drug intoxication 123 (5.8%)

CDI Elements Number of Patients (%)

Traumatic injury/mechanism of injury 317 (14.9%)

History of intracranial hemorrhage 134 (6.3%)

Focal neurologic deficit(s) 56 (2.6%)

Active malignancy 46 (2.2%)

Intraventricular shunt 22 (1.0%)

Additionally, patient information was collected via chart review and re- quired interpretation of encounter notes; this may have resulted in pa- tient misclassification. Lastly, while the LIMIT-NI can improve on baseline Clinical performance, the wide confidence intervals, a result of the lower than anticipated prevalence of positive CTs, may affect its implementation.

1. Discussion

The decision to obtain neuroimaging in recurrent seizure patients is complex; although majority of patients will not demonstrate acute in- tracranial abnormalities, it is the possibility of missing a life threatening finding in a minority of patients that emphasizes the need for a clinical decision instrument.

Application of the LIMIT-NI CDI decreased neuroimaging by 15.8% (relative reduction 40.2%), when compared with unstructured clinical judgement. The substantial reduction in neuroimaging can optimize pa- tient throughput while limiting harmful cumulative consequences of CT overutilization. While the LIMIT-NI CDI is as sensitive as unstructured clinical judgement, it is more specific, and, in this way, it can improve upon bedside judgement.

The LIMIT-NI CDI is a one-way rule; it is meant to rule out the need for emergent neuroimaging; if the patient is positive for a criterion, the rule does not suggest that the patient requires neuroimaging. If the rule

Table 2

Patients with Positive CT Findings.

Age CDI Factor Findings Clinical Course

43 Trauma Trace subarachnoid hemorrhage along right inferior frontal lobe No acute surgical intervention, admitted to trauma service, discharged

on hospital day 1

38 Trauma Bilateral parietal-occipital subcortical edema with sulcal effacement,

consistent with PRES

53 Active malignancy Extensive vasogenic edema of frontoparietal white matter highly

suspicious for underlying neoplasm with associated mass effect

No acute surgical intervention, admitted to medicine service, discharged on hospital day 8

No acute surgical intervention, admitted to medicine service, eloped on hospital day 4

95 Trauma Minimal focal subarachnoid hemorrhage at left frontal vertex No acute surgical intervention, admitted to trauma service, discharged

on hospital day 3

36 Trauma Longitudinal fracture of the petrous portion of the left Temporal bone involving the External auditory canal and extending to the middle

ear cavity and temporomandibular joint

No acute surgical intervention, admitted to trauma service, discharged on hospital day 1

58 History of intracranial hemorrhage

Large right frontal intraparenchymal hematoma with regional mass effect as well as small amount of regional subarachnoid hemorrhage

No acute surgical intervention, admitted to neurology service; suffered cardiac arrest in setting of intracranial hemorrhage on hospital day 4, unable to be resuscitated, declared deceased

29 Trauma Acute hemorrhagic contusions No acute surgical intervention, admitted to trauma service, discharged on hospital day 4

67 Active malignancy, focal neurologic deficit(s)

65 Trauma, history of intracranial hemorrhage

3 hyperattenuating mass lesions within the right frontal and temporal subcortical white matter with surrounding vasogenic edema consistent with metastasis

Focus of subarachnoid hemorrhagic overlying the high left frontal cortex

No acute surgical intervention, admitted to medicine service, discharged to hospice on hospital day 9

No acute surgical intervention, admitted to trauma service, discharged on hospital day 2

63 Trauma Right sided subarachnoid and subdural hemorrhage No acute surgical intervention, admitted to trauma service, discharged

on hospital day 3

71 Trauma Acute subarachnoid hemorrhage in posterior frontal lobe at vertex No acute surgical intervention, admitted to trauma service, discharged

on hospital day 3

45 Trauma Left temporal lobe intraparenchymal hematoma No acute surgical intervention, intubated due to Seizure activity,

admitted to medicine service, discharged on hospital day 43

74 Trauma, history of intracranial hemorrhage, active malignancy

Left temporal parenchymal hematoma, increased vasogenic edema along left occipital lobe likely new hemorrhagic metastases

No acute surgical intervention, admitted to neurology service, discharged to hospice on hospital day 6

39 Trauma Trace subarachnoid hemorrhage in left frontal lobe sulci, minimal

subdural hemorrhage at left temporal fossa

No acute surgical intervention, admitted to trauma service, discharged on hospital day 2

78 Focal neurologic deficit (s)

Acute to subacute left PCA territory infarct, faint lesion with prominent surrounding vasogenic edema in left posterior occipital subcortical white matter, highly suspicious for metastasis

No acute surgical intervention, admitted to neurology service, discharged on hospital day 2

Abbreviations: PRES, posterior reversible encephalopathy syndrome; PCA, posterior cerebral artery.

Table 3

Performance of LIMIT-NI CDI versus Clinician	Judgement.
	Sensitivity (95% CI)	Specificity (95% CI)	Positive Predictive Value (95% CI)	Negative Predictive Value (95% CI)	Positive Likelihood Ratio (95% CI)
LIMIT CDI LIMIT-NI	92.3% (64.0%-99.8%) 100.0%	79.2% (77.3%-80.9%) 77.1%	2.7% (1.4%-4.7%) 3.2%	99.9% (99.7%-100.0%) 100.0%	4.4 4.4
CDI	(79.4%-100.0%)	(75.2%-78.9%)	(1.8%-5.2%)	(99.8%-100.0%)
Unstructured Physician Judgement	100.0%	61.2%	1.9%	100.0%	2.6
	(79.4%-100.0%)	(59.0%-63.3%)	(1.1%-3.1%)	(99.7%-100.0%)

is incorrectly applied in a two-way fashion, this will likely increase the rate of neuroimaging.

Compared to the LIMIT CDI, the LIMIT-NI CDI is more inclusive and, therefore, easier to apply; this improved ease of application is demon- strated by the greater sensitivity.

64 patients were excluded from analysis, because they did not receive neuroimaging in the emergency department and had no follow up visits within one year of their index ED visit. It is possible that one of the patients in this group may have had an abnormality on neuroimaging; however, given that 1290 patients did not receive neuroimaging and did not dem- onstrate new neurologic findings on follow up visit, this would be unlikely. LIMIT-NI CDI can be used by clinicians along with clinical judgement to reduce neuroimaging in the recurrent seizure patient. Future investi- gation should explore the validation of the LIMIT-NI CDI in a larger, dif-

ferent sample of recurrent seizure patients.

Grant or financial support

None.

CRediT authorship contribution statement Derek Isenberg: Writing - review & editing, Writing - original draft,

Visualization, Validation, Supervision, Software, Resources, Project ad- ministration, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Melissa Gunchenko: Writing - review & editing, Writing - original draft, Formal analysis, Data curation.

Conflicts of interest

None.

Acknowledgements

The authors certify that they have no affiliations with or involve- ment in any organization or entity with any financial interest or non- financial interest in the subject matter or materials discussed in this manuscript.

References

1. Bellolio MF, Heien HC, Sangaralingham LR, et al. Increased computed tomography utilization in the emergency department and its association with hospital admission. West J Emerg Med. 2017;18(5):835. https://doi.org/10.5811/WESTJEM.2017.5. 34152.
2. Litkowski PE, Smetana GW, Zeidel ML, Blanchard MS. Curbing the urge to image. Am J Med. 2016;129(10):1131-5. https://doi.org/10.1016/J.AMJMED.2016.06.020.
3. What are the radiation risks from CT? | FDA. Accessed February 26, 2022. https:// www.fda.gov/radiation-emitting-products/medical-x-ray-imaging/what-are- radiation-risks-ct.
4. NCRP Report 160 - NCRP | Bethesda, MD. Accessed February 26, 2022. https:// ncrponline.org/publications/reports/ncrp-report-160-2/.
5. Avoid Unnecessary Treatments in the ER | choosing wisely. Accessed February 26, 2022. https://www.choosingwisely.org/patient-resources/avoid-unnecessary- treatments-in-the-er/.
6. Green SM, Schriger DL, Yealy DM. Methodologic standards for interpreting clinical decision rules in emergency medicine: 2014 update. Ann Emerg Med. 2014;64(3): 286-91. https://doi.org/10.1016/J.ANNEMERGMED.2014.01.016.
7. Harden CL, Huff JS, Schwartz TH, et al. Reassessment: neuroimaging in the emer- gency patient presenting with seizure (an evidence-based review): report of the therapeutics and technology assessment subcommittee of the American academy of neurology. Neurology. 2007;69(18):1772-80. https://doi.org/10.1212/01.WNL. 0000285083.25882.0E.
8. Isenberg DL, Lin A, Kairys N, et al. Derivation of a clinical decision instrument to identify patients with status epilepticus who require emergent brain CT. Am J Emerg Med. 2020;38(2):288-91. https://doi.org/10.1016/J.AJEM.2019.05.004.
9. Isenberg DL, Muller M, Rodrigues L, et al. Validation of a clinical decision instrument for emergent neuroimaging after a seizure: Let’s image malignancy, intracranial hemorrhage, and trauma (LIMIT). Acad Emerg Med. 2021;28(5):562-8. https:// doi.org/10.1111/ACEM.14205.
10. Isenberg D, Gunchenko M, Fenstermacher R, Gentile N. The LIMIT clinical decision instrument reduces neuroimaging compared to unstructured clinician judgement in recurrent seizures. Am J Emerg Med. 2021. https://doi.org/10.1016/J.AJEM.2021.

10.024. Published online October 25.

1. Harris PA, Taylor R, Minor BL, et al. The REDCap consortium: building an interna- tional community of software platform partners. J Biomed Inform. 2019; 95:103208. https://doi.org/10.1016/J.JBI.2019.103208.
2. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)-A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42 (2):377-81. https://doi.org/10.1016/J.JBI.2008.08.010.