a b s t r a c t

Objectives: To determine the diagnostic performances of several single question delirium screens. To the patient we asked: “Have you had any difficulty thinking clearly lately?” To the patient’s surrogate, we asked: “Is the pa- tient at his or her baseline mental status?” and “Have you noticed the patient’s mental status fluctuate through- out the course of the day?” Methods: This was a prospective observational study that enrolled English speaking patients 65 years or older. A research assistant (RA) and emergency physician independently asked the patient and surrogate the single question delirium screens. The reference standard for delirium was a consultation-liaison psychiatrist’s assess- ment using Diagnostic and Statistical Manual of Mental disorders, Fourth Edition, Text Revision (DSM-IV-TR) criteria. All assessments were performed within 3 h and were all blinded to each other.

Results: Of the 406 patients enrolled, 50 (12%) were delirious. A patient who was unable to answer the question “Have you had any difficulty thinking clearly lately?” was 99.7% (95% CI: 98.0%-99.9%) specific, but only 24.0% (95% CI: 14.3%-37.4%) sensitive for delirium when asked by the RA. The baseline mental status surrogate ques- tion was 77.1% (95% CI: 61.0%-87.9%) sensitive and 87.5% (95% CI: 82.8%-91.1%) specific for delirium when asked by the RA. The fluctuating course surrogate question was 77.1% (95% CI: 61.0%-87.9%) sensitive and 80.2% (95% CI: 74.8%-84.7%) specific. When asked by the EP, the single question delirium screens’ diagnostic per- formances were similar.

Conclusions: The patient and surrogate single question delirium assessments may be useful for delirium screening in the ED.

(C) 2018

Introduction

Delirium occurs in approximately 8 to 17% of older emergency de- partment patients (ED) [1-4], and is associated with higher mortality [4], and accelerated functional and cognitive decline [5]. However, this form of acute brain dysfunction is missed in up to 83% of the cases [2], because it is not routinely screened for by health care providers [3]. This may lead to delayed diagnosis of their underlying medical illness

[6] and inappropriate discharges home or to a psychiatric unit [1,6].

* Corresponding author at: Vanderbilt University Medical Center, Department of Emergency Medicine, 311 Oxford House, Nashville, TN 37232-4700, United States.

E-mail address: [email protected]. (J.H. Han).

To help improve delirium recognition in the ED, the Brief Confusion Assessment Method (bCAM) was validated for older ED patients [7]. Al- though the bCAM takes less than 2 min perform, some ED health care providers may feel that this is too long especially in busy ED environ- ments with significant time constraints [8]. Single item delirium screen- ing questions that are asked to the patient or surrogate may an appealing method to screen for delirium in the ED and could easily be integrated into the clinical workflow.

For this investigation, we studied the diagnostic performances of several single questions that could be asked to the patient or their sur- rogates using a psychiatrist’s comprehensive delirium assessment as the reference standard. To the patient we asked: “Have you had any dif- ficulty thinking clearly lately?” To the patient’s surrogate or caregiver,

https://doi.org/10.1016/j.ajem.2018.03.060 0735-6757/(C) 2018

1250 J.H. Han et al. / American Journal of Emergency Medicine 36 (2018) 1249-1252

we asked: “Is the patient at his or her baseline mental status?” and “Have you noticed the patient’s mental status fluctuate throughout the course of the day?”

Table 1

Alternative forms of the questions of interest.

Primary question Alternative questions

Methods

Study design and setting

This was part of series of preplanned secondary analyses of a pro- spective observational trial designed to validate brief delirium assess- ments. The results of this analysis have not been previously published and the methods have been previously described in detail [7,9,10]. Briefly, this was a prospective observational study conducted at a tertia- ry care, academic ED that sees approximately 57,000 patients annually. The local institutional review board reviewed and approved this study.

“Have you had any difficulty thinking clearly lately?”

“Is the patient at his or her baseline mental baseline?”

“Have you noticed the patient’s mental status fluctuate throughout the course of the day?”

“Do you feel like your ability to think is foggy?”

“Do you feel like your brain has difficulty processing?”

“Is the patient acting normally to you right now?”

“Does the patient seem more confused to you right now?”

“Have you noticed the patient have good moments and bad moments with regard to his/her thinking?”

“Have you noticed his/her confusion get better and worse throughout the course of the day?”

Selection of participants

We enrolled a convenience sample of patients from July 2009 to Feb- ruary 2012 from 8 AM to 4 PM. The enrollment window was based upon the psychiatrist’s availability and was limited to one patient per day. Pa- tients were included if they were 65 years or older, in the ED for b12 h at the time of enrollment, and not in a hallway bed. Patients were excluded if they were non-English speaking, previously enrolled, deaf or blind, co- matose, non-verbal or unable to follow simple commands prior to their acute illness, or did not complete all the study assessments (e.g., discharged prior to the reference standard delirium assessment). Patients who were non-verbal or unable to follow simple commands prior to their acute illness were considered to have end-stage dementia; diagnosing delirium in this patient population is difficult, even for a psychiatrist.

Single-item delirium screens

A research assistant and board-certified emergency physician asked the patient “Have you had any difficulty thinking clearly lately?” There were three possible responses: “No”, “Yes”, and “Unable to answer.” If the patient replied “I don’t know”, then this response was classified as “Unable to answer.” If a patient surrogate was with the patient in the ED, then they were asked “Is the patient at his or her baseline mental status?” and “Have you noticed the patient’s mental status fluctuate throughout the course of the day?” At the interviewers’ own discretion, they were permitted to use alternative forms of these questions (Table 1) if further clarification was required. Surrogates were typically family members such as the patient’s spouse or children. The research assistant and emergency physician asked these questions within 3- hours of each other and were blinded to each other. Typically, the re- search assistant asked these questions first after consent was obtained.

Reference standard for delirium

The reference standard for delirium was performed by one of three consultation-liaison psychiatrists within 3 h of the Research staff‘s as- sessment. These psychiatrists had a mean 11 years of clinical experience and diagnosed delirium as part of their routine clinical practice. They used Diagnostic and Statistical Manual of Mental Disorders, Fourth Edi- tion, Text Revision (DSM-IV-TR) criteria to diagnose delirium [11]. They performed comprehensive a patient examination that used all means of Patient evaluation and testing as well as data gathered from those who best understood the patient’s current mental status (e.g., the patient’s surrogates, physician, and nursing staff). They also routinely performed a battery of bedside cognitive tests and a focused neurological examina- tion. The consultation-liaison psychiatrists were blinded to the single item delirium screens.

Raters were allowed to ask alternative forms of the questions to provide clarification to the patient or surrogate. Which alternative questions to use was left to the discretion of the rater.

Additional variables collected

Dementia was ascertained using the medical record; the patient’s clinical problem list or physician history and physical examination from the outpatient or inpatient settings were reviewed. A research as- sistant initially performed the medical record review and entered the data directly into an electronic database. Validation rules were used to minimize data entry errors. The principal investigator then reviewed the medical record and double checked the database for accuracy.

Data analysis

Measures of central tendency and dispersion for continuous vari- ables were reported as medians and interquartile ranges (IQR). Categor- ical variables were reported as absolute numbers and proportions. Sensitivities, specificities, positive likelihood ratios (PLRs), and negative likelihood ratios (NLRs) were calculated with their 95% confidence in- tervals (95% CI) for both the research assistant and physician. The re- sponses for the patient question (“Have you had any difficulty thinking clearly?”) were considered to have a hierarchical order; pa- tients who gave the “No” response were considered to have the least amount of acute brain dysfunction and those who were “Unable to an- swer” the question were considered to have the worst acute brain dys- function. Patients who responded “Yes” were considered to have an intermediate degree of acute brain dysfunction. Therefore, the diagnos- tic performances for two cut-offs were calculated: 1) “Yes” or “unable to answer” response was considered a positive screen whereas a “No” re- sponse was considered a negative screen and 2) “unable to answer” was considered a positive screen whereas a “No” or “Yes” response was considered a negative screen. For the surrogate questions, the diag- nostic performances for “Is the patient at his or her baseline mental sta- tus?” and “Have you noticed the patient’s mental status fluctuate throughout the course of the day?” were calculated for each question in- dividually and in combination. All statistical analyses were performed with SAS 9.4 (SAS Institute, Carey, NC).

Results

A total of 406 patients were enrolled, and of these, 50 (12.3%) were delirious. Their characteristics can be seen in Table 2. The diagnostic performance of asking the patient “Have you been thinking clearly?” can be seen in Table 3. If the patient responded “Yes” or was unable to answer this question, then this question had modest sensitivity (62.0% to 68.0%) and specificity (78.9% to 80.1%); the PLR and NLR at this cutoff minimally changed the probability of delirium as diagnosed by the psy- chiatrist. If the patient was unable to answer the question (“Yes” and

J.H. Han et al. / American Journal of Emergency Medicine 36 (2018) 1249-1252 1251

Table 2

Patient characteristics and demographics

Enrolled patients (n = 406)

status or a fluctuating course demonstrated very good sensitivity and specificity if taken individually and had a moderate effect on increasing or decreasing the likelihood of delirium. When used in combination, however, the sensitivity and negative likelihood ratio appeared to im-

Median age (IQR) 73.5 (69, 80)

Female gender 202 (49.8%)

Non-white race 57 (14.0%)

Education

Elementary or below	9 (2.2%)
Middle school	48 (11.8%)
High school	163 (40.2%)
College	118 (29.1%)
Graduate school	67 (16.5%)
Missing	1 (0.3%)

Dementia in medical record 24 (5.9%) ED chief complaint

Abdominal pain 17 (4.2%)

Altered mental status 23 (6.2%)

Chest pain 67 (16.5%)

Generalized weakness 40 (9.9%)

Shortness of breath 46 (11.3%)

Syncope 23 (5.7%)

Admitted to the hospital 294 (72.4%) Abbreviations: IQR, interquartile range; ED, emergency department.

“No” responses were considered a negative response), then the specific- ity increased to approximately 99% and the PLR was over 20 for both raters, strongly increasing the likelihood of having delirium. The sensi- tivity at this cutoff, however, was only 24.0% and 14.0% for the research assistant and physician, respectively. The weighted kappa between the two interviewers was 0.62 (95% CI: 0.40-0.85) indicating moderate inter-rater reliability.

The diagnostic performances of the surrogate questions can be seen be seen in Table 4. Surrogates were available in the ED for 283 (69.7%) and 272 (67.0%) of the patients for the research assistant and physician, respectively. The question “Is the patient at his or her baseline mental status?” had very good sensitivity (77.1% to 82.4%) and specificity (87.5% to 88.7%) for both raters. Based upon the likelihood ratios, the presence or absence of altered mental status according to the surrogate moderately increased or decreased the likelihood of delirium, respec- tively. The kappa was 0.75 (95% CI: 0.69-0.88) indicating very good inter-rater reliability between the research assistant and physician. The question “Have you noticed the patient’s mental status fluctuate throughout the course of the day?” had similar diagnostic characteris- tics and inter-rater reliability as the altered mental status question. When the two surrogate questions were combined, the sensitivity and NLR improved for both raters with a minimal decrease in specificity (Table 4).

Discussion

In this investigation, we evaluated three simple questions that may have the potential to improve delirium recognition in the ED. Though insensitive to rule-out delirium, a patient who was unable to answer “Have you had any difficulty thinking clearly today?” was highly likely to be delirious. Asking the surrogate if the patient had altered mental

prove. Consequently, all three questions may have clinical utility in de- tecting delirium in the busy ED environment with sufficient reliability. To our knowledge, no study has evaluated the patient’s ability to de- termine if they have delirium. Given delirium’s heterogeneous nature, it was not surprising to observe that there were some Delirious patients who were cognizant of their acute brain dysfunction while others had little insight. Patients who were unable to respond to this question had the highest likelihood (PLR N 20) of having delirium due to its high specificity. These patients had more significant impairments in global cognition, disorganized thinking, or inattention (Supplemental) to the extent in which they were unable to answer the question. This cut-off, however, was only 14% to 24% sensitive indicating that answer- ing “Have you been thinking clearly?” with a “Yes” or “No response did not reduce the likelihood of delirium. There were a significant propor- tion of patients who responded “Yes” to this question (~20%) but did not have delirium. It is possible that these patients had very subtle im- pairments in cognition but did not meet the threshold of delirium as di- agnosed by the psychiatrists. Future studies should investigate if subtle Cognitive impairments exist and if these patients have clinical sequelae

such as the development of delirium during hospitalization.

The diagnostic accuracies of our surrogate delirium questions were similar to what was observed by Sand et al. In 21 oncology inpatients, they asked a patient’s family member or friend “Do you think [name of the patient] has been more confused lately?” and called this the Sin- gle Question in Delirium (SQiD) [12]. They observed that this question was 80% (95% CI: 28%-99%) sensitive and 71% (95% CI: 42%-92%) specif- ic for delirium [12]. We expected the surrogate questions to have better sensitivity, because family members or caregivers should have the most knowledge about the patient’s mental status. It is possible that the patient’s cognitive deficits may have been subtle enough where they were not readily apparent to the surrogates. Surrogates may have also attributed the delirious patient’s change in mental status as a normal part of the patient’s illness or a medication they received in the ED. Many of the surrogates were the patient’s spouses and it is possible that they may have underlying cognitive impairment. Lastly, the dis- crepancy may have been secondary to time; the psychiatrists’ reference assessments were performed within 3 h of each other. The patient may not have been delirious during the research team’s evaluation and be- came delirious during the psychiatrist evaluation.

This study has several limitations. At the design of the study, we used the term “mental status” which required clarification in most cases; though the research staff was trained on how to provide clarifying ques- tions, these questions were not standardized and may have decreased our inter-rater reliability. Despite this, however, the kappas were above 0.70 indicating very good inter-rater reliability. We enrolled a convenience sample which may have introduced selection bias. The 95% CI for sensitivities were wide and we were unable to determine if one question or the combination of questions were better than the other with any statistical certainty. Our reference standard for delirium was based upon DSM-IV-TR criteria. The new DSM-5 criteria may be

Table 3

Diagnostic performance of “Have you had any difficulty thinking clearly today?” asked to the patient.

	Positive response	TP	FN	TN	FP	Sensitivity (95% CI)	Specificity (95% CI)	PLR (95% CI)	NLR (95% CI)
RA	Yes or unable to answer	34	16	285	71	68.0% (54.2%-79.2%)	80.1% (75.6%-83.9%)	3.41 (2.57-4.52)	0.40 (0.27-0.60)
	Unable to answer	12	38	355	1	24.0% (14.3%-37.4%)	99.7% (98.4%-100.0%)	85.44 (11.35-643.07)	0.76 (0.65-0.89)
Physician	Yes or unable to answer	31	19	281	75	62.0% (49.2%-74.1%)	78.9% (74.4%-82.9%)	2.94 (2.19-3.96)	0.48 (0.34-0.69)
	Unable to answer	7	43	354	2	14.0% (7.0%-26.2%)	99.4% (98.0%-99.9%)	24.92 (5.32-116.63)	0.86 (0.77-0.97)

“Have you had any difficulty thinking clearly today?” was asked to 406 older emergency department patients. This question was asked by research assistants (RA) and a physician. Three responses were possible: “Yes”, “No”, “Unable to answer.” Sensitivities, specificities, positive likelihood ratios (PLR) and negative likelihood ratios (NLR) are reported with their 95% con- fidence intervals (95% CI). The reference standard for delirium was a psychiatrist assessment using Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision criteria. The weighted kappa was 0.62 (95% CI 0.40-0.85) indicating good inter-rater reliability between the raters. TP, true positive; FN, false negative; TN, true negative; FP, false positive.

1252 J.H. Han et al. / American Journal of Emergency Medicine 36 (2018) 1249-1252

Table 4

The diagnostic performance of surrogate questions for delirium

	Question	TP	FN	TN	FP	Sensitivity (95% CI)	Specificity (95% CI)	PLR (95% CI)	NLR (95% CI)
RA	Altered mental status	27	8	217	31	77.1% (61.0%-87.9%)	87.5% (82.8%-91.1%)	6.17 (4.24-8.98)	0.26 (0.14-0.48)
	Fluctuating course	27	8	201	47	77.1% (61.0%-87.9%)	80.2% (74.8%-84.7%)	4.07 (2.97-5.57)	0.28 (0.15-0.52)
	Altered mental status or fluctuating course	30	5	193	55	85.7% (70.6%-93.7%)	77.8% (72.3%-82.6%)	3.86 (2.95-5.06)	0.18 (0.08-0.41)
Physician	Altered mental status	28	6	211	27	82.4% (66.5%-91.7%)	88.7% (84.0%-92.1%)	7.26 (4.93-10.70)	0.20 (0.10-0.41)
	Fluctuating course	27	7	189	49	79.4% (63.2%-89.7%)	80.3% (74.7%-84.8%)	3.86 (2.85-5.22)	0.26 (0.13-0.50)
	Altered mental status or fluctuating course	31	3	183	55	91.2% (77.0%-97.0%)	76.9% (71.1%-81.8%)	3.95 (3.06-5.09)	0.11 (0.04-0.34)

The research assistants (RA) and a physician asked the patient’s surrogate two questions: [1] “Is the patient at his/her baseline mental status?” and [2] “Have you noticed a fluctuation is his/her mental status”. Surrogates were present for 283 (69.7%) and 272 (67.0%) of the patients. Sensitivities, specificities, positive likelihood ratios (PLR) and negative likelihood ratios (NLR) are reported with their 95% confidence intervals (95% CI). The reference standard for delirium was a psychiatrist assessment using Diagnostic and Statistical Manual of Mental Dis- orders, Fourth Edition, Text Revision criteria. The kappa for the altered mental status and fluctuation questions was 0.75 (95% CI: 0.69-0.88) and 0.74 (95% CI: 0.65-0.83), respectively indicating very good inter-rater reliability between the raters. TP, true positive; FN, false negative; TN, true negative; FP, false positive.

more restrictive, which may slightly change the assessments’ sensitivi- ties and specificities [13]. We relied upon the electronic medical record to determine dementia status which likely underestimated its true prevalence [14]. Consequently, only 24 (5.9%) patients were classified as having dementia, and we were unable to reliably determine if the di- agnostic performances of the single question delirium screens were af- fected by dementia status. Because detecting delirium superimposed on dementia can be challenging [15], future studies with larger sample sizes are needed to determine how dementia status affects the diagnos- tic performances of these single questions. This study was conducted in a single ED located at an urban, academic hospital and included patients who were 65 years and older because they are disproportionately af- fected by delirium. Our findings may not be generalizable in other set- tings and patients b65 years of age.

In conclusion, asking the patient “Have you had any difficulty think- ing clearly?” strongly increased the likelihood of delirium if the patient was unable to answer the question, but had poor sensitivity. Asking their surrogates “Is the patient at his/her baseline mental status?” or “Have you noticed the patient’s mental status fluctuate throughout the course of the day?” had very good sensitivity and specificity for deliri- um. Using the combination of both surrogate mental status questions may increase sensitivity. Given these questions’ brevity and moderately good diagnostic performance, health care providers should consider using these questions as part of their routine clinical practice to screen for delirium.

Supplementary data to this article can be found online at https://doi. org/10.1016/j.ajem.2018.03.060.

Source of funding

Dr. Han and this study were funded by the Emergency Medicine Foundation Career Development Award and National Institute on Aging of the National Institutes of Health under award number K23AG032355. This study was also supported by the National Center for Research Resources, Grant UL1 RR024975-01 and is now at the Na- tional Center for Advancing Translational Sciences, Grant 2 UL1 TR000445-06. Dr. Ely was supported in part by the National Institutes of Health R01AG027472 and R01AG035117, and a Veteran Affairs MERIT award and Geriatric Research, Education, and Clinical Center. Drs. Schnelle and Dittus are also supported by the Veteran Affairs

Geriatric Research, Education, and Clinical Center. The content is solely the responsibility of the authors and does not necessarily represent the official views of Vanderbilt University Medical Center, Emergency Medicine Foundation, National Institutes of Health, and Veterans Affairs.

References

1. Hustey FM, Meldon SW, Smith MD, Lex CK. The effect of mental status screening on the care of elderly emergency department patients. Ann Emerg Med 2003;41(5): 678-84.
2. Elie M, Rousseau F, Cole M, Primeau F, McCusker J, Bellavance F. Prevalence and de- tection of delirium in elderly emergency department patients. CMAJ 2000;163(8): 977-81.
3. Han JH, Zimmerman EE, Cutler N, et al. Delirium in older emergency department pa- tients: recognition, risk factors, and psychomotor subtypes. Acad Emerg Med 2009; 16(3):193-200.
4. Han JH, Shintani A, Eden S, et al. Delirium in the emergency department: an inde- pendent predictor of death within 6 months. Ann Emerg Med 2010;56(3):244-52.
5. Han JH, Vasilevskis EE, Chandrasekhar R, et al. Delirium in the emergency depart- ment and its extension into hospitalization (DELINEATE) study: effect on 6-month function and cognition. J Am Geriatr Soc 2017;65(6):1333-8.
6. Reeves RR, Parker JD, Burke RS, Hart RH. Inappropriate psychiatric admission of el- derly patients with unrecognized delirium. South Med J 2010;103(2):111-5.
7. Han JH, Wilson A, Vasilevskis EE, et al. Diagnosing delirium in older emergency de- partment patients: validity and reliability of the delirium triage screen and the brief confusion assessment method. Ann Emerg Med 2013;62(5):457-65.
8. Wong CL, Holroyd-Leduc J, Simel DL, Straus SE. Does this patient have delirium? Value of bedside instruments. JAMA 2010;304(7):779-86.
9. Han JH, Wilson A, Graves AJ, Shintani A, Schnelle JF, Ely EW. A quick and easy delir- ium assessment for nonphysician research personnel. Am J Emerg Med 2016;34(6): 1031-6.
10. Han JH, Wilson A, Graves AJ, et al. Validation of the confusion assessment method for the intensive care unit in older emergency department patients. Acad Emerg Med 2014;21(2):180-7.
11. American Psychiatric Association. Task Force on DSM-IV. Diagnostic and Statistical Manual of Mental Disorders: DSM-IV-TR. 4th ed. Washington, DC: American Psychi- atric Association; 2000.
12. Sands MB, Dantoc BP, Hartshorn A, Ryan CJ, Lujic S. Single question in delirium (SQiD): testing its efficacy against psychiatrist interview, the confusion assessment method and the memorial delirium assessment scale. Palliat Med 2010;24(6): 561-5.
13. Meagher DJ, Morandi A, Inouye SK, et al. Concordance between DSM-IV and DSM-5 criteria for delirium diagnosis in a pooled database of 768 prospectively evaluated patients using the Delirium Rating Scale-revised-98. BMC Med 2014;12(1):164.
14. Laurila JV, Pitkala KH, Strandberg TE, Tilvis RS. Detection and documentation of de- mentia and delirium in acute geriatric wards. Gen Hosp Psychiatry 2004;26(1): 31-5.
15. Morandi A, McCurley J, Vasilevskis EE, et al. Tools to detect delirium superimposed on dementia: a systematic review. J Am Geriatr Soc 2012;60(11):2005-13.