Article, Psychiatry

Suicide screening tools and their association with near-term adverse events in the ED

a b s t r a c t

Objectives: The goal of this study was to evaluate the relationship between various Suicide screening tools and clinical impression with subsequent patient psychiatric admission and near-term adverse emergency depart- ment (ED) events.

Methods: We performed a prospective observational study of 50 patients with suicidal ideation in the ED. Subjects completed a series of depression/suicide screening tools: the Columbia Suicide Severity Scale, SAD PERSONS scale, Patient Health Questionnaire 9, and Beck Scale for Suicidal Ideation. Clinicians were also asked about their impression on likelihood of patient admission. Outcome measures were as follows: need for psychiatric hospital admission, prolonged stay at psychiatric facility, and any adverse events during ED stay including need for unscheduled psychiatric or sedating medications, need for physical restraints, and need for intervention by security staff.

Results: The Beck Scale for Suicidal Ideation, Patient Health Questionnaire 9, and Columbia Suicide Severity Scale did not significantly predict within-ED adverse events or admissions to psychiatric facilities. Wald test for indi- vidual parameters at an ? of .10 level found that patients who were screened positive by their nurse had 3.37 times the odds of adverse within-ED events; patients with a positive SAD PERSONS score had 8.18 times the odds of psychiatric admission greater than 5 days. However, at the ? of .05 level, no screening tools correlated with patient ED course or likelihood of psychiatric admission.

Conclusion: Clinical impression alone and the suicide screening tools showed poor predictive value for near-term events. Data from this study highlight the need for the development of ED-based suicide screening instruments capable of identifying those patients with suicidal ideation at greatest risk.

(C) 2015



Suicide is the 16th leading cause of death worldwide and accounted for 38364 deaths in the United States in 2010 [1]. Self-injurious thoughts and behavior (SITB) such as suicidal ideation (SI) are also com- mon, with lifetime prevalence among US adults of 15.6%, 5.4%, and 5.0% [2]. Self-injurious thoughts and behavior is a common condition evalu- ated in the emergency department (ED), with nearly 420000 annual ED visits for SITB [3]. Given the frequency of SITB seen in the ED and the

? Grant: None/unfunded.

?? Meetings: Previously presented at the Society for Academic Emergency Medicine An- nual Conference San Diego, May 12 to May 15.

* Corresponding author at: Department of Emergency Medicine, VC 2nd Floor Suite 260, 622 West 168th Street, New York, NY 10032. Tel.: +1 212 305 2995.

E-mail address: [email protected] (B.P. Chang).

severity of untreated disease, it is critical to be able to identify SITB pa- tients at greatest risk for an acute event.


Multiple instruments for suicide screening exist ranging from struc- tured interviews [4], self-report surveys [5], to clinician assessments [6]. Although some of these instruments have predicted subsequent risk for death by suicide, past work has focused on long-term suicide risk on the order of years to decades, timeframes impractical for the emergency physician [7]. Few studies have evaluated the utility of these instru- ments in assessing near-term suicide risk for patients in the ED.

Goals of this investigation:

The goal of this pilot study was to explore the relationship of several commonly used suicide screening instruments, along with clinician “in- tuition” (ie, providers’ perception of the patient’s likelihood of having an acute psychiatric emergency requiring admission) with their associa- tion to subsequent patient clinical course in ED patients endorsing SITB.

0735-6757/(C) 2015

B.P. Chang, T.M. Tan / American Journal of Emergency Medicine 33 (2015) 16801683 1681


Study design and setting

We performed a prospective correlation study of 50 patients with a chief complaint of SITB and their 150 ED providers (nurse, resident phy- sician, and attending physician) to evaluate the association of suicide in- strument scores and subjective clinical impression with near-term clinical course. The study setting was a single urban, university- affiliated teaching hospital ED staffed by board-certified emergency medicine physicians and 24-hour Comprehensive Psychiatric Emergen- cy Program. As this represents a pilot study of suicide screening instru- ments not previously studied in the ED, we chose a convenience sample of 50 patients from which test characteristics such as sensitivity and specificity could be inferred, allowing for future studies of adequate power and size. The study protocol was approved by the X Medical Cen- ter Institutional Review Board.

Selection of participants

We identified 50 patients aged 18 years or older presenting to the ED with a chief complaint of “suicidal ideation,” “thinking of hurting my- self,” “I want to die,” or “SI.” Patients with concomitant alcohol or drug intoxication, acute medical illness, or being evaluated with actual sui- cide attempt were excluded. In addition to interviewing each of the 50 patients, we also interviewed the nurse, resident physician, and super- vising attending physician assigned to care for the patient.

Outcome measures

Although completed suicide would be an ideal outcome measure, given the extremely low base rate of completed suicide and coupled with a small sample size for a pilot study, we sought other variables in- dicative of a complicated near-term clinical course. We identified 3 such outcome measures: need for psychiatric hospital admission; prolonged stay at a psychiatric facility (N 5 days); and adverse events during the ED stay including need for unscheduled psychiatric or sedating medica- tions, physical restraints, or security staff intervention.

Data collection and analysis

After obtaining informed consent from the patient and the 3 associ- ated clinicians, patients were administered 4 suicide screening tools: the Patient Health Questionnaire 9 (PHQ-9) [8], the Beck Scale of Suicid- al Ideation [9], the Columbia Suicide Severity Rating Scale (C-SSRS) self- report screener [6], and the SAD PERSONS scale [10]. Clinical providers were given a questionnaire asking years of experience and clinical im- pression, using a 1 to 10 scale, of the likelihood that the patient’s presen- tation represented a high-risk psychiatric emergency requiring hospitalization. Two weeks after ED evaluation, research assistants reviewed medical records using a standardized protocol for subsequent ED and hospital course. Research assistants first noted the ultimate dis- position of the patient followed by a review of any indication of a com- plicated course based on the aforementioned outcome measures by reviewing the medical record, including the nursing/physician notes, medications administered, and report of any events during the hospital course.

The goal of our study was to evaluate if any of the suicide screening instruments had a stronger association with near-term adverse out- comes and to quantify any relationship between clinician intuition and subsequent patient course. We were also interested if years of expe- rience and cadre of provider altered the relationship between clinician intuition and near-term outcomes. To address these questions, we cal- culated receiver operating characteristic curves of the suicide in- struments and ED provider clinical impression relative to the near-term outcome measures. receiver operating characteristic curve analysis was

used to choose screening instruments and ED provider groups for fur- ther study and to determine cutoff values for the screening instruments and ED provider clinical impression ratings. We then performed logistic regressions and used the Wald test for individual parameters to calcu- late odds ratios for screening instruments and clinical impression rat- ings in predicting adverse near-term outcomes. Odds ratios for ED provider clinical impressions were adjusted for provider’s years of experience.


Characteristics of the 50 patients in the study sample are described in Table 1. Twenty-two patients (44%) were admitted to a psychiatric Inpatient unit, 13 of whom (26%) had admissions of greater than 5 days. A Composite outcome measure of adverse within-ED events was calculated for each patient using the aforementioned criteria; 19 pa- tients (38%) had at least 1 within-ED adverse event. Of the 28 patients who were discharged from the ED after psychiatric evaluation, 4 re- presented to the ED within 2 weeks for a chief complaint of SI/depres- sion (3 of the patients) or intoxication (1 patient). Of these 4 patients who re-presented to the ED within the 2-week period, 1 was admitted to a psychiatric facility, and 3 were discharged directly from the ED after psychiatric evaluation.

Receiver operating characteristic curves were constructed for each suicide risk instrument and for the clinical impression score given by nurses, attending physicians, and resident physicians, using each of the 3 outcome measures. Areas under the ROC curves (AUCs) are reported in Table 2. The Beck Scale of Suicidal Ideation, PHQ-9, and C-SSRS instru- ments performed poorly at predicting within-ED adverse events, admis- sions to psychiatric facilities, and prolonged psychiatric admissions, with AUCs less than or not significantly greater than 0.5. The SAD PERSONS scale was better at predicting near-term outcomes, with an AUC of 0.72 (95% confidence interval [CI], 0.67-0.87; P = .009) for predicting psychi- atric facility admission and 0.76 (95% CI, 0.61-0.91; P = .006) for predicting prolonged psychiatric hospitalization; both of these associa- tions were significantly different from the null hypothesis AUC of 0.5 at the ? of .05 level of significance. The SAD PERSONS scale, however, was not predictive of within-ED adverse events.

Nurse and attending physician clinical impressions all had AUCs greater than 0.5, but attending physician clinical impressions were not statistically significant in predicting any outcomes. Nurse clinical im- pression was significantly predictive of Prolonged hospitalization (AUC, 0.71; 95% CI, 0.51-0.92; P = .023) but not predictive of within- ED adverse events (AUC, 0.65; 95% CI, 0.49-0.81; P = .084) or psychiat- ric admission (AUC, 0.59; 95% CI, 0.41-0.77; P = .28). Resident physician

Table 1

Study sample characteristics and suicide screening measures

Sample characteristics

n (%)

Age in years (mean, range)

36.4 (20-57)


28 (44%)

History of prior suicide attempt

24 (48%)

Existing psychiatric comorbidity

47 (96%)

Suicide screening measures

Becka suicide intent scale (mean, range, SD)

28.60 (12-36, 5.47)

PHQ-9b score (mean, range, SD)

22.74 (10-27, 3.24)

SAD PERSONSc scale (mean, range, SD)

7.24 (4-10, 1.61)

ED provider clinical impression scored

Nurse (mean, range, SD)

5.34 (1-10, 2.83)

Attending physician (mean, range, SD)

5.86 (1-10, 2.46)

Resident physician (mean, range, SD)

7.16 (1-10, 2.34)

a Scaled from 0 to 38, higher scores indicating higher suicide intent.

b Scaled from 1 to 27, higher scores indicating higher suicide risk.

c Scaled from 0 to 10, higher scores indicating higher suicide risk.

d Scaled from 1 to 10, higher scores indicating more likely need for admission to psy- chiatric facility.

1682 B.P. Chang, T.M. Tan / American Journal of Emergency Medicine 33 (2015) 16801683

Table 2

Areas under ROC curves, by screening tool and near-term outcome

Screening measure Outcome

Within-ED adverse event Admission to psychiatric facility Prolonged (N5 d) hospitalization

AUC (95% CI)


AUC (95% CI)


AUC (95% CI)



0.44 (0.27-0.61)


0.43 (0.25-0.60)


0.34 (0.14-0.54)



0.37 (0.21-0.52)


0.44 (0.27-0.61)


0.43 (0.22-0.65)



0.45 (0.28-0.62)


0.72 (0.67-0.87)


0.76 (0.61-0.91)



0.52 (0.35-0.69)


0.65 (0.49-0.81)


0.56 (0.37-0.76)



0.65 (0.49-0.81)


0.59 (0.41-0.77)


0.71 (0.51-0.92)



0.55 (0.38-0.71)


0.52 (0.36-0.69)


0.53 (0.34-0.72)



0.44 (0.27-0.60)


0.51 (0.34-0.68)


0.45 (0.25-0.65)


a P value for test of null hypothesis that true AUC is equal to 0.5.

b Receiver operating characteristic curve for n = 47, all others for n = 50.

clinical impressions had AUCs close to or below 0.5 for predicting all 3 near-term outcomes.

Receiver operating characteristic curve analysis was performed for the SAD PERSONS scale and for nurse and attending physician clinical impression. Taking a balanced approach maximizing sensitivity without overly sacrificing specificity, cutoff values were chosen for “positive” and “negative” screenings. For SAD PERSONS, a positive screen for values of 7 or higher corresponded to sensitivity and specificity of 63% and 29% for within-ED adverse events, 77% and 39% for psychiatric ad- mission, and 92% and 41% for prolonged psychiatric hospitalization. For nurse clinical impression, a positive screen for values of 4 or higher corresponded to sensitivity and specificity of 84% and 39% for within-ED adverse events, 68% and 29% for psychiatric admission, and 77% and 32% for prolonged psychiatric hospitalization. For attending physician clini- cal impression, a positive screen for values of 5 or higher corresponded to sensitivity and specificity of 68% and 35% for within-ED adverse events, 64% and 32% for psychiatric admission, and 69% and 35% for prolonged psychiatric hospitalization.

The results of logistic regression to calculate odds ratios relating screening measure and near-term outcomes are reported in Table 3. At the ? of .10 level of significance, nursing impression was correlated with adverse ED events, and the SAD PERSONS scale was correlated with prolonged psychiatric hospitalization. Patients receiving predic- tion scores of 4 or higher from their nurse had 3.37 times the odds of re- quiring Psychotropic medications in the ED, a security intervention, or physical restraints. Patients with SAD PERSONS scores of 7 or higher had 8.18 times the odds of being admitted to a psychiatric facility for more than 5 days. Attending physician clinical impression using a cutoff score of 5 of 10 was not associated with any near-term outcomes. We also adjusted for years of experience using a multiple regression model including years of experience together with provider clinical im- pression scores. In each case, years of experience was not statistically significant and also did not meaningfully change the significance or the odds ratios of the nurse or attending physician clinical impressions. At the ? of .05 level, none of the screening measures were significantly associated with higher odds of near-term outcomes.


A prospective observational study of 50 patients with SI in the ED was performed to study the relationship between suicide screening tools and their relationship to near-term adverse events and psychiatric admission. Overall, our data found that none of the suicide screening in- struments significantly predicted near-term outcomes. Within the con- text of the ED, a pressing question for the emergency physician is the following: within a population of individuals endorsing some form of SI, who among this population is at highest risk? Our data suggest that the existing suicide screening instruments frequently used in the ED are unable to distinguish patients with regard to that question. Although none of the measures showed significance, it appeared that the SAD PERSONS scale approached significance and warrants further investiga- tion in a more powered study. In addition to the lack of strong Predictive power for these instruments, we also found that clinician impression was a poor predictor of Negative outcomes. Of note, among clinicians, it was found that nurses, regardless of experience, were most predictive compared to physicians. This may be driven by the comparatively lon- ger periods of face-to-face interaction with patients that is often re- quired with nursing care, which may help nurses get a more nuanced view of the patient’s clinical course.

Several implications can be drawn from this study. The first is from the standpoint of near-term risk prediction; existing instruments may not be as useful in aiding suicide risk stratification for emergency physi- cians. Our data suggest a need for the development of ED-based screen- ing tools for patients with SITB. With a more robust sample size, we may gain improved understanding of the specific relationship between some of the individual questions of the existing tools and use that data to con- struct a tool that aids emergency physicians in assessing near-term sui- cide risk and potential risk stratification. Future work may also integrate existing neuropsychological testing examining SITB and cognition. For example, work by Nock et al [11] found that patients with active SI had deficits on various attention and emotional tasks. Investigating the use of such tools in the ED may provide a useful complement to existing self-report instruments.

Table 3

Results of logistic regression analysis for association between suicide screening measures and near-term outcomes

Screening measure


Within-ED adverse event

Admission to psychiatric facility

Prolonged (N5 d) hospitalization

OR (95% CI, P value)

OR (95% CI, P value)

OR (95% CI, P value)


0.70 (0.21-2.36, .57)

2.20 (0.63-7.69, 0.22)

8.18 (0.96-69.71, .055)

Attending physician clinical impression

1.19 (0.35-4.02, .78)

0.83 (0.26-2.69, .76)

1.22 (0.31-4.74, .78)

Nurse clinical impression

Adjusted for years of experience

3.37 (0.81-14.05, .096)

0.86 (0.25-2.89, .80)

1.60 (0.37-6.90, .53)

Attending physician clinical impression

1.19 (0.35-4.04, .78)

0.83 (0.25-2.74, .76)

1.23 (0.31-4.78, .77)

Nurse clinical impression

3.17 (0.75-13.4, .12)

0.61 (0.16-2.39, .48)

1.34 (0.29-6.13, .70)

Abbreviation: OR, odds ratio.

B.P. Chang, T.M. Tan / American Journal of Emergency Medicine 33 (2015) 16801683 1683


This study was conducted at a single academic medical center and may not be generalizable to other ED settings. In addition, given the relatively small sample size of this pilot study, the power needed to obtain potentially significant results may have hidden possible associations. As a pilot study, this work was done to get a broad overview of the nature of the problem, and ongoing data collection will guide future power esti- mates and target sample sizes. In addition, although we attempted to in- clude common suicide screening tools used in the ED, many other suicide tools have been used previously in the ED. For our pilot, we focused on tools that were adopted across several clinical settings, did not require a subspecialist or extensive training, and could be completed quickly.

In attempting to define our outcome measures for our study, we sought to try to capture elements of what we considered any adverse outcome events associated with patients with SI in the ED. Although near-term death by suicide or suicide attempt would have been “ideal” outcomes, given the rarity of such events (no subjects in our study had any such events in the period of our study) we sought to operationalize the idea of adverse outcome events via other variables. Psychiatric admission and prolonged psychiatric admission seemed a relevant outcome for this question as presumably more serious SI pa- tients would require psychiatric hospitalization and prolonged psychi- atric admission. The inclusion of PRN medications given and security interventions in the ED was included as additional measures, based on past research in the psychiatric literature finding an association be- tween increased Psychomotor agitation and suicidality [12,13]. One concern is that agitation itself may have been a separate issue distinct from suicide risk. However, framed from the from the standpoint of looking at the relationship of the suicide screening measures and subse- quent near-term adverse outcome events, an understanding of any such relationships between the 2 would still be useful for clinicians working in the ED, regardless of whether the near-term adverse event was truly capturing acute suicidality or agitation. Future studies may focus on attempting to tease apart the issues of psychomotor agitation, agitation in general, and suicide risk toward creating a more nuanced under- standing of the operationalization of near-term suicide risk.

Our study focused on patients endorsing SI but not actual suicide at- tempt patients in the ED. Such patients were not included in this study as the goal of our study was to attempt to understand the predictive value in clinician intuition and suicide screening instruments for pa- tients in presumed low to moderate risk of near-term negative out- comes. Patients who were being evaluated for actual suicide attempt would be by definition very high-risk patients as their outcome measure (near-term risk) was already clearly established. The utility of using clinical intuition or suicide screening instruments for patients with known suicidal attempt would likely be of less clinical utility because the high-risk categorization of the patients was already established. However, as our study was focused on SI and not actual suicide attempt,

they may represent 2 different populations with different sensitivities and associations with clinical intuition and the screening instruments.


Our prospective observation study examined the effectiveness of various suicide instruments in predicting near-term patient outcome. The lack of strong relationships between these instruments, clinical in- tuition, and patient outcomes suggest that, to date, no reliable strong predictors for acute suicide risk stratification exist in the ED. Future studies devoted to understanding the complex processes underlying suicidality will help guide the development of near-term suicide instru- ments and risk stratification models.


The authors would like to thank Joel Park and Tanveer Mollah for their help in collecting the data and running the study in the ED.


  1. Borges G, Nock M, Haro K, Hwang I, Sampson NA, Alonso J, et al. Twelve-month prevalence of and risk factors for suicide attempts in the World Health Organization World Mental Health Survey. J Clin Psychiatry 2010;71:1617-28.
  2. Nock M, Hwang I, Sampson N, Kessler RC, Angermeyer M, Beautrais A, et al. Cross- national analysis of the associations among Mental disorders and suicidal behavior: findings from the WHO World Mental Health Surveys. PLoS Med 2009;6:e1000123.
  3. Ting S, Sullivan A, Boudreaux ED, Miller I, Camargo C. Trends in US emergency de-

    partment visits for attempted suicide and self-inflicted injury 1993-2008. Gen Hosp Psychiatry 2012;34:557-65.

    Linehan M, Comtois K, Brown M, Heard H, Wagner A. Suicide Attempt Self-Injury In- terview (SASII): development, reliability, and validity of a scale to assess suicide at- tempts and intentional self-injury. Psychol Assess 2006;18:303-12.

  4. Beck A, Steer R, Ranieri W. Scale for Suicidal Ideation: psychometric properties of a self-report version. J Clin Psychol 1979;44:499-505.
  5. Posner K, Brown G, Stanley B, Brent DA, Yershova KV, Oquendo MA, et al. The Co- lumbia Suicide Severity Rating Scale: initial validity and internal consistency find- ings from three multi-site studies with adolescents and adults. Am J Psychiatry 2011;168:1266-77.
  6. O’Connor E, Gaynes B, Burda B, Williams C, Whitlock E. Screening for suicide risk in primary care: a systematic evidence review for the U.S. Preventive Service Task Force. Agency for Healthcare Research and Quality; 2013[Apr. Report No: 13- 05188-EF-1].
  7. Kroenke K, Spitzer R, Williams J. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001;16:606-13.
  8. Beck A, Steer R, Rantieri W. Scale for Suicide Ideation: psychometric properties of a self-report version. J Clin Psychol 1988;44:499-505.
  9. Patterson W, Dohn H, Bird J, Patterson G. Evaluation of suicidal patients: the SAD PERSONS scale. Psychosomatics 1983;24:343-9.
  10. Nock M, Park J, Finn CT, Deliberto TL, Dour H, Banaji M. Measuring the suicidal mind: implicit cognition predicts suicidal behavior. Psychol Sci 2010;21:511-7.
  11. Akiskal H, Benazzi F, Perugi G, Rihmer Z. Agitated “unipolar” depression re- conceptualized as a depressive mixed state: implications for the antidepressant- suicide controversy. J Affect Disord 2005;85:245-58.
  12. Olgiati P, Serretti A, Colombo C. Retrospective analysis of psychomotor agitation, hy- pomanic symptoms, and suicidal ideation in unipolar depression. Depress Anxiety 2006;23:389-97.

Leave a Reply

Your email address will not be published. Required fields are marked *