Original Contribution

predictive validity of the Triage Risk Screening Tool for elderly patients in a Canadian emergency department^B

Jerome Fan MD*, Andrew Worster MD, MSc, Christopher M.B. Fernandes MD

Department of Emergency Medicine, McMaster University, Hamilton, Ontario, Canada L8N 3Z5

Received 8 November 2005; accepted 15 January 2006

Abstract

Purpose: We sought to externally evaluate the predictive validity of the Triage Risk Screening Tool (TRST) for elderly patients in a Canadian setting.

Methods: A prospective, observational cohort study of a convenience sample of patients more than 64 years old was assessed using the TRST before discharge. The Composite outcome of any emergency department (ED) revisit, hospital admission, or long-term care placement at 30 and 120 days was measured. Likelihood ratios (LRs) and 95% confidence intervals (CIs) were calculated.

Main Findings: Of 218 patients screened, 120 patients were enrolled. At 30 and 120 days, the positive LRs were 1.4 (95% CI, 0.9-2.0) and 1.4 (95% CI, 1.0-1.9), respectively. The negative LRs were 0.7

(95% CI, 0.4-1.3) and 0.7 (95% CI, 0.4-1.0), respectively.

Conclusion: The TRST cannot be used as a single diagnostic test to predict whether Canadian ED elders will have an ED revisit, hospital admission, or long-term care placement at 30 or 120 days.

D 2006

Introduction

Elderly patients often present to emergency departments (EDs) with complex medical and social problems, thereby requiring greater resources and investigations than younger

Presentation information: Accepted for presentation at the Annual Scientific Conference of the Society of Academic Emergency Medicine, New York City, New York, USA May 2005; Accepted for presentation at Annual Scientific Conference of the Canadian Association of Emergency Physicians, Edmonton, Alberta, Canada, June 2005.

^B Source of Support: St. Peter’s Hospital Regional Geriatric Program

($2,000) and Canadian Association of Emergency Physicians ($3,200). Neither funding sources were involved in the planning, conduct, Data interpretation, or manuscript drafting. None of the authors had any financial connections with the sponsors.

* Corresponding author. Tel.: +1 905 521 2100×76207; fax: +1 905

521 2337.

ED patients [1]. Descriptive studies have revealed that after an index ED visit, 10-20% of elderly patients return to the hospital within 1 month, of which, the majority subsequent- ly are admitted [2-4]. Furthermore, 8% of this ED elderly population will subsequently become frequent ED utilizers, with more than 3 visits within 6 months [5]. In addition to high utilization rates, elderly ED patients face a 6-month risk of up to 10% mortality and 15-20% risk of increased functional dependence [6,7].

Multiple studies have examined the risk factors that predispose elderly patients to hospital admission or ad- verse health outcomes [3-6,8-11]. These include age, ac- tivities of daily living, isolation or lack of support, poor health, heart disease, diabetes, depression, recent Hospital visits or admissions, and polypharmacy [4,5]. Most current Screening tools to identify elderly patients at risk of adverse

0735-6757/$ – see front matter D 2006 doi:10.1016/j.ajem.2006.01.015

events are labor-intensive, complex, and not validated for ED use [12-15].

Meldon et al [16] prospectively derived and validated a simple 5-question screening tool called the Triage Risk Screening Tool (TRST) for elderly (N64 years old) ED patients. This tool was designed to predict resource utilization defined as ED revisits, hospital admission, and long-term care (LTC) placement at 30 and 120 days after an ED presentation. They determined that a TRST score z2 defined patients as high risk for the above outcomes. This tool has been prospectively tested in 2 American academic EDs but not externally validated for routine clinical use [16]. The objective of this study is to determine the predictive validity of the TRST in a Canadian elder ED population.

Methods

This study was formally reviewed and approved by the local Review Ethics Board.

Study design

This was a prospective, observational study of a convenience sample of ED patients more than 64 years old.

Setting and selection of participants

During a 1-month period in 2004, patients were screened at an academic ED with 29000 visits annually in a city with a population of 491 000, of which 15% are older than the age of 64 years [17]. Patients were excluded if they were residents of an LTC facility, previously enrolled in this study, or if cognitively impaired and simultaneously had no available proxy to answer the TRST screening questions. Cognitive impairment was defined as failure to be oriented to person, place, and time, or failure to follow a simple 3-step command (receive a piece of paper, fold it in half, and hand it back). Patients were excluded from follow-up if they were admitted to a hospital on the index visit.

Data collection and processing

ED nurses were trained to use the TRST over a 4-month period using a single 30-minute didactic session and receive feedback during the first week of study enrolment. The TRST assesses 5 risk factors: cognitive impairment, Self-reported difficulty in walking or transferring, the use of 5 or more medications, an ED visit within the previous 30 days or a hospital admission within the previous 90 days, and bED nurse concerns.Q ED nurse concerns is defined as presence of suspected abuse/neglect/self-neglect/exploitation, medica- tion noncompliance if using fewer than 5 medications, suspected substance abuse, problems meeting instrumental activities of daily living, and other concerns. Problems meeting instrumental activities of daily living is defined as difficulties with any of the following: obtaining prescriptions, obtaining food, transportation, cleaning the house, or

personal hygiene. The presence of a risk factor was given 1 point, with the TRST score ranging from 0 to 5 [16].

During the enrollment period, the nurses screened all eligible patients from Monday to Friday, 8 am to 6 pm, using a checklist containing the TRST questions. After providing a standard information sheet explaining the study protocol to patients and/or their proxy, a research assistant obtained a written consent that included subsequent access to medical records. Physicians were blinded to the results of the screen- ing tool. Patient care and disposition were up to the judgment of physicians according to usual standard of practice.

Patient identifiers (name, hospital identification number, provincial health insurance number, age, sex, martial status, Canadian Triage and Acuity Scale [CTAS] level, and presence or absence of ambulance arrival) as well as ED visits and hospital admissions were extracted from the hospital administrative database and charts for all consent- ing patients with completed TRST tool forms. A second, community-wide database was accessed to provide infor- mation on LTC placements. These databases and charts were examined for 30- and 120-day outcomes from each patient’s index ED visit date by a researcher blinded to the TRST screening results.

Outcome measures

The primary outcome was the composite of any ED revisit, hospital admission, or LTC placement at 30 and 120 days. The secondary outcomes were the individual compo- nents of the primary outcome for each time interval. ED revisit was defined as any unscheduled return for direct ED care. Hospital admission was defined as any unscheduled hospital admission. LTC placement was defined as any permanent change of residence into an LTC facility. Similar to the original derivation study, a TRST score z2 was used to categorize patients as high risk for the occurrence of the primary outcome.

Data analysis

Using data from the original derivation paper (where 40.2% of patients with a TRST score z2 and 18.1% of patients with a TRST score b2 had a 120-day primary outcome event) coupled with a 1-sided alpha of 0.05 and power of 0.8, we calculated the need for a sample size of approximately 74 patients in each category (low vs high risk) [16]. The summary measures are reported as likelihood ratios (LRs) with corresponding 95% confidence intervals (CIs).

Descriptive statistics were used to describe the following demographic characteristics: age, sex, marital status, proxy use, arrival by ambulance, ambulatory at registration, and CTAS level (ordinal ranking from 1 [resuscitation] to 5 [nonurgent]).

We calculated the LRs positive for an event at 30 and 120 days for all 3 outcome possibilities and the composite outcome in those with a score z2 and the LRs negative for an event in those with a score b2 for the same times. In

cases where the categorical values were 0, 0.5 was added to each cell for the summary measure calculations. All summary measures are reported with 95% CIs where applicable. Death on a repeat ED visit or subsequent hospital admission or LTC placement counted as an event for the location at which it occurred, that is, ED, hospital floor, LTC facility. Death in the community (at home) did not count as an event because this screening tool was designed for predicting resource utilization, not ad- verse outcomes; hence, these cases were excluded from the analysis. Statistical analyses were performed using SPSS Statistical Software version 11.0.1 (SPSS Inc, Chicago, Ill).

Table 2 Positive and negative LRs for 30- and 120-day

outcomes

See the Appendix A for the 2 x 2 tables used in calculating the

above LRs.

^a Composite outcome of ED revisit, hospital admission, or LTC placement.

Time	Outcome	LR + (95% CI)	LR — (95% CI)
30 d	Compositea	1.36 (0.91-2.04)	0.67 (0.35-1.30)
	ED revisit	1.31 (0.78-1.81)	0.72 (0.35-1.22)
	Hospital admission	1.57 (0.84-2.00)	0.48 (0.14-1.16)
	LTC placement	1.03 (0.11-1.96)	0.98 (0.11-1.86)
120 d	Compositea	1.37 (0.95-1.87)	0.70 (0.44-1.05)
	ED revisit	1.31 (0.89-1.81)	0.74 (0.46-1.11)
	Hospital admission	1.54 (1.01-2.01)	0.55 (0.26-0.99)
	LTC placement	1.81 (0.74-2.10)	0.27 (0.03-1.25)

Results

A total of 218 patients were screened using the TRST tool, of which, 193 patients consented and 73 excluded: 7 were currently LTC residents and 66 were admitted to a hospital on their index visit. This resulted in a total of

120 enrolled participants. The enrolled patient character- istics and TRST score distributions are found in Table 1. There were 60 high-risk and 60 low-risk patients in the study group.

The LRs for all outcome possibilities are listed in Table 2. Using a cutoff level greater than or equal to 2, none of the 95% CIs of the positive or negative LRs for either the

Table 1 Baseline study cohort characteristics Characteristics n (%)

Age (y)
65-69	21 (17.5)
70-74	30 (25.0)
75-79	35 (29.2)
80-84	23 (19.1)
N84	11 (9.2)
Male gender	53 (44.1)
Married	70 (58.3)
CTAS level
1	0 (0.0)
2	18 (15.0)
3	61 (50.8)
4	37 (30.8)
5	4 (3.3)
Ambulatory at registration	78 (65.0)
Arrival by ambulance	92 (76.6)
Proxy used	13 (10.8)
TRST score
0	29 (24.17)
1	31 (25.83)
2	40 (33.33)
3	16 (13.33)
4	4 (3.33)
5	0
Total	120 (100.0)

composite or secondary outcomes achieved clinically useful diagnostic levels of N10 or b0.1.

Discussion

This study demonstrates that the TRST is a poor diag- nostic test to predict ED revisit, hospital admission, or LTC placement at 30 and 120 days as witnessed by the failure of the LR CIs to achieve levels of clinical significance.

In Meldon’s original derivation and validation study, Meldon demonstrated that a 6-question TRST with a cutoff level of 2 had a sensitivity and specificity of 64% and 63%, respectively, at 30 days, and a sensitivity and specificity of 55% and 66%, respectively, at 120 days. Using logistic regression modeling, the authors removed the blives aloneQ item in the original tool because it did not add to its Predictive ability, creating the final 5-question TRST used in our study. Unfortunately, the authors did not recal- culate the revised tool’s sensitivity and specificity. Luckily, their published data allowed us to calculate a sensitivity and specificity of 58% and 69%, respectively, at 30 days, and 48% and 72% at 120 days for the 5-question TRST tool. The positive and negative LRs are therefore

1.9 and 0.6, respectively, at 30 days, and 1.7 and 0.7 at

120 days.

From these calculations, it can be seen that the LRs are comparable to our validation study. However, Meldon concludes that the TRST is clinically useful contrary to our interpretation. These diametrically opposing conclu- sions are a result of the subjective valuation of the information that is obtained from using the TRST. This is best illustrated with an example: If the pretest probability (prevalence) of an elderly ED patient of having the composite outcome of ED revisit, hospital admission, or LTC placement is 20%, then after using the TRST tool, the posttest probability is 25% if the patient has a score z2, or 15% if the score is b2 using a positive and negative LR of

1.4 and 0.7, respectively.

Therefore, in a resource-rich environment where cost is not an issue, this slight risk separation may compel one to use this tool to target this bslightlyQ higher risk group with an intervention. However, in a resource-scarce environment where costs are much higher, the impact of TRST screening poses significant Opportunity costs to other elements in our EDs. Not only would screening require nursing time, a positive screen requires us to intervene on many patients who will not have an outcome event, and a negative screen will miss many who will have an outcome event. Furthermore, the use of expensive interventions (eg, comprehensive ED Geriatric evaluations) would increase the opportunity cost by diverting necessary resources away from other ED operations.

Despite the failure of the TRST in predicting these composite outcomes“>important outcomes, it is still necessary to continue efforts to develop reliable and valid measurement tools to predict functional outcomes and resource utilization. These tools are important to help clinicians and health care workers target the right resources for the right patient.

Future derivation studies of ED elderly Prediction tools for resource utilization behaviors should be based on the Anderson model of emergency medical service use [18]. A simple risk factors-based screening tool may not adequately capture the complex constructs that explain health care utilization behaviors. The Anderson model takes into account the Predisposing factors specific to primary care (sociodemographics, health beliefs), enabling factors spe- cific to primary care (family resources, community resour- ces), and perceived and evaluated health care need in explaining ED versus primary medical service use. The systematic literature review of McCusker et al [18] supports the use of this model in explaining why elderly patients use the ED. Having a solid theoretical foundation coupled with rigorous scale development methodology as described by Streiner and Norman [19], a reliable and valid prediction tool can then be developed for both clinical and research purposes for this complex ED population.

The first potential criticism of this study is that it used a weekday/daytime convenience sample; however, this was deliberately chosen because it would simulate the most feasible application of general ED elderly screening for potential geriatric nurse interventions in our community setting, thereby assessing the validity of the TRST in the manner in which it would be used clinically. Therefore, a comparison between those who were screened versus not screened was deemed not necessary.

A second criticism of this study was the potential missed visits to other hospitals providing short-term care because of the nature of only having administrative database follow-up. This limitation may be minimal because there are significant distances requiring automotive access and highway driving capabilities that separate the city within which this study was conducted from surrounding cities with hospitals providing short-term care. Transportation access to medical care is often cited as a major barrier to health care for the elderly.

A third criticism of this study was our inability to reach our desired sample size because of financial constraints. However, despite this, our CIs do not reach values of clinical significance. Therefore, the conclusions of this study would remain the same even if the requisite sample size was achieved.

Conclusion

The TRST cannot be used as a single diagnostic test to predict whether Canadian ED elders will have an ED revisit, hospital admission, or LTC placement at 30 or 120 days.

Acknowledgments

Author contributions: JF, CF, and AW conceived the idea and designed the study. JF obtained research funding. JF conducted the study and performed data collection under the supervision of CF and AW. JF managed the data. JF and AW performed the statistical analyses. JF, CF, and AW interpreted the data. JF drafted the manuscript, and CF, and AW contributed to its revision. JF takes responsibility for the paper as a whole.

Extended thanks to the ED nurses and clerks of the Henderson General Hospital, Hamilton, and Karen Nekoda, Judy Lever, and Dr Robin Roberts for all their tireless work and advice with the conduct of this study.

Appendix A
Composite outcomes
30 days
TRST score	Yes	No	Subtotals
High risk (z 2)	11	49	60
Low risk (b 2)	6	54	60
Subtotals	17	103	120
120 Days
TRST score	Yes	No	Subtotals
High risk (z2)	24	36	60
Low risk (b2)	15	44	59a
Subtotals	39	80	119

a One patient was excluded from calculation because of death on day 40 at home.

ED revisit outcomes

30 days

TRST score	Yes	No	Subtotals
High risk (z 2)	10	49	59a
Low risk (b 2)	6	54	60
Subtotals	16	103	119

a One patient lost-to-follow-up after direct admission to hospital on day 2.

120 Days
TRST score	Yes	No	Subtotals
High risk (z 2)	21	37	58a
Low risk (b 2)	14	44	58b
Subtotals	35	81	116

a One patient lost-to-follow-up after direct admission to hospital on day 2. One patient died on day 40 after direct hospital admission.

b One patient died on day 40 at home. One patient was

lost-to-follow-up after direct hospital admission on day 82.

Hospital admission outcomes

30 days
TRST score	Yes	No	Subtotals
High risk (z 2)	6	53	59a
Low risk (b 2)	2	58	60
Subtotals	8	111	119
a One patient died on day 17 on an ED revisit. 120 Days
TRST score	Yes	No	Subtotals
High risk (z 2)	14	44	58a
Low risk (b 2)	6	53	59b
Subtotals	20	97	117

a Both patients died after hospital admission on an ED revisit (days 17 and 83).

b One patient died on day 40 at home.

LTC placement outcomes

30 days
TRST score	Yes	No	Subtotals
High risk (z 2)	0	57	57a
Low risk (b 2)	0	60	60
Subtotals	0	117	117

a One patient died on day 17 on an ED revisit. Two patients were lost-to-follow-up after hospital admissions on day 2.

120 Days

TRST score	Yes	No	Subtotals
High risk (z 2)	3	52	55a
Low risk (b 2)	0	58	58b
Subtotals	3	110	113
a Three patients	died on	days 17, 40,	and 83. Two

patients were lost-to-follow-up after hospital admission on day 2.

b One patient died on day 40 at home. One patient was

lost-to-follow-up after hospital admission on day 82.

References

1. Singal BM, Hedges JR, Rousseau EW, et al. Geriatric patient emergency visits, part I: comparison of visits by geriatric and younger patients. Ann Emerg Med 1992;21(7):802 – 7.
2. Friedmann PD, Jin L, Karrison TG, et al. Early revisit, hospitalization, or death among older persons discharged from the ED. Am J Emerg Med 2001;19:125 – 9.
3. McCusker J, Cardin S, Bellavance F, et al. Return to the emergency department among elders: patterns and predictors. Acad Emerg Med 2000;7(3):249 – 59.
4. Caplan GA, Brown A, Croker WD, et al. Risk of admission within

4 weeks of discharge of elderly patients from the emergency department–the DEED study. Age Ageing 1998;27:697 – 702.

1. McCusker J, Bellavance F, Cardin S, et al. Prediction of hospital utilization among elderly patients during the 6 months after an emergency department visit. Ann Emerg Med 2000;36(5):438 – 45.
2. McCusker J, Bellavance F, Cardin S, et al. Detection of older people at increased risk of adverse health outcomes after an emergency visit: the ISAR screening tool. J Am Geriatr Soc 1999;47:1229 – 37.
3. Hedges JR, Singal BM, Rousseau EW, et al. Geriatric patient emergency visits, part II: perceptions of visits by geriatric and younger patients. Ann Emerg Med 1992;21(7):808 – 13.
4. McCusker J, Bellavance F, Cardin S. Screening for geriatric problems in the emergency department: reliability and validity. Acad Emerg Med 1998;5:883 – 93.
5. Chin MH, Jin L, Karrison TG. Older patients’ health-related quality of life around an episode of emergency illness. Ann Emerg Med 1997;34:595 – 603.
6. Rowland K, Maitra AK, Richardson DA. The discharge of patients from an accident and emergency department: functional changes and risk of readmission. Age Ageing 1990;19:415 – 8.
7. Richardson DB. Elderly patients in the emergency department: a prospective study of characteristics and outcome. Med J Aust 1992; 157:234 – 9.
8. Sanger MA, Rudberg MA, Jalauddin M, et al. Hospital admission risk profile (HARP): identifying older patients at risk for functional decline following acute medical illness and hospitalization. J Am Geriatr Soc 1996;44:914 – 21.
9. Winograd CH, Gerety MB, Chung M, et al. Screening for fraility: criteria and predictors of outcomes. J Am Geriatr Soc 1991;39: 778 – 84.
10. Boult C, Dowd B, McCaffrey D, et al. Screening elders for risk of hospital admission. J Am Geriatr Soc 1993;41:811 – 7.
11. LaForge RG, Spector WD, Sternberg J. The relationship of vision and hearing impairment to one-year mortality and functional decline. J Aging Health 1992;4:126 – 48.
12. Meldon SW, Mion LC, Palmer RM, et al. A brief Risk-stratification tool to predict repeat emergency department visits and hospitalizations in older patients discharged from the emergency department. Acad Emerg Med 2003;10:224 – 32.
13. Hamilton District Health Council. Hamilton health system fact book. Hamilton (Ontario)7 Ministry of Health — City of Hamilton; 2002.
14. McCusker J, Karp I, Cardin S, et al. Determinants of emergency department visits by older adults: a systematic review. Acad Emerg Med 2003;10(12):1362 – 70.
15. Streiner D, Norman G. Health measurement scales: a practical guide to their development and use. 3rd ed. New York (NY)7 Oxford University Press Inc; 2004. p. 4 – 192.