Original Contribution

Application of San Francisco Syncope Rule in elderly ED patients

Regis Schladenhaufen DO, Steven Feilinger MD, Marc Pollack MD, PhD?, Ronald Benenson MD, Amy L. Kusmiesz MS

Department of Emergency Medicine, York Hospital, York, PA 17403-3676, USA

Received 18 July 2007; revised 30 October 2007; accepted 30 October 2007

Abstract

Objectives: The San Francisco Syncope Rule (SFSR) is a decision rule with the potential to identify patients at risk for serious outcomes within 7 days of the emergency department (ED) visit for syncope. The initial studies of the SFSR reported a high sensitivity and specificity for identifying patients, of all ages, with serious outcomes. Our objective was to determine if the SFSR can be safely and accurately applied to ED patients aged 65 and older with syncope or near-syncope.

Methods: A retrospective review of ED patients aged 65 years and older with syncope or near-syncope between January 2000 and August 2001 was performed. Charts were reviewed for evidence of SFSR risks for the ED visit and serious outcomes within 7 days of the ED visit.

Results: Of 773 subjects identified as having syncope or near-syncope, 517 subjects were included. There were 98 patients with serious outcomes. Twenty-three patients who were negative on SFSR had serious outcomes. The sensitivity and specificity of the SFSR were 76.5% (95% confidence interval [CI], 66.7%-84.3%) and 36.8% (95% CI, 32.2%-41.6%), respectively. The negative and

positive predictive values were 87.0% (95% CI, 80.9%-91.4%) and 22.1% (95% CI, 17.8%-26.9%), respectively.

Conclusions: In our cohort of elderly ED patients, the SFSR had a lower sensitivity and specificity. The SFSR may not be applicable to the elderly ED population. Future prospective validation is necessary before application to the ED elderly population.

(C) 2008

Presented at the Fourth European Congress on Emergency Medicine, October 2006 (oral presentation); the National Society for Academic Emergency Medicine Conference, May 2006 (oral presentation); and the PA Chapter, American College of Emergency Physicians annual meeting, April 2006 (poster presentation).

* Corresponding author. Tel.: +1 717 851 2450; fax: +1 717 851 3469.

E-mail address: [email protected] (M. Pollack).

Introduction

Background

Syncope is a common emergency department (ED) complaint with a variety of etiologies. It accounts for approximately 1% to 3% of ED visits and up to 6% of hospital admissions [1-6]. The more serious causes of syncope are generally seen in elderly patients, such as

0735-6757/$ - see front matter (C) 2008 doi:10.1016/j.ajem.2007.10.042

arrhythmias or myocardial infarctions (MIs) [7-9]. In addition, cardiogenic syncope doubles the risk of death and increases the risk of fatal and nonfatal cardiovascular events [10]. Therefore, the elderly are more likely to have serious outcomes after a syncopal event in comparison with a younger population and are more likely to have a cardiac etiology for syncope.

In 2004, Quinn et al [11] published the derivation of a clinical decision rule to aid in the disposition of patients presenting to the ED with syncope or near-syncope. This initial study evaluated 684 patients, but the study population had a large age range, from 10 to 102 years (mean, 62.1 years). Having an abnormal electrocardiogram (ECG), a complaint of shortness of breath, a Hematocrit level less than 30%, a systolic blood pressure less than 90 mm Hg, or a history of congestive heart failure were found to be sensitive predictors of serious outcomes within 7 days. When applying these criteria to their population, the rule had a sensitivity of 96.2% (95% confidence interval [CI], 92%-100%) and a specificity of 61.9% (95% CI, 58%-66%) for serious outcomes. The negative predictive value (NPV) was 99.2% (95% CI, 98%- 100%), and the positive predictive value (PPV) was 24.8% (95% CI, 20%-30%) for serious outcomes.

In May 2006, the same authors published a prospective validation study [12] in which they applied the rule to 713 patients with an age range of 6 to 99 years (mean, 61 years) using the same criteria. In this study, the authors looked for adverse outcomes over a 30-day period instead of a 7-day period as they had originally described. The rule was found to be 98% sensitive (95%, CI 89%-100%) and 56% specific (95% CI, 52%-60%). The NPV was 99.7%

(95% CI, 98%-100%), and the PPV was 15% (95% CI,

12%-20%). A recent prospective external validation of the San Francisco Syncope Rule (SFSR), of all age groups, using a 7-day serious-outcomes criterion found lower sensitivities and specificities than the original derivation and validation studies [13].

Importance

There is a large Economic burden and resource allo- cation associated with the evaluation of syncope and near- syncope [14,15]. The use of a clinical decision rule to help guide the decision for admission could decrease unneces- sary admissions and decrease the overall cost of the workup. However, such a rule must be safe for all popula- tions especially the elderly as they are at greatest risk for serious outcomes.

Goals of this investigation

The objective was to determine if the SFSR can be safely and accurately applied to patients aged 65 years and older who present to the ED with syncope or near-syncope.

Methods

Study design and setting

This was an institutional review board-approved, single- center, retrospective study of patients seen in the ED from January 2000 to August 2001. The study site is a community teaching hospital and level II trauma center, with an emergency medicine residency and an annual census of 61000.

Selection of participants

The study sample was identified from the hospital’s computerized database of all ED encounters. The reason for visit at triage was searched for the following keywords: syncope, near-syncope, faint, or passed out. In addition, the International Classification of Diseases, Ninth Revision (ICD-9) code 780.2 (syncope and near-syncope) was searched by both final ED diagnoses and admitting diagnoses (if admitted). The top 5 ED diagnoses and the top 3 admitting diagnoses were reviewed for the ICD-9 code. Exclusion criteria included patients with no documenta- tion of syncope or near-syncope on chart review, out-of-state residents, and patients whose symptoms for the visit were due to head trauma, seizure, altered mental status, or intoxication. Patients with incomplete data or uncertain outcomes were excluded. If, however, a patient was positive for any of the SFSR risk factors, that patient was included as positive on SFSR, even if another risk factor of the SFSR was missing. For a patient to be negative on SFSR, all risk

factors had to be documented and negative.

Chart review methodology recommended by Gilbert et al

[16] was followed. Two reviewers collected data from each patient’s medical record using a structured data collection form (Appendix A). The 2 reviewers were study authors (RS, SF) and were not blinded to the Study objectives. A sample of 10 patient charts were abstracted for 30 data points per chart by both reviewers and evaluated for interrater reliability. Data points (variables) were defined by all 4 authors before the data collection process began. The overall ? was more than .99. Each individual ? was more than .95 except for ECG interpretation, which showed a ? of 0.23. Based upon the initial review, specific criteria for abnormal and normal ECGs were determined. Questionable ECGs were discussed by the 2 abstractors during data collection.

Reviewers flagged records for secondary review when questions of a data element abstraction arose. Questions were resolved by consensus of the study authors. The patient’s ED record, admission history and physical examination, and prior discharge summaries were reviewed for elements of the SFSR. The cardiologist’s official interpretation of the ECG was reviewed and compared with previous ECGs when considered abnormal. A nonsinus rhythm or new ECG changes were considered abnormal. A

Characteristic n (%)

Age, mean 78.8 Female 282 (54.5%) Admitted 312 (60.3%) Patients with serious outcomes 98 a Arrhythmia 69 (63.8%) Return for hospitalization 18 (16.7%) MI 12 (11.1%) Death 5 (4.6%) Pulmonary embolism 2 (1.9%) Cerebral vascular accident 2 (1.9%)

a Some patients had more than 1 serious outcome.

normal ECG demonstrated no change from a prior ECG. If no prior ECG was available, a normal ECG must have a rate between 50 and 100, no nonspecific changes, a normal axis, normal intervals, and no ectopy.

Table 1 Characteristics of the study population (N = 517)

The ED record and the admission history and physical exa- mination results were reviewed for the presence of shortness of breath or congestive heart failure. Hematocrit level was reviewed for each patient and was determined to be less than 30% or at least 30%. Triage and/or prehospital systolic blood pressure were reviewed for each patient, and the latter was determined to be less than 90 mm Hg or at least 90 mm Hg.

Outcome measures

Death, MI, arrhythmia, pulmonary embolism, stroke, subarachnoid hemorrhage, significant hemorrhage, or “any condition causing or likely to cause a return ED visit and

hospitalization for a related event” within 7 days were considered serious outcomes. Definitions for serious out- comes described in the initial derivation study by Quinn and colleagues [1] were used.

Patients with a hospital admission of at least 7 days with no record of a defined serious outcome in their discharge summary were considered to not have a serious outcome.

Death was confirmed by documentation in the medical record. Criteria for MI included ST elevation, new Left bundle branch block, or elevation of troponin I. Arrhythmia was defined as a rhythm that had the potential to be the cause of a syncopal or Near-syncopal event such as a tachyarrhythmia, bradyarrhythmia, sick sinus syndrome, or high-degree heart block. Pulmonary embolism was confirmed by positive results on CT scan of the chest, ventilation-perfusion scan, or angiography. Stroke and subarachnoid hemorrhage were confirmed with documentation in the medical record and supported by neuroimaging. Significant hemorrhage was considered to be any source of bleeding that required a transfusion. Outcomes were only considered to be serious if they appeared to be related to the syncopal event.

For patients who had an admission length of less than 7 days or who were discharged from the ED, records were reviewed from subsequent inpatient and outpatient visits, and data were reviewed for evidence of the serious outcomes. Patients who had no subsequent visits were excluded.

Primary data analysis

Sensitivity and specificity were calculated for the study population. Positive and negative predictive values were also

Fig. 1 The number of patients with and without serious outcomes along with the results of the use of SFSR.

Table 2 Performance of the SFSR in an elderly population
	No. of patients	7-d Serious outcomes
SFSR+	340	75	Sensitivity = 76.5%	PPV = 22.1%
SFSR-	177	23	(95% CI, 66.7%-84.3%) Specificity = 36.8%	(95% CI, 17.8%-26.9%) NPV = 84.0%
			(95% CI, 32.2%-41.6%)	(95% CI, 80.9%-91.4%)
Total	517	98
SFSR+ indicates patients with at least 1 SFSR risk; SFSR-, patients in whom all SFSR risks were negative.

determined with 95% CIs. The impact of missing data was assessed by imputing results for the patients with incomplete data. Patients with incomplete data were added back to the study population as true positives, true negatives, false- positives, or false-negatives. The sensitivity, specificity, PPV, and NPV were then calculated for each of these possibilities. A ? calculation was used to determine interrater reliability of the 2 chart reviewers. Data were analyzed using SPSS (version 14.0; SPSS, Chicago, Ill).

Results

Characteristics of study subjects

There were 103239 ED visits during the study period, of which 21293 were patients aged at least 65. Our search criteria identified 773 ED Patients with syncope or near- syncope. One hundred thirty-four were determined to not have syncope or near-syncope after the chart reviews, leaving 639 patients with syncope. These 639 patients represent 3% of the elderly ED patients. An additional 122 patients were excluded: 47 had incomplete ED records, and

75 had uncertain outcomes. The final study population consisted of 517 patients with a mean age of 78.8 years. The characteristics of the final study population (517) are described in Table 1.

Main results

Of the 517 patients, 98 had 108 serious outcomes (Fig. 1). Serious outcomes included 69 arrhythmias, 18 returns for

Table 3 Outcomes for 23 patients with serious outcomes and a negative SFSR

Adverse outcome	Incidence of adverse outcome a
Arrhythmia	17 (73.9%)
Pacemaker/defibrillator	11 (64.7%)
Return for hospitalization	6 (26.1%)
MI	1 (4.34%)
Cerebral vascular accident	1 (4.34%)
a Two patients had more than 1 serious outcome.

hospitalization, 12 MIs, 5 deaths, 2 Pulmonary embolisms, and 2 cerebral vascular accidents (Table 1). The sensitivity and specificity of the SFSR for the study population were 76.5% (95% CI, 66.7%-84.3%) and 36.8% (95% CI, 32.2%-

41.6%), respectively (Table 2). The NPV and PPV were 87.0% (95% CI, 80.9%-91.4%) and 22.1% (95% CI, 17.8%-

26.9%), respectively (Table 2).

Twenty-three patients with a negative SFSR had a 7-day serious outcome. Of these, 17 patients had arrhythmias (Table 3). Two patients had 2 serious outcomes, both an arrhythmia and a return for hospitalization. Of the 17 patients with arrhythmias, 11 had pacemakers/defibrillators that were inserted in the hospital.

Further analysis of the data was required because we excluded 122 patients with syncope from the initial analysis because of incomplete data or no follow-up. We added these patients back with varying assumptions and recalculated the sensitivity, specificity, and predictive values.

The least optimistic scenario assumes that all 122 patients were false-negatives (negative on SFSR with a serious outcome). This would yield a sensitivity of 34.1% (95% CI, 27.9%-40.8%), a specificity of 36.8% (95% CI, 32.2%-

41.6%), an NPV of 51.5% (95% CI, 45.7%-57.3%), and a

PPV of 22.1% (95% CI, 17.8%-26.9%). The most optimistic scenario assumes that all 122 are true positives (positive on SFSR with a serious outcome). This would yield a sensitivity of 89.5% (95% CI, 84.5%-93.1%); a specificity of 36.8%

(95% CI, 32.2%-41.6%); an NPV of 87% (95% CI, 80.9%-

91.4%); and a PPV of 42.6% (95% CI, 38.1%-47.3%).

Discussion

Our study retrospectively evaluated the SFSR specifically for the elderly population and found lower sensitivity and specificity for 7-day serious outcomes than the original studies. The SFSR was originally derived “to help guide the decision to admit patients presenting with syncope.” [1] However, this derivation study published in 2004, as well as the prospective validation study published in 2006, included patients with syncope of all age groups. A decision rule from such a heterogeneous group (with ages 10 to 102 years) seems unlikely to be truly applicable because serious etiologies are age related [17]. Also, in the derivation

study, it was shown that the group with serious outcomes had a higher mean age (75 years) in comparison with the group without serious outcomes (61 years) [1].

An editorial accompanying the derivation study sug- gested possible selection bias for failure to enroll some of the sicker patients, which would result in their higher sensitivities [18]. A recently published prospective external validation study of the SFSR by Sun et al [13] demon- strated a lower sensitivity and specificity. Additional external validation studies reported lower sensitivity and specificity [19-21]. These external validation studies eva- luated all age groups.

Our findings in an elderly ED population found a lower sensitivity (76.5%) and specificity (36.8%). We found an NPVof 87.0%. This suggests that an elderly ED patient with syncope and a negative SFSR evaluation would still be at significant risk for a 7-day serious outcome. We identified 23 such patients. Seventeen (73.9%) had an arrhythmia, and 11 (64.7%) had a pacemaker or defibrillator inserted. Arrhyth- mias are a common serious cause of elderly syncope and can sometimes be predicted by ECG, but older age is an additional predictor [22]. All of our patients who were SFSR negative and had a serious outcome of arrhythmia (17 patients) had a normal ECG on ED presentation. Elderly patients who recall the event preceding syncope and had a history of arrhythmia are more likely to have a cardiac etiology of their syncope [23]. A normal ECG in the ED is not sufficient evaluation of an elderly patient with possible arrhythmia-caused syncope.

A concern of ours was the impact the excluded patients would have on our results. If we add back these excluded patients into the analysis, with the most optimistic assump- tion, the sensitivity is only 89.5%. Alternatively, if we add back the excluded patients with the most pessimistic assumption, the sensitivity is 34.1%. The real value is likely somewhere in between.

Limitations

There are several potential limitations of this study. The retrospective design potentially affects the case selection, data abstraction, and outcome. However, we were quite rigorous in our case selection criteria that included the reason for visit at triage, final ED diagnosis by ICD-9 code 780.2, and admitting diagnosis by ICD-9 code 780.2. Accuracy of coding is dependent on coders’ recognition of key words in the medical record. Errors in coding could have led to missed cases. The likelihood of missing the syncope diagnosis is reduced by the fact that the ICD-9 search included the top 5 ED diagnoses and the top 3 admitting diagnoses.

Accuracy of the data abstraction is another potential problem of a retrospective design. We did asses the interrater reliability of the chart abstraction by the 2 reviewers (? = 0.99). However, we did not specifically evaluate the

interrater reliability of whether patients met the exclusion criteria or of outcomes ascertainment.

One hundred twenty-two patients were excluded from the study because of incomplete data or incomplete outcomes. A sensitivity analysis was performed by adding those patients to the study population under varied assumptions. By recalculating with those scenarios, we were able to estimate maximum and minimum sensitivity and specificities for our population.

The SFSR would not necessarily be applied to patients with an obvious reason for admission. Although our chart review was of a dictated or handwritten note, we did not attempt to determine which patients had an obvious reason for admission. By including patients with an obvious reason for admission, we may have artificially increased the true- positive rate, thus overestimating the sensitivity of the SFSR in our study population.

The chart abstractors were also study authors and were aware of the goals of the chart abstraction; this potentially introduced bias.

Conclusion

This retrospective study of the SFSR as applied to elderly ED patients found lower sensitivity and specificity than previously reported for 7-day serious outcomes. Additional analysis accounting for excluded subjects and making the most optimistic assumptions still had lower sensitivity and specificity. The most common missed serious outcome by the SFSR in the elderly was arrhythmia. Our findings are consistent with a recently published external validation study [13]. Our results suggest that future Prospective validation studies of elderly ED patients with syncope should be performed before application of the SFSR to this group of patients.

Appendix A. Supplementary data

Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j.ajem.2007. 10.042.

References

1. Quinn JV, Stiell IG, McDermott DA, et al. The San Francisco Syncope Rule vs. physician judgment and decision making. Am J Emerg Med 2005;23:782-6.
2. Day SC, Cook EF, Funkenstein H, et al. Evaluation and outcome of emergency room patients with transient loss of consciousness. Am J Med 1982;73:15-23.
3. Ammirati F, Colivicchi F, Santini M. Diagnosing syncope in clinical practice. Implementation of a simplified diagnostic algorithm in a

multicentre prospective trial-the OESIL 2 Study (Osservatorio Epidemiologico della Sincope nel Lazio). Eur Heart J 1999;21:935-40.

1. Crane SD. Risk stratification of patients with syncope in an accident and emergency department. Emerg J Med 2002;19:23-7.
2. Shen WK, Decker WW, Smars PA, et al. Syncope Evaluation in the Emergency Department Study (SEEDS), a multidisciplinary approach to syncope management. Circulation 2004;110:3636-45.
3. Sun BC, Emond JA, Camarga Jr CA. Characteristics and admission patterns presenting with syncope to U.S. emergency departments, 1992-2000. Acad Emerg Med 2004;11:1029-34.
4. Brignole M, Alboni P, Benditt D, et al. Guidelines on management, diagnosis and treatment of syncope-update 2004. Europace 2004;6: 467-537.
5. Strickberger SA, Benson DW, Biaggioni I, et al. AHA/ACCF scientific statement on the evaluation of syncope. J Am Coll Cardiol 2006;47: 473-84.
6. Manolis AS. Evaluation of patients with syncope: focus on age related differences. Part I-etiologies. ACC Curr J Rev 1994;3:13-6.
7. Soteriades ES, Evans JC, Larson MG, et al. Incidence and prognosis of syncope. N Engl J Med 2002;347:878-85.
8. Quinn JV, Stiell IG, McDermott DA, et al. Derivation of the San Francisco Syncope Rule to predict patients with Short-term serious outcomes. Ann Emerg Med 2004;43:224-32.
9. Quinn JV, McDermott DA, Stiell IG, et al. Prospective validation of the San Francisco Syncope Rule to predict patients with serious outcomes. Ann Emerg Med 2006;47:448-54.
10. Sun BC, Zargaraff G, Shlamovitz G, et al. Validation of the “San Francisco Syncope Rule”. Ann Emerg Med 2007;43:420-7.
11. Nyman JA, Krahn AD, Bland PC, et al. The costs of recurrent syncope of unknown origin in elderly patients. Pacing Clin Electrophysiol 1999;22:1386-94.
12. Zaidi AM, Fitzpatrick AP. Investigation of syncope: increasing the yield and reducing the cost. Eur Heart J 2000;21:877-80.
13. Gilbert EH, Lowenstein SR, Koziol-McLain J, et al. Chart reviews in emergency medicine: where are the methods? Ann Emerg Med 1996; 27:305-8.
14. Kapoor W, Snustad D, Peterson J, et al. Syncope in the elderly. Am J Med 1986;80:419-28.
15. Gallagher EJ. Shooting an elephant. Ann Emerg Med 2004;43: 233-7.
16. Stracner DL, Kass LE. Validation of the San Francisco Syncope Rule. Acad Emerg Med 2003;10:87a-8a [abstract].
17. Fisher CM, Shapiro NI, Lipsitz L, et al. External validation of the San Francisco Syncope Rule. Acad Emerg Med 2005;12:127 [abstract].
18. Reed MJ, Newby DE, Coull AJ, Jacques KG, Prescott RJ, Gray AJ. The risk stratification of syncope in the Emergency Department (ROSE) pilot study: a comparison of existing syncope guidelines. Emerg Med J 2007;24(4):270-5.
19. Sarasin FP, Hanusa BH, Perneger T, Louis-Simonet M, Rajeswaran A, Kapoor WN. A risk score to predict arrhythmias in patients with unexplained syncope. Acad Emerg Med 2003;10(12):1312-7.
20. Kadri N, Chassagne P, Benichou J, Hellot MF, Delangre T, et al. The value of clinical characteristics from a standard questionnaire to discriminate cardiac from neurological syncope in the elderly. J Nutr Health Aging 2006;10(6):546-53.