Comparison of the Sequential Organ Failure Assessment, Acute Physiology and Chronic Health Evaluation II scoring system, and Trauma and Injury Severity Score method

for predicting the outcomes of intensive care unit trauma patients

Seong Youn Hwang MD ^a, Jun Ho Lee MD ^a,?, Young Hwan Lee MD ^a,

Chong Kun Hong MD ^a, Ae Jin Sung MD ^a, Young Cheol Choi MD ^b

^aDepartment of Emergency Medicine, Samsung Changwon Hospital, Sungkyunkwan University School of Medicine,

Changwon 630-522, South Korea

^bDepartment of Surgery, Samsung Changwon Hospital, Sungkyunkwan University School of Medicine, Changwon 630-522, South Korea

Received 17 March 2011; revised 6 April 2011; accepted 19 May 2011

Abstract

Purpose: The aim of this study was to assess the ability of the Sequential Organ Failure Assessment , Acute Physiology and Chronic Health Evaluation II scoring system, and Trauma and Injury Severity Score method to predict group mortality for intensive care unit (ICU) trauma patients. Methods: The medical records of 706 consecutive major trauma patients admitted to the ICU of Samsung Changwon Hospital from May 2006 to April 2010 were retrospectively examined. The SOFA and the APACHE II scores were calculated based on data from the first 24 hours of ICU admission, and the TRISS was calculated using initial laboratory data from the emergency department and operative data. The probability of death was calculated for each patient based on the SOFA score, APACHE II score, and TRISS equations. The ability to predict group mortality for the SOFA score, APACHE II score, and TRISS method was assessed by using 2-by-2 decision matrices and receiver operating characteristic curve analysis and calibration analysis. Results: In 2-by-2 decision matrices with a decision criterion of 0.5, the sensitivities, specificities, and accuracies were 74.1%, 97.1%, and 92.4%, respectively, for the SOFA score; 58.5%, 99.6%, and 91.1%, respectively, for the APACHE II scoring system; and 52.4%, 94.8%, and 86.0%, respectively, for the TRISS method. In the receiver operating characteristic curve analysis, the areas under the curve for the SOFA score, APACHE II scoring system, and TRISS method were 0.953, 0.950, and 0.922, respectively.

Conclusion: The results from the present study showed that the SOFA score was not different from APACHE

II scoring system and TRISS in predicting the outcomes for ICU trauma patients. However, the method for calculating SOFA scores is easier and simpler than APACHE II and TRISS.

(C) 2012

* Corresponding author. Tel.: +82 055 2906560; fax: +82 055 2906117.

E-mail address: [email protected] (J.H. Lee).

0735-6757/$ - see front matter (C) 2012 doi:10.1016/j.ajem.2011.05.022

Introduction

Illness severity scoring systems have become important tools for studying patient outcomes. Early efforts to measure the efficacy of trauma centers and Trauma systems assessed the rates of preventable mortality. With increasing incidence of trauma, a multitude of statistical models has been developed in an attempt to accurately predict outcomes for trauma patients. Instruments for scoring illness severity, such as the Acute Physiology and Chronic Health Evaluation (APACHE) II and III or the Simplified Acute Physiology Score II, are widely used for critically ill patients. More recently, the Sequential organ failure assessment scoring system was developed and validated.

Trauma scoring systems were initially developed to triage patients in the field and needed to be straightforward and user friendly. There are several systems such as the triage Revised Trauma Score, Triage Score, and Trauma and Injury Severity Score (TRISS) for predicting the severity of trauma patients’ conditions along with patient outcomes. The Major Trauma Outcome Study and TRISS methodology were a major advance toward establishing an objective measurement of trauma center care with stratification according to the magnitude of patients’ injuries [1,2]. However, there are few methods for precisely and easily predicting the outcomes of intensive care unit (ICU) trauma patients. The purpose of this study was to assess the ability of the SOFA score, APACHE II scoring system, and TRISS method to predict group mortality among ICU trauma patients.

Materials and methods

Study design

This was a retrospective cohort study conducted from May 2006 to April 2010.

Study setting

This study was conducted at a regional emergency center affiliated with an academic university hospital in Changwon, South Korea.

Ethics with study approval

A retrospective chart review was performed after this study was received institutional review board approval from the Samsung Changwon Hospital, Sungkyunkwan University School of Medicine; the consent form was exempted.

Data collection

The medical records of 796 consecutive ICU trauma patients admitted to the emergency center were carefully

examined. Among these, data from 706 patients were included in this research. Ninety patients were excluded from this study for the following reasons: (1) younger than 15 years (n = 32); (2) burn injuries (n = 19); (3) death within 24 hours of ICU admission (n = 13); and (4) missing data (SOFA, 8 missing values; APACHE II, 11 missing values; and TRISS, 7 missing values). Medical records were carefully reviewed for the following parameters: demo- graphic and clinical data, mechanism of injury, systolic and mean arterial blood pressure (mm Hg), heart rate, respiratory rate, body temperature, initial Glasgow Coma Scale score, arterial blood gas analysis (pH, PaO2, PaCO2, oxygen saturation, and base excess), FIO2, laboratory data (white blood cell count, hematocrit, platelet count, and serum levels of sodium, potassium, creatinine, and bilirubin), urine output, amount of vasopressor, presence of chronic diseases or immune-compromised state, revised trauma score, Abbreviated Injury Scales, injury severity score (ISS), SOFA score, APACHE II score, and TRISS. For the purpose of the study, 2 trauma team experts performed extensive chart and computer checks for Data completeness, accuracy, and consistency. The ISS was calculated using the Abbreviated Injury Scales 2005. The outcome parameter was 30-day mortality. Severe chronic illness included liver cirrhosis with portal hypertension, New York Heart Association class IV congestive heart failure, chronic respiratory disease, end-stage renal disease, and an im- mune-compromised state (eg, leukemia, lymphoma, or AIDS). Typically, the most abnormal values for each parameter are observed within the first 24 hours after ICU admission. These are noted along with the location of the patient at the time of notation (eg, emergency, recovery, or operating room). The SOFA score, APACHE II score, and TRISS were calculated as previously described [3-5].

Statistical analysis

Data were analyzed using SPSS (SPSS 18.0 version; SPSS, Inc, Chicago, IL) and MedCalc (MedCalc 10.0 version; MedCalc, Inc, Mariakerke, Belgium). Univariate analysis was performed with the Student t test for continuous variables or the ?2 test for categorical variables. All variables found to be significant by univariate analysis underwent multivariate logistic regression analysis. Calibration was assessed using the Hosmer-Lemeshow goodness-of-Fit test, which compares the number of observed and predicted deaths in risk groups for the entire range of probabilities of death. The area under the receiver operating characteristic curve was calculated to determine the ability of the scores to discriminate using mortality as an independent variable. The differences between the sensitivities, specificities, and accuracies of these scores were determined by using the McNemar test. Type I error was corrected by the Bonferroni method for comparisons of 3 or more variables. Statistical significance was accepted at the P b .05 level.

3. Results Of the 706 trauma patients enrolled in the study, blunt trauma was the reason for 94.8% of the ICU admissions. The	Table 2 The mean and SD for the SOFA, APACHE II, and TRISS values Survivors Nonsurvivors P SOFA a 0.073 +- 0.134 0.722 +- 0.342 b.001
overall survival rate was 79.2%. Table 1 shows the variables	APACHE II a	0.071 +- 0.079	0.514 +- 0.236	b.001
that were found to be significantly different between the	TRISS	0.090 +- 0.167	0.566 +- 0.359	b.001

survivor and nonsurvivor groups. Compared with the nonsurvivors, survivors were generally younger patients (mean, 44.2 vs 50.1 years; P b .001), had a higher O₂ index (308.0 +- 109.8 vs 237.2 +- 138.6; P b .001), and had higher systolic blood pressure and GCS scale (Table 1). Nonsurvi- vors were also more likely to undergo an operation (P b .001). The probability of death estimates for the SOFA, APACHE II, and TRISS is shown in Table 2 for the survivors and nonsurvivors group. The overall methods were significant association between survivors and nonsurvivors with good probability of death. The Predictive power of the mortality estimates for each scoring system for the overall study population according to the SOFA, APACHE II, and TRISS is shown in Table 3. The sensitivity of SOFA was remarkably higher than that of APACHE II or TRISS (P b

^a Probability of death.

.001 vs SOFA). The specificity of SOFA was also higher than that of TRISS (P = .024) but lower than APACHE II (P b .001). The accuracy rate of SOFA was higher than that of TRISS (P b .001) but was not significantly different than that of APACHE II (P = .200).

For assessing goodness-of-fit, as measured by the Hosmer-Lemeshow ?2 statistic of predicted mortality risk, the calibration of SOFA (Hosmer-Lemeshow ?2 = 8.006, 8 degrees of freedom, P = .757), APACHE II (Hosmer- Lemeshow ?2 = 9.521, 8 degrees of freedom, P = .638), and TRISS (Hosmer-Lemeshow ?2 = 10.992, 8 degrees of freedom, P = .364).

Receiver operating characteristic curves for each system are shown in Fig. 1. The area under the curve (AUC) for each system was 0.953 for SOFA, 0.922 for TRISS, and 0.950 for APACHE II. Comparison of the AUCs showed that when compared with TRISS (P = .134 vs SOFA) and APACHE II (P = .736 vs SOFA), SOFA had as much Discriminatory power as these other 2 systems.

Table 1 Characteristics of survivors and nonsurvivors

4. Discussion

Regardless of the number of patients studied, prognostic estimates are still only estimates. Providing intensive medical care to individuals will always require experienced clinical judgment and careful integration of objective data with other relevant information such as an individual’s reaction to treatment and the personal wishes of the patient. Given the high patient mortality rates and long-term reimbursement shortfalls of trauma centers, it is important that trauma care providers continue to actively conduct rigorous studies that can help set uniform standards for judging institutional performance.

Several scoring systems have been developed in an attempt to accurately predict outcomes in trauma patients. Most scoring systems focus on mortality as the main outcome. The TRISS method was proposed by Champion et al [6] in the 1980s.

This method uses both anatomical and physiologic data and previously held the most promise for characterizing injury severity [3,7,8]. Trauma and Injury Severity Score has been the most commonly used method for calculating the Probability of survival for an individual trauma patient. In addition, this method calculates the probability of survival for a trauma patient using a formula that accounts for the mechanism of injury (blunt or penetrating), patient age (older or younger than 55 years), ISS, and revised trauma score. However, it had several limitations. For example, the complexity of this method has limited its role in initial Patient evaluation. Furthermore, the TRISS has been shown to be a poor predictor of survival after falls at a low level

Survivors (n = 559)		Nonsurvivors (n = 147)	P
Age (y)	44.2 +- 16.90	50.1 +- 18.50	b.001
Sex			.582
Male	423 (75.7%)	108 (73.5%)
Female	136 (24.3%)	39 (26.5%)
SBP	145.9 +- 31.10	92.5 +- 69.10	b.001
O2 index	308.0 +- 109.8	237.2 +- 138.6	b.001
GCS	12.90 +- 2.8	5.4 +- 3.6	b.001
Operation	317 (56.7%)	91 (61.9%)	b.001
Blunt trauma	525 (93.9%)	144 (98.0%)	.080
Values are expressed as the mean +- SD or frequencies. SBP indicates systolic blood pressure (millimeters of mercury).

Table 3 Sensitivities, specificities, and accuracy rates of the SOFA, APACHE II, and TRISS scoring systems for predicting mortality
	SOFA	APACHE II	TRISS
Sensitivity	109/147	86/147 ?	77/147 ?
	(74.1%)	(58.5%)	(52.4%)
Specificity	543/559	557/559	530/559 ?
	(97.1%)	(99.6%)	(94.8%)
Accuracy rate	652/706	643/706	607/706 ?
	(92.4%)	(91.1%)	(86.0%)
* P b .05 vs SOFA (McNemar test with Bonferroni correction).

Fig. 1 Receiver operating curves for predicting death according the SOFA, APACHE II, and TRISS scoring systems. The AUCs and 95% confidence interval for those indicators were 0.953 (0.937- 0.969) for SOFA, 0.950 (0.931-0.965) for APACHE II, and 0.922

(0.900-0.941) for TRISS.

[9,10]. The inability to distinguish between types of Penetrating injuries, such as gunshot wounds and stabbing injuries, has also been identified as a limitation of the TRISS methodology [9]. Norris et al [11] reported that TRISS needs to be updated for meaningful interpretation; modifications need to be made, and coefficients must be revised. Nevertheless, the TRISS methodology has become a standard tool for objective evaluation of the performance of trauma centers and identification of the records of individual patients for critical review. Therefore, the study included TRISS methodology in predicting mortality in ICU trauma patients.

The SOFA score was developed during a consensus conference organized by the European Society of Intensive Care and Emergency Medicine [4]. The SOFA calculates a summary value for the degree of dysfunction for 6 organs (respiratory, coagulation, liver, cardiovascular, central ner- vous system, and renal). Four levels of dysfunction are identified for each of the organ systems for the SOFA score. Organ dysfunction is associated with high rates of ICU morbidity and mortality; and as such, treatment of these disorders accounts for a high proportion of the ICU budget [12,13]. In this study, although SOFA scores were calculated on the initial day of admission, SOFA scores were useful for predicting the outcomes of ICU trauma patients. Further- more, this methodology is simple and easily compared with other scoring systems.

The APACHE II uses a point score based upon initial values of 12 routine physiologic measurements, patient age,

and medical history to provide a general measure of disease severity in a patient. This system is able to stratify a wide variety of patients according to prognosis because of the strong and consistent relationship between acute physiologic dysfunction and the risk of death due to acute illness [14-16]. In this study, APACHE II scores were useful for predicting the outcomes of ICU trauma patients. However, the APACHE II system is relatively complex, similar to the TRISS scoring system.

Vassar et al [5] found that both the TRISS and the APACHE II systems were both poor predictors of the risk for hospital death among ICU trauma patients. In our study, both APACHE II and TRISS were useful for predicting the outcomes of ICU trauma patients. However, this usefulness was restricted because these 2 methods were too complex for initial use in trauma patients being admitted to the ICU. On the other hand, SOFA scores were easily and simply calculated. The SOFA score is really a nominal scale with little or no internal consistency-merely providing a simple way for calibrating similarity based on consensus expertise. However, these data suggest that there is no difference between the SOFA score and APACHE II and TRISS in predicting the outcomes of ICU trauma patients. Although our conclusions are limited by the fact that this was a single- center study, the study highly recommend the SOFA score for predicting the outcomes of ICU trauma patients in the way that SOFA score was easily and simply calculated.

References

1. Champion HR, Copes WS, Sacco WJ, Lawnick MM, Keast SL, Bain LW, et al. The major trauma outcome study: establishing national norms for trauma care. J Trauma 1990;30:1356-65.
2. Boyd CR, Tolson MA, Copes WS. Evaluating trauma care: the TRISS method. J Trauma 1987;27:370-8.
3. Hannan EL, Farrell LS, Gorthy SFH, Bessey PQ, Cayten CG, Cooper A, et al. Predictors of mortality in adult patients with Blunt injuries in New York State: a comparison of the Trauma and Injury Severity Score (TRISS) and the International Classification of Disease, Ninth Revision based Injury Severity Score (ICISS). J Trauma 1999;47:8-14.
4. Vincent JL, Moreno R, Takala J, Willatts S, De Mendonca A, Bruining H, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. Intensive Care Med 1996;22: 707-10.
5. Vassar MJ, Wilkerson CL, Duran PJ, Perry CA, Holcroft JW. Comparison of APACHE II, TRISS, and a proposed 24-hour ICU point system for prediction of outcome in ICU trauma patients. J Trauma 1992;32:490-9.
6. Champion HR, Sacco WJ, Carnizzo AJ. Trauma score. Crit Care Med 1981;9:676-8.
7. Rutledge R, Osler T, Emery S, Emery S, Kromhout-Schiro S. The end of injury severity score (ISS) and the Trauma and Injury Severity Score (TRISS): ICISS, an international classification of disease, ninth revision based Prediction tool, outperforms both ISS and TRISS as predictors of trauma patients survival, hospital charges and hospital length of stay. J Trauma 1988;44:41-7.
8. Bouillon B, Lefering R, Vorweg M, Tiling T, Neugebauer E, Troidl H, et al. Trauma score systems: cologne validation study. J Trauma 1997;42:652-8.
9. Cayten C, Stahl W, Murphy J, Agarwal N, Byrne D. Limitations of the TRISS method for interhospital comparison: a multihospital study. J Trauma 1991;31:471-82.
10. Lee Kennedy R, Grant P, Blackwell D. Low-impact falls: demands on a system of trauma management, prediction of outcome, and influence of comorbidities. J Trauma Inj Infect Crit Care 2001;51:717-24.
11. Norris R, Woods R, Harbrecht B, Fablan T, Rhodes M, Morris J, et al. TRISS unexpected survivors: an outdated standard. J Trauma 2002;52: 229-34.
12. Tran DD, Groeneveld ABJ, Vander Meulen J, Nauta JJP, Strack Van Schijndel RJM, Thijs LG. Age, chronic disease, sepsis, organ system

failure, and mortality in a medical intensive care unit. Crit Care Med 1990;18:474-9.

1. Deitch EA. Multiple organ failure: pathophysiology and potential future therapy. Ann Surg 1992;216:117-34.
2. Knaus WA, Wanger DP, Draper EA. Relationship between acute physiologic derangement and risk of death. J Chronic Dis 1985;38:295.
3. Wilson RF, Gibson D, Percinel AK, Ali MA, Baker G, LeBlanc LP, et al. Severe alkalosis in critically ill surgical patients. Arch Surg 1972;105:197.
4. Shoemaker WP, Chang P, Czer L. Cardiovascular monitoring in postoperative patients. Crit Care Med 1979;7:237.