Brief Report

Automated vs manual triage for bioterrorist disaster:

a blinded crossover feasibility study comparing personal digital assistant to paper-based triage^B

Cathy L. Schell MD^a,*, Ron Wohl ARNP^b,c, Richard Rathe MD^d, William J. Schell^e

^aMedical Decisions Software, Inc., Earleton, FL 32631, USA

^bAlachua County Jail, Gainesville, FL 32609, USA

^cAmerican Red Cross, North Central Florida Chapter, Gainesville, FL 32609, USA

^dDepartment of Medical Informatics, University of Florida College of Medicine, Gainesville, FL 32610, USA

^eMedical Decisions Software, Inc. St. Pete Beach, FL 33706, USA

Received 3 February 2006; accepted 30 March 2006

Abstract

Introduction: This article reports results of a National Institutes of Health/National Library of Medicine Small Business Innovation Research-funded research grant comparing paper-based and automated Palm handheld computer disaster triage documentation.

Objectives: The aim of this study was to test the feasibility of automated handheld computer triage and compare it to handwritten triage.

Methods: A paired t test was used in an intraindividual, blinded, crossover study to compare the 2 methods of disaster triage by 2 objective measures-time and accuracy. A total of 57 experienced, licensed First responders participated. Results are from analysis of 8 disaster scenarios with a total of 400 patients triaged using the 2 methods of documentation, crossed over, blinded, and paired per participant. Results: The study demonstrated the feasibility of using TriageDoc, a Palm personal digital assistant (PDA ) program, as a viable alternative to current manual disaster triage. Furthermore, the PDA program gave advantage to bioterrorist agent identification.

Conclusions: The feasibility of an automated Palm (Palm, Inc., Sunnyvale, CA) PDA triage program was demonstrated in this study. Study limitations, by the number of participants and the fact it is feasibility research, are acknowledged. Nevertheless, the research demonstrated TriageDoc was as accurate or more accurate as the Manual method of triage with a tendency to require less time. Also there was no statistically significant difference between research sites with respect to accuracy or time to completion when the TriageDoc system was used. The program provided consistency and had flexibility in adapting to the various differences in triage methods at different locations. Hence, PDA programs such as TriageDoc may have potential advantages over handwritten documentation for disaster triage. D 2006

^B NIH completely and solely funded this research through NIH/National Library of Medicine (Bethesda, Md) Small Business Innovation Research grant R43LM008175-01.

* Corresponding author. P.O. Box 518 Earleton, FL 32631, USA. Tel.: +1 352 468 2737/352 214 0152.

E-mail addresses: [email protected] (C.L. Schell)8 [email protected] (R. Wohl).

0735-6757/$ - see front matter D 2006 doi:10.1016/j.ajem.2006.03.027

Introduction

The need for improving triage is clear and has been well studied [1-4]. Because medical triage involves the task of organizing patients, it makes sense that personal digital assistant (PDA) software might be helpful in triage, as it is a powerful sorting tool. A literature review found many articles looking at the potential of a PDA in medical documentation, but none were found that compared standard manual triage with PDA triage. One article reported using a PDA for medical triage [5], and other studies have showed the efficiency of using a PDA in medical care. In a study comparing manual and PDA documentation for acute pain management, 650 sheets of paper and 130 man-hours were saved by using a PDA [6]. Furthermore, satisfaction studies of a PDA triage program in an emergency department showed favorable trend to the PDA [7-9]. A National Institutes of Health (NIH)/National Library of Medicine Small Business Innovation Research grant award (R43LM008175-01) allowed us to test feasi- bility. Results are presented herein.

Background

We surveyed first responders via the Web regarding current practices. A call for participation was sent to emergency services e-mail lists. This resulted in 63 anony- mous and fully completed surveys. Respondents selected methods they most commonly use in triage from list of 32 different sorting methods identified from the literature. Glasgow Coma Score was used by 37 of the 67, followed by 31 for simple triage and Rapid treatment (START), 12 for Trauma Score (TS), and 11 for Revised Trauma Score . Five selected they do not use any method to assist with sorting to triage. Less than 10 respondents selected the remaining 29 methods. With these results and the experience of first

Table 1 Descriptive statistics for the accuracy score Scenario Site Manual TriageDoc

		n	Mean	SD		n	Mean	SD
Practice	JAX	18	8.7	2.6	18		14.7	4.6
	MIA	13	9.2	3.5	13		11.2	2.2
	Combined	31	8.9	3.0	31		13.2	4.1
Fire	JAX	18	11.3	3.8	18		12.0	3.5
	MIA	13	12.1	3.1	13		12.5	2.6
	Combined	31	11.6	3.5	31		12.2	3.2
MCI	JAX	18	13.7	5.0	18		15.9	8.0
	MIA	13	16.8	4.4	13		16.5	7.3
	Combined	31	15.0	4.9	31		16.1	7.6
MVA	JAX	18	12.6	2.7	18		11.3	3.6
	MIA	13	12.5	1.3	13		12.5	3.0
	Combined	31	12.5	2.2	31		11.8	3.3
Total	JAX	18	46.2	10.1	18		53.9	14.3
	MIA	13	50.7	8.9	13		52.7	10.1
	Combined	31	48.1	9.7	31		53.4	12.5

Table 2 Descriptive statistics for time in minutes Scenario Site Manual TriageDoc
		n	Mean	SD		n	Mean	SD
Practice	JAX	18	15.9	2.7		18	14.1	1.3
	MIA	13	15.6	1.3		13	15.6	1.5
	Combined	31	15.8	2.2		31	14.8	1.6
Fire	JAX	18	5.5	1.5		18	6.6	1.4
	MIA	13	6.5	1.7		13	6.3	1.3
	Combined	31	5.9	1.6		31	6.5	1.1
MCI	JAX	18	36.7	2.0		18	35.0	1.8
	MIA	13	36.8	3.6		13	34.3	1.1
	Combined	31	36.6	2.7		31	34.7	1.6
MVA	JAX	18	6.3	2.1		18	6.1	0.3
	MIA	13	7.6	1.5		13	6.1	0.6
	Combined	31	6.9	2.0		31	6.1	0.5
Total	JAX	18	61.6	2.5		18	64.2	5.2
	MIA	13	66.2	4.4		13	62.3	3.3
	Combined	31	65.2	5.1		31	62.0	2.9

responder consultants, we designed a PDA program and a series of 8 disaster scenarios.

The 8 disaster scenarios were created with a range of complexity from a simple motor vehicle accident of 6 persons to 40 patients exposed to a tanker leaking Chlorine gas and a biological agent threat. Two scenarios were derived from standard textbook triage teaching cases in the advanced trauma life support course. We also created 4 duplicate disaster scenarios, which were disguised by an entirely different setting, background, and presentation order. Great care in video editing was taken to obscure the original patients’ identities.

The PDA program, TriageDoc, was developed to accom- modate different triage methods from basic tag color to RTS, TS, and elapsed time. From this basic input data, TriageDoc calculates Glasgow Coma Score, RTS, and TS, and time stamps the entry ready for the next patient to be triaged.

The resulting triage list is exported to the Palm memo pad (Palm, Inc., Sunnyvale, CA), so that TriageDoc provides transferability of the triage. Any additional documented findings help with differential diagnosis of bioChemical agents.

The first practice site was in a large urban setting with paid rescue workers. The second was a rural ambulance service with a staff of all volunteers. Incentive money from the grant compensated for participation in a full-day competition. Participants with the top scores and best time won additional prizes. Enrollment excluded those with less than 1 year of first responder experience. An active basic life support or advanced cardiac life support card and professional license was required. Prior PDA experience was not a prerequisite. All were provided basic instruction on how to use a Palm PDA in 45 minutes followed by 45 minutes of instruction on TriageDoc.

After the program and contest were refined by recom- mendations from 2 practice sites, we were ready to do the research. Miami and Jacksonville provided volunteers from

		n	Mean	SD	P
Practice	JAX	18	5.2	5.4	.0723
	MIA	13	2.0	4.1
	Combined	31	4.3	5.0	b.0001
Fire	JAX	18	3.2	3.1	.7788
	MIA	13	0.7	3.4
	Combined	31	0.6	3.2	.3176
MCI	JAX	18	2.2	7.9	.3940
	MIA	13	-0.4	8.5
	Combined	31	1.1	8.1	.4455
MVA	JAX	18	-1.2	3.8	.2869
	MIA	13	0.0	2.5
	Combined	31	-0.7	3.3	.2417
Total	JAX	18	7.6	11.6	.2166
	MIA	13	2.0	12.7
	Combined	31	5.3	12.2	.0223

several fire rescue and first responder organizations. Total for the number of research participants was 31 despite Hurricane Dennis. This brought the total to 57 participants from 4 different locations. Seventy percent of the total 57 had no prior PDA experience. Focused statistical analysis was done on the 2 research sites. The mean average number of years of first responder field experience of these 31 research participants was 8.16 years.

Table 3 Table of descriptive statistics and P values for the difference in accuracy scores between TriageDoc and manual method assessments

Scenario Site Difference

Methods

A paired t test was used in an intraindividual, blinded, crossover study to compare the 2 methods of disaster triage documentation by 2 objective measures: elapsed time in minutes and accuracy points based on a master key. The accuracy score was a composite of points from (1) overall matching triage order, (2) selection of top 5 most critical patients regardless of order, and (3) accurate identification of biochemical agent(s) if present. Participants drew random card numbers to determine their PDA number so individual names were not associated with scores. Half of the participants manually documented the same scenario at the same time as the other half using the TriageDoc program. Then, for the next scenario, they were switched so those documenting with a PDA would be documenting with a pen. Seat assignment was done by red or blue card draw. Those documenting by hand used whatever sort of method they commonly used to determine rank order for triage.

Results

Detailed results of the t test comparison are presented in table format. In each table, a brief scenario title such as Fire,

MVA, Practice, and mass casualty index describe the 4 disaster scenarios and then MIA (Miami) or JAX (Jackson- ville) abbreviates the site. The homogeneity of sites was assessed via a 2-sample t test on the differences between the TriageDoc accuracy scores and the manual accuracy scores and the TriageDoc time and the manual time. The combined difference between the TriageDoc and manual assessments was tested for a difference from zero using a paired t test. Tables 1 and 2 are the descriptive statistics for accuracy and time, respectively, from which Tables 3 and 4 on statistical significance are based. Results indicate that there was no difference between Jacksonville and Miami with respect to the difference between the scores for TriageDoc and Manual triage evaluations. Table 4 shows time results with descriptive statistics and P values for the difference in time between TriageDoc and manual method assessments. These results indicate that there was no difference between Jacksonville and Miami with respect to the difference between the mean times for evaluation for TriageDoc and manual evaluations. The results also indicate that the mean total time for the TriageDoc was statistically less than the mean total time for the manual evaluation.

From all 4 sites combined, mean average accuracy scores of all 57 participants presented tended to be higher with TriageDoc. From all 4 sites, biochemical agent accuracy identification points indicate some advantage of using TriageDoc. A total of 170 accuracy points came from manual identification of biochemical agents, whereas 182 points came TriageDoc. One example is that no one doing manual triage documentation during the same practice scenario throughout all 4 sites correctly identified tularemia in the disaster, whereas 3 using TriageDoc correctly identified it as a bioterrorist agent implicated in the disaster. One of these 3 had never used a PDA before.

Table 4 Table of descriptive statistics and P values for the difference in time for assessment between TriageDoc and manual method assessments

Scenario Site Difference

		n	Mean	SD	P
Practice	JAX	18	-1.8	2.8	.0524
	MIA	13	0.0	1.7
	Combined	31	-1.0	2.5	.0308
Fire	JAX	18	1.1	2.0	.1185
	MIA	13	-0.2	2.4
	Combined	31	0.5	2.3	.1902
MCI	JAX	18	-1.7	2.3	.7511
	MIA	13	-2.1	3.5
	Combined	31	-1.9	2.8	.0008
MVA	JAX	18	-0.2	2.1	.0741
	MIA	13	-1.5	1.8
	Combined	31	-0.8	2.1	.0466
Total	JAX	18	-2.6	6.0	.5318
	MIA	13	-3.8	4.9
	Combined	31	-3.1	5.5	.0034

Conclusion

The feasibility of TriageDoc as a viable alternative to the current manual method of triage in emergency situations was demonstrated in this study. It is acknowledged that this study is limited by the number of participants and that it is feasibility research. Nevertheless, the research demonstrated that TriageDoc was as accurate or more accurate than the manual method of triage with a tendency to require less time. These statistical results are more remarkable in light of the fact that all participants were experienced in emergency situations; however, a majority (70%) had no experience with a handheld palm device. So, TriageDoc does not have a prolonged learning curve.

It should also be noted that there was no statistically significant difference between the Miami and Jacksonville sites with respect to accuracy or time to completion using the TriageDoc system. This demonstrates that the device is flexible in adapting to the various differences in triage methods at different locations. It is concluded that PDA programs such as TriageDoc may have potential advantages over handwritten documentation for disaster triage.

Acknowledgments

The following participated in this study: Gary Stevens, PhD, biostatistician, University of Florida, Gainesville; Al Borchers, PhD, computer scientist, former PI, Google, Inc; Randy McLaughlin, programmer, Red Wing, Minn; George Jahn, EMT-P scenario design, Hoags Ambulance Assoc Inc, New York, NY; Ed Cobb, REMT-P coordinator, Jacksonville Fire Rescue, Fla; Nick M. Wohl, REMT-P coordinator, City of Miami Fire Rescue, Fla; Paul Glazer, REMT-P coordina- tor, Hoags Ambulance Assoc Inc,; Laurie Romig, MD,

FACEP consultant, director, Pinellas County EMS, Fla; Louise Jahn, administrative assistant, E Nassau, New York, NY; Charles Poulton, video editing media specialist, Medical Informatics, University of Florida, Gainesville; P Peggy Hsu, AMLS, University of Florida Health Science Center Librar- ies, Gainesville; Moon Metz, statistical support, St Pete Beach, Fla; and Harry Metz, site setup organizer, St Pete Beach, Fla.

References

1. Salhanick SD, Sheahan W, Bazarian JJ. Use and analysis of filed triage criteria for Mass gatherings. Prehospital Disaster Med 2003; 18(4):347 - 52.
2. Schmidt TA, Cone DC, Mann NC. Criteria currently used to evaluate dispatch Triage systems: where do they leave us? Prehosp Emerg Care 2004;8(2):126 - 9.
3. Hay E, Bekerman L, Rosenber F, Peled R. Quality assurance of nurse triage consistency of results over three years. Am J Emerg Med 2001; 18(2):113 - 7.
4. Robison JL. Army nurses’ Knowledge base for determining triage categories in a mass casualty. Mil Med 2002;167(10):812 - 6.
5. Sadeghi S, Barzi A, Zamin-Khameh N. Decision support system for medical triage. Stud Health Technol Inform 2001;81:440 - 2.
6. Chan SS, Chu CP, Cheng BC, Chen PP. Data management using the personal digital assistant in an acute pain service. Anaesth Intensive Care 2004;32(1):81 - 6.
7. Change P, Tzeng YM, Wu SC, Sang YY, Chen SS. Development and comparison of user acceptance of advanced comprehensive triage PDA support system with a traditional terminal. AMIA Annu Symp Proc 2003;140 - 4.
8. Lin YH, Jan ID, Ko PC, Chen YY, Wong JM, Jan GJ. A wireless PDA physiological monitoring system for Patient transport. IEEE Trans Inf Technol Biomed 2004;8(4):439 - 47.
9. Chang P, Hsu YS, Tzeng YM, Sang YY, Hou IC, Kao WF. The development of intelligent, triage-based, mass-gathering emergency medical service PDA support systems. J Nurs Res 2004;12(3):227 - 36.