a b s t r a c t

Background: Conducting a thorough after-action review (AAR) process is an important component in improving preparedness for Mass casualty incidents (MCIs).

Purposes: The study aimed to develop a structured AAR tool for use by medical teams in emergency departments after an MCI and to identify the best possible procedure for its conduct.

Basic procedures: On the basis of knowledge acquired from an extensive literature review, a structured tool for conducting an AAR in the emergency department was developed. A modified Delphi process was conducted to achieve content validity of the tool, involving 48 medical professionals from all 6 level I trauma centers in Israel. The AAR tool was tested during a simulated MCI drill.

Main findings: All experts support the conduct of an AAR in the ED after an MCI to build and maintain capacity for an adequate emergency response. More than 80% agreement was achieved regarding 14 components that were implemented in the proposed AAR tool. Ninety-four percent perceived that AARs should be conducted within 24 hours from the event using both written reports and face-to-face discussions. Both physicians and nurses should participate. The incident manager should lead the AAR, limiting the time allocated for each speaker and for the AAR in whole.

Principle conclusions: Conducting a structured AAR in all emergency departments after an MCI facilitates both learning lessons regarding the function of the medical staff and ventilation of feelings, thus mitigating anxieties and expediting a speedy return to normalcy.

(C) 2013

Background

Emergency management of mass casualty incidents (MCIs) is characterized by a need to respond swiftly to unexpected complex situations [1]. Often, MCIs necessitate admitting and treating multiple injuries in casualties of varying types and severities, requiring the deployment of multidisciplinary medical teams at receiving hospitals [1]. decision-making processes used during a response for MCIs

? Authors contributions: Joint first authorship: T.G. and B.A. jointly participated in writing the manuscript. G.T., E.D., C.T., and A.T. conceived the study and designed the

modified Delphi and exercise. A.B. and A.D.L. supervised the conduct of the study and data collection. A.B. and A.D.L. provided advice on study design and analysis of the data.

G.T. and A.B. drafted the manuscript, and all authors contributed substantially to its revision. A.B. takes responsibility for the paper as a whole.

?? There are no conflicts of interests and no financial support.

? The manuscript was presented in the International Preparedness and Response for

Emergencies and Disasters 2012 conference.

* Corresponding author. Department of Emergency Medicine, The Leon and Mathilda Recanati School for Community Health Professions, Ben-Gurion University of the Negev, POB 653, Beer-Sheva 84105, Israel. Tel.: +972 54 804 5700; fax: +972 77 910 1882.

E-mail address: [email protected] (A. Bruria).

differ from routine protocols; therefore, lessons learned and experiences gathered from various MCIs should be studied, integrated into the organizational Knowledge base, and implemented in future situations [2].

Emergency departments (EDs) are a central component of the response model for MCIs because they admit, triage, and provide lifesaving medical care to a large number of casualties [3]. The need to expand capacity to meet the surge created by the MCI, as well as additional activities and pressures exerted on the medical teams, varies significantly from the routine function of the ED staff [4]. Frequently, as a result of the complex medical condition of the casualties that are admitted and the external personnel that are deployed to reinforce the routine workforce to better manage the MCI, gaps in communication and coordination are created in the ED, resulting in a less-than-optimal response to the situation [2,5,6].

As mentioned earlier, an important component in improving preparedness for MCIs is learning lessons from both exercises and real-life events. The most common method for reviewing what happened and identifying ways to improve future performance is through an after-action review (AAR) [5,7,8]. An AAR seeks to present answers to questions such as What was supposed to happen? What

0735-6757/$ - see front matter (C) 2013 http://dx.doi.org/10.1016/j.ajem.2013.01.025

actually happened? Why were there differences? What gaps materialized between planning and execution? What can we learn from this experience? [2,9]. After-action reviews are designed to facilitate learning from errors and from successes, to identify strengths that should be maintained and weaknesses that should be rectified, and to reveal misses and near-misses [2,5,7,9]. After-action reviews have the potential to enhance organizational sensitivity and resilience and provide opportunities to acknowledge individual and institutional expertise [7]. It has been recommended that AAR be conducted soon after the MCI has concluded, regardless of whether negative results have occurred [7,9].

Importance

After-action reviews are used constantly in military settings based on advanced tools that have been created to facilitate this process [10,11]. Implementation of AARs after an MCI is highly encouraged in civilian settings too, but structured tools that can be used for this purpose are lacking [6,12]. To date, there is no validated, widely accepted AAR tool that can be used by medical institutions to guide and improve their preparedness and response for emergencies; more so, the few performance-based tools that were developed have not been fully tested for reliability and validity [13].

effective management of the AAR must be used in order for the framework to facilitate understanding and insights to the lessons to be learned, as well as awareness and agreement on actions that should be taken to improve emergency response to future events [9]. Topics for discussion, expectations of the AAR process, and methodology for its conduct must be professionally prepared to assure effectiveness [9,14]. In light of the importance of learning lessons after an MCI and the need to use an effective mechanism for optimizing the process in the ED, the current study was designed.

Goals of this investigation

The study aimed to use a scientific approach for developing a structured AAR tool for the use of medical teams in the ED after an MCI. The study further sought the best possible procedure for its conduct.

Methods

An extensive literature review of components relevant to response of EDs to MCIs and tools that are in place in military and civilian settings was conducted. Based on the knowledge acquired, a structured tool for conducting an AAR in the ED was developed.

To estimate the content validity of the tool, a modified Delphi process was conducted involving medical teams from all 6 level I trauma centers in Israel. The AAR tool was disseminated to 48 staff (physicians and nurses), requesting their opinions regarding the following issues: their perception of the need for review of ED performance after an MCI, relevance of the various components of the AAR tool with regard to the goal of learning lessons following an MCI, preferred format for evaluating those elements, and recommenda- tions regarding modifications that are needed in the proposed tool or the procedure through which it should be conducted. The recom- mendations were evaluated, and the level of consensus between the various content experts was compared. The required level of

Fig. 1. Process for developing and validating the AAR tool.

Results

Tool structure

The tool consisted of 3 main sections:

• Closed questions regarding the function of the ED’s medical teams during the emergency response (such as use of equipment, personal safety, registration, etc)

• Closed questions pertaining to ED managers (such as control and command, manpower operation, and patient evacuation)
• Open-ended questions focusing on “ventilating” emotions and reflections of the medical staff during and after the event (such as effectiveness of support teams, feeling of security, and personal lessons learnt).

  Results of the modified Delphi process regarding the development of the AAR tool

  Of 48 content experts from the EDs of the 6 level I trauma centers, 39 responded to the modified Delphi (81% response rate). Thirty- three (85%) of them were involved in 3 or more MCIs, within the

  Table 1

  Characteristics of the respondents to the modified Delphi cycle

  Topic Characteristic % of content experts (n = 39)

  professional experience 1-5 y 13

  6-10 y 18

  11-15 y 10

  N 15 y 59

  agreement between experts was predefined as 80% or higher. On the basis of these recommendations, revisions were made to the preliminary AAR tool, which was then tested during a drill simulating an MCI scenario. Subsequently, the revised tool was reviewed by medical staff of a hospital that participated in the previously mentioned drill, both before and immediately after the drill. The process of the study is described in Fig. 1.

  Time frames of previous experience in MCIs

  Extent of previous experience in MCIs

  Experienced in the last year 62

  Experienced in the last 3 y 23

  Experienced in the last >=4 y 7.5

  No previous experience 7.5

  No experience 7.5

  Experienced one MCI 7.5

  Experienced 3-5 MCIs 15

  Experienced N 5 MCIs 70

  Table 2

  Levels of agreement among content experts regarding relevance of the different components that should be incorporated in the AAR tool

  Level of agreement (n = 39) (%)	No. of parameters	% of parameters
  100	4	22
  90-99	8	44
  80-89	2	11
  60-79	0	0
  30-59	1	6
  b30	3	17
  Total	18	100

  previous 3 years. The characteristics of the respondents are summa- rized in Table 1.

  All experts (n = 39) expressed the view that an AAR must be conducted in the ED after an MCI and that this action is an important component of building and maintaining capacity for an emergency response. An agreement of more than 80% was achieved regarding 14 of

  18 components that were identified in the literature review and implemented in the proposed AAR tool. The levels of agreement among content experts regarding the relevance of the different components that should be incorporated in the tool are presented in Table 2. The 4 components that did not achieve the targeted level of agreement between content experts (annotated by gray background in the table) included the following: (1) Do you have recurrent disturbing visions (thoughts) regarding the MCI? (33% supported inclusion in the AAR).

  (2) Describe pictures from the MCI that you remember well (18% supported inclusion in the AAR). (3) Do you suffer from Sleep deprivation as a result of the MCI? (10% supported inclusion in the AAR). (4) Who do you confide in regarding the complex elements that you have experienced? (13% supported inclusion in the AAR).

  Almost all (94%) experts believe that AARs should be conducted within a short time span from the event, ranging from immediately (86%) to no longer than within 24 hours (8%). Only 2 content experts (6%) think that the AAR can be conducted within 3 days or longer from the occurrence of the MCI. The format of the AAR should consist of a combination of techniques, using both written reports and face-to- face discussions (n = 35; 89%). Finally, an agreement of 80% and above was reached between experts regarding the preferred format for conducting the AARs, including who should manage the procedure, which professions should be included, and overall time

  that should be allotted for the AAR and for each individual speaker. The positions of the content experts regarding the preferred format for conducting the AARs are presented in Fig. 2.

  Results of the modified Delphi process regarding the pilot study

  Six (75%) of 8 content experts from the hospital that participated in the MCI drill responded to the questionnaire before and after the drill. Their answers were identical before and after the drill. The additional 2 professional who did not respond in writing stated orally to the researchers of this study that their position regarding all aspects of the questionnaire had not changed and their initial responses to the first Delphi cycle were equally valid for the second cycle. The AAR tool was modified according to the Delphi findings, and the pilot study and is presented in Annex 1.

  Limitations

  The study was conducted among content experts from the 6 level I trauma centers in Israel and did not encompass all 28 acute care hospitals in the country. Nevertheless, the assumption is that these sophisticated trauma centers are the most experienced and best prepared for MCIs; therefore, their leading ED staff are best equipped to review the proposed AAR tool.

  Another limitation that should be considered is the absence of structured AAR tools for EDs designated for use after a mass casualty event. Therefore, it was not possible to compare the proposed tool to similar tools, being used at various hospitals.

  Discussion

  Effective management of MCIs requires development and use of structured management tools including AARs [1,2,5,11]. After-action reviews must be conducted in a nonjudgmental manner, focusing on learning and constructive criticism aimed at improving readiness for future emergency events [6]. A structured format is required for managing AARs, to understand expectations and perspectives of personnel involved in the MCI, generate insight to strengths and weaknesses, change behaviors, and achieve agreement concerning needed actions [9]. On the basis on the opinion of content experts who participated in this study, it can be inferred that a structured AAR tool

  

  Fig. 2. Views of content experts regarding preferred format for conducting the AAR.

  would facilitate the opportunity of ED medical personnel to learn lessons and express emotions after MCIs, thus improving performance in the next emergency event.

  The literature describes various tools for conducting AARs in military settings and recommends their implementation in civilian medical facilities [6,9-12]. It has often been stated that implementa- tion of AARs after MCIs is important to promote an effective learning opportunity, encourage ongoing improvement, and guide action to enhance patient safety and care [2,7,9,15]. However, although the ED is a dominant actor in responding to MCIs, the existing tools are not well suited for conducting AARs in this department [16].

  As part of the current study, a tailor-made AAR tool was developed specifically targeted at the ED. The tool enables physicians and nurses to systematically examine their performance during an MCI, learn lessons and share their thoughts, conclusions, and feelings, in a friendly nonjudgmental atmosphere. The tool was applied before and after an MCI drill at a hospital’s ED. The high consensus levels that were achieved in response to the newly developed AAR tool and its endorsement by the participating medical teams, both before and after the drill, seem to indicate that the staff recognizes the importance and benefit of performing AARs of MCIs.

  The developed AAR tool can be instrumental for both routine medical teams that are used in EDs and reinforcing personnel that are deployed to this department upon occurrence of an MCI [2,15]. As was found in other studies, the AAR tool may enable staff involved in an MCI to share their emotions and receive support from their colleagues, prevent anxieties, and facilitate a speedy return to readiness [2,7,14]. Existing literature has brought to light the fact that a mass casualty event is, at times, also a “multicaregiver event.” For this reason, friction often exists between different sectors [4] and must be discussed as part of any AAR. The findings of the current study recognize and support the need for joint operation of the 2 professions. Therefore, it is recommended that the AAR be conducted with the participation of both physicians and nurses under the leadership of the official who

  functioned as the facility’s incident manager during the MCI.

  Conclusions

  To enhance preparedness for MCIs, a structured AAR should be conducted in EDs, immediately or within a short time frame, after an MCI. Contents of such an AAR and its format for implementation have been proposed in the present study. The AAR should incorporate written reports and face-to-face discussions, with joint participation of both physicians and nurses. The incident manager should lead the AAR where the time allocated for the AAR and for each speaker should be limited. The process of AAR will not only facilitate learning lessons regarding the function of the medical staff but also enable ventilation of feelings, thus mitigating anxieties and facilitating a speedy return to normalcy. It is highly recommended that such an AAR tool and procedure be implemented in all hospitals’ EDs.

  The results of the current study suggest that use of a customized AAR tool could prove to be productive for other hospital’s units involved in the response to MCIs, as well as First responders such as ambulance crews, police, and firefighters.

  Annex 1. AAR tool

  General Questions for all participating staff (physicians and nurses)

  In your opinion, was the ED sufficiently staffed during the MCI with nursing and medical staff to care for all patients and casualties?

• Were the necessary equipment and supplies accessible?

  Yes No Clarify:

  Did you act according to the security and Safety precautions procedure?

  Yes No Clarify:

  Is there a unidirectional track for all casualties (separate entrances and exits)?

  Yes No Clarify:

  Were the registration and documentation of every identified and anonymous casualty carefully and effectively monitored?

  Yes No Clarify:

  Was there a need to reinforce the ED with external emergency equipment? If so, was the reinforcement performed effectively?

  Yes No Clarify:

  Was there sufficient support of reinforcing teams in the MCI, such as stretcher-bearers, security personnel, social workers, etc?

  Yes No Clarify:

  Specific questions for ED medical and nursing managers

  How secure and competent did you feel in operating reinforcing medical and nursing teams, as well as additional team members?

  Yes No Clarify:

  Do you think the crew was allocated within the admitting sites according to their skills, capabilities, and expertise?

  Yes No Clarify:

  Did you feel that you were in control of the event (confidence in your functional skills and professional knowledge)?

  Yes No Clarify: Yes No Clarify:

  Were the evacuation and discharge of patients performed in a structured process at a defined discharge location?

  Yes No Clarify:

  Open-ended questions

  What emotions arose in you during and after the event?

• Did you feel confident in your function and performance throughout the MCI?
• What personal lessons have you learned from the event? What elements will you maintain? What elements will you improve?

  References

  1. Halpern P, Tsai MC, Arnold JL, Stok E, Ersoy G. Mass-casualty, terrorist bombings: implications for emergency department and hospital emergency response (part II). Prehosp Disaster Med 2003;18(3):235-41.
  2. Schindler M, Eppler MJ. Harvesting project knowledge: a review of project learning methods and success factors. International J Project Management 2003;21:219-28.
  3. Administration USF. The After-Action Critique: Training Through Lessons Learned. Department of Homeland Security, 2008 April 2008. Report No:USFA-TR-159.
  4. Juffermans J, Bierens JJ. Recurrent medical response problems during five recent disasters in the Netherlands. Prehosp Disaster Med 2010;25(2):127-36.
  5. Baily C. Application of the After-Action Review Paradigm to Fire Department Operation. tinhelmetcom.
  6. Fishbane M, Kist A, Schiever RA. Use of the emergency Incident Command System for school-located mass influenza vaccination clinics. Pediatrics 2012;129(sup- plement 2):S101-6.
  7. Allen JA, Baran BE, Scott CW. After-action reviews: a venue for the promotion of safety climate. Accid Anal Prev 2010;42:750-7.
  8. Fontaine M, Martin GA, Daly J, Thurston C. Technological and usability-based aspects of distributed after action review in a game-based training setting. In: Harris D, editor. Engin. Psychol. And Cog. Ergonomics. Springer-Verlag Berlin Heidenlberg; 2011. p. 522-9.
  9. Cronin G, Andrews S. After action reviews: a new model for learning. Emerg Nurse

     2009;17(3):32-5.

     Bliss JP, Minnis SA, Wilkinson J, Mastaglio T, Barnett JS. Establishing an intellectual and theoretical foundation for the after action Review process-a literature review. US Army Research Institute for the Behavioral and Social Sciences, Arlington, Virginia. 2011.

  10. Mastaglio T, Wilkinson J, Jones PN, Bliss JP, Barnett JS. Current practice and theoretical foundations of the after action review. Portsmouth, VA: NYMIC; 2011.
  11. Ramachandran S, Jensen R, Bascara O, Carpenter T, Denning T, Sucillon S. After action review tools for team training with chat communications. The interservice/industry training, simulation & education conference (I/ITSEC) volume 2009.
  12. McCarthy ML, Brewster P, Hsu EB, Macintyre AG, Kelen GD. Consensus and tools needed to measure health care emergency management capabilities. Disaster Med Public Health Preparedness 2009;3(Suppl 1):S45-51.
  13. Fidler DP. H1N1 after action review: learning from the unexpected, the success and the fear. Future Mictobiol 2009;4(7):767-9.
  14. Wilson K, Brownstein JS, Fidler DP. Strengthening the international health regulations: lessons from the H1N1 pandemic. Health Policy Plan 2010;25:505-9.
  15. Riba S, Reches H. When terror is routine: how Israeli nurses cope with multi- casualty terror. Online J Issues Nurs 2002;7(3):6.