Original Contribution

Agreement on sedation-related events between a procedural sedation registry and computerized medical records

Kyle Shaver MD, Steven Weiss MD, MS?, Darren Braude MD, MPH

Department of Emergency Medicine, University of New Mexico Health Sciences Center, Albuquerque, NM 87131-0001, USA

Received 18 August 2008; revised 8 November 2008; accepted 19 November 2008

Abstract

Objectives: Little is known about the accuracy of the medical record to document sedation-related events (SREs). Our hypotheses were that, when compared to a reference database (RD), a procedural Sedation quality assurance registry (PSQAR) and medical records were Accurate documentation of SREs.

Methods: All cases in our PSQAR over 13 months were examined. Those with SREs were entered into the RD. We reviewed the computerized medical record (CMR; both physician documentation and nursing documentation) to determine the presence of procedural sedation notes and documentation of SRE. Results: A total of 203 sedation cases were entered into the PSQAR. The RD consisted of 74 SREs during 60 cases. The PSQAR included 61 (82%) of 74 events (95% confidence interval [CI], 72%-90%), whereas the CMR included 44 (60%) of 74 events (95% CI, 47%-71%). Physician documentation completely matched the RD in 12 (20%) 60 cases (95% CI, 11-32) and nursing documentation completely matched the RD in 24 (40%) of 60 cases (95% CI, 28-53). Physician and nursing documentation were complete and agreed with each other in only 4 (9%) of 60 cases (95% CI, 2-16; ? = 0.13).

Conclusion: When compared to the RD, the accuracy of the PSQAR was very good. However, the accuracy of the CMR, including both nursing and physician documentation, was poor and there was poor agreement between them. This has implications for chart review-based research and quality improvement.

(C) 2010

Introduction

Chart review is commonly used for patient care, medical research, risk management, and quality assurance. Little is known, however, about the accuracy of the medical record when it comes to documenting complications of medical care. Although physicians and nurses are often admonished about the importance of complete and honest documentation,

* Corresponding author. Tel.: +1 505 272 5062; fax: +1 505 272 6503.

E-mail address: [email protected] (S. Weiss).

there are competing concerns of limited time and medico- legal exposure. We did not find any published reports comparing the accuracy of medical records and research/ quality assurance databases.

This study piloted a new procedural sedation quality assurance registry (PSQAR). We developed a reference database (RD) consisting of all sedation-related events (SREs). PSQAR and CMR were evaluated for SREs in these cases and all of these were included in the RD. Our hypotheses were that, when compared to this RD, (1) a PSQAR was an accurate documentation of SREs, and

0735-6757/$ - see front matter (C) 2010 doi:10.1016/j.ajem.2008.11.019

Pi ocedtir al Sedation Lo etteck sticker hefe

Fig. 1 The form developed by our experts to document SREs that occur during the procedural sedation process in the ED.

(2) nurse and physician charting on the computerized medical record (CMR) was an accurate documentation of SREs.

Methods

Study design

Our PSQAR was intended to record all procedural sedations performed in our urban, academic, and tertiary care adult and pediatric EDs for both quality assurance and research purposes. All PSQAR and CMR entries for a 13- month period (March 2006 to April 2007) were examined. An RD was developed to include all cases of procedural sedation and all SREs associated with the visit. The cumulative total of events between the CMR and PSQAR was used as the RD for evaluation of the registry, physicians, and nurses. Records were considered complete if the registry or note identified all SREs recorded in the RD. To eliminate bias, physicians and nurses were not aware of the purpose of the study. The study was approved by our institution’s human research review committee.

Study setting and population

This study was conducted in an urban tertiary care level 1 trauma center with a census of 66 000 patient visits per year.

Measurement

The PSQAR contained prospectively collected data about procedural sedations performed in the ED and was completed by the physicians at the time of the procedural sedation. Emergency medicine, internal medicine, and family practice residents rotating in the ED are responsible for the care and data form completion of the majority of the registry patients. The PSQAR data entry form was a single page; the first half includes patient demographics, reasons for the sedation, fasting status, depth of sedation, and drugs used (Fig. 1). The second half of the form included checkboxes for the most common SREs, free text areas, and information

Table 1 Types of procedural sedation documented in the RD

Sedation agent	Number of cases	% of total cases (N = 203)
Ketamine	87	43%
Propofol	72	35%
Narcotic + benzo	12	6%
Propofol + ketamine	10	5%
Etomidate	8	4%
Brevital	7	3%
Etomidate + propofol	1	b1%
Other	6	3%

Sedation-related events	Number of cases	% of total events (N = 74)	% of total cases (N = 203)
Decreased saturation	38	51%	19%
or apnea
Myoclonus	8	11%	4%
Vomiting or aspiration	5	7%	2%
Agitation	5	7%	2%
Unpleasant emergency/	5	7%	2%
weird dreams
Hypo- or hypertension	4	5%	2%
Arrhythmia (tachycardia	3	4%	2%
or bradycardia)
Prolonged sedation	3	4%	2%
Stridor	1	1%	b1%
Hives	1	1%	b1%
inadequate sedation	1	1%	b1%

about the person performing the sedation. The forms were completed by the performing resident or supervising attending physician.

The CMR included dictations, scanned copies of all paper notes, ED charts and triage notes, nursing flowsheets, procedural sedation flowsheets, trauma flowsheets, etc. For each sedation case, the hospital CMR and the PSQAR were reviewed by 2 of the principal investigators to determine the presence of procedural sedation notes and documentation of SREs. Each case may have multiple events recorded, each of which was counted separately.

Table 2 Sedation-related events recorded in the RD

For our outcome measure, the PSQAR and the CMR were separately compared to the RD. Comparisons were also made between the nurse and physician documentation in the CMR.

Data analysis

Statistics were descriptive with 95% confidence intervals. ? was calculated for comparison between nurse and physician documentation. P b .05 was considered significant.

Results

Two hundred three sedation cases were entered into the PSQAR during the study period. Table 1 shows the agents documented for sedation in the registry. The RD consisted of

60 (29%) of 203 cases (95% CI, 23%-36%) that had documentation of SREs. Based on review of the CMR and PSQAR, the cumulative total of SREs entered into the RD was 74 in those 60 cases, with 13 of the cases having more than 1 SRE.

The 74 total SREs are categorized in Table 2. PSQAR information was complete for 61 (82%) of 74 events (95% CI, 72%-90%), whereas the CMR was complete for 44

Number		Percentage	95% CI
Registry	61	82%	72%-90%
CMR-Physician and nurse	44	60%	47%-71%
Physicians	17	23%	14%-34%
Nurses	33	45%	33%-57%
Neither physician nor nurse	33	45%	33%-57%
noted SRE

(60%) of 74 events (95% CI, 47%-71%). Events not recorded consistently included maneuvers to improve oxygenation (O₂, jaw thrust, oral airway), prolonged postprocedure sedation, myoclonus, and emergence reactions (Table 3).

Table 3 Comparison of registry and CMR to the RD for SREs recorded (out of 74 events recorded in the RD)

There were 60 patients with SREs included in the RD. Nine (15%) of the 60 cases (95% CI, 7-27) with SREs in the RD were noted to have neither physician nor nursing procedure notes in the CMR. Another 13 cases (21%; 95% CI, 25%-50%) had at least one procedure note but no reference to any SREs. Physician procedural sedation notes were present in 33 (55%) of 60 cases (95% CI, 42%-68%). Physician documentation included all events and matched the RD exactly in 12 (20%) of 60 cases (95% CI, 11-32). Nurse documentation was present in 44 (73%) of 60 cases (95% CI, 60%-84%) and completely matched the RD in 24 (40%) of 60 cases (95% CI, 28-53) (Table 4).

There was a significant difference between physician and nurse documentation matching to the cumulative total (20% vs 40%; diff, 20%; 95% CI, 4-36). Physician and nurse documentation were complete and agreed exactly with each other in only 6 (10%) of 60 cases (95% CI, 4-21; ? = 0.13).

Discussion

When compared to the RD, we found that the accuracy of the registry was very good. However, the accuracy of the CMR, including both nurse and physician documentation, was poor and there was poor agreement between them. This incomplete picture of SRE rates in the CMR has implications for research, quality assurance, and billing.

We identified a few studies that found similar result about the ability to perform retrospective research in different clinical scenarios, although they all compared registries to administrative information [1]. A 2003 study looked at central venous catheter complication rates using multiple administrative hospital codes, billing information, and medical record abstraction. They found poor correlation between identifying patients with catheters and were unable to assess complication rates. They concluded that using administrative databases for identification of patients needed “extensive modification and validation” [2]. Another study comparing a validated registry and statewide administrative database for cardiac surgery showed a 27% disparity in actual

number of procedures performed and a statistically signifi- cant difference in reported mortality. They concluded that administrative data were “problematic” for quality assurance measures [3]. Finally, there are several papers within the trauma literature that examine hospital discharge codes with trauma registries and found they accurately described the actual injuries, except in traumatic brain injury [4-6]. We could not identify any studies that looked at specific aspects of care and compared medical records to the registries.

There are also quality assurance implications to the results of this study. The 60 cases in the registry involving SREs obviously represent the patients in whom complete medical record documentation may be the most important. Although no permanent disability was experienced by any of the 60 patients, the documentation was sparse about SREs and interventions performed to protect them from harm. In our current practice environment, documentation often seems to be done as much for quality purposes as for medical purposes. Another area of risk is the determination of billing and coding. Just as important as the billing and collections from procedures appropriately performed is the mandate to adequately document to avoid charges of Medicare fraud and the ramifications associated with a conviction. Multiple

commentaries have been published about this effect [7].

The registry was by far the most complete record of SREs. These results may indicate that teaching efficient and complete documentation is often overlooked in a busy, urban, teaching hospital. Residents performed many of the sedations and were responsible for the majority of the charting in our ED. Perhaps more pertinent and focused teaching about the unique aspects of documentation and billing/coding could have improved charting [8]. Another possibility is that documentation is different when a specific box on a form asks for very specific information. As the registry contained checkboxes of possible SREs, these boxes could have aided in improving the documentation in the cases. The data also suggest there may be other methods of charting (dictation, templates, etc) that could improve the completeness of the medical record, especially for multifactorial processes such as sedation and accompanying procedures. This study was not

Table 4 Comparison of registry and CMR to the RD for fully documented cases with SREs (60 cases in the RD)

Number		Percentage	95% CI
Registry complete	47	78%	66%-88%
Physicians chart complete	12	20%	11%-32%
Nurses chart complete	24	40%	28%-53%
Physician and nurse	6	10%	4%-21% ?
No physician or nurse’s	9	15%	7%-27%
procedure note present
Procedure note present but	13	37%	25%-50%
neither physician nor
nurse charting noted
any SREs
* Agreement between nurses and physicians-? = 0.13.

powered or designed to examine this, but it is an area that may be pertinent for future research.

Limitations

In terms of limitations, this study only applies to the specific finding in procedural sedation cases and cannot be generalized to other types of procedures. It is possible that the complex events in a dynamic setting such as procedural sedation are particularly problematic when documenting in the CMR, but further investigation should be performed in other similar registries. Also, a high percentage of charts in the CMR did not include any specific note regarding the procedural sedation from which to extrapolate data. Simpler data abstraction for less complex procedures may be less influenced by this phenomenon, but this has never been studied. Given the current move toward quality measures and pay-for-performance, more studies are needed to identify the most accurate and complete method for recording these data and representing the actual events of the clinical encounter. We did not perform a chart review to evaluate our overall capture rate of procedural sedations. Our number of patients with SREs of 29% is higher than that in other recently published series [9,10]. It is possible that providers were more likely to fill out registry forms in cases which involved SREs. We feel that focusing on the 60 identified cases in the 203 entries into the registry gave us a “best-case” scenario in which we focused only on those patients in whom the physician documented an SRE in the registry and would

therefore seem to be likely to also record it in the CMR.

Our form did not document either clear-cut documentation of when patients were capable of discharge. This study did not use an Aldrete score [11] for that purpose which could have helped determine fitness for discharge. However, all SREs occurred during the sedation phase of the study and none was recorded as occurring after sedation was completed.

Conclusion

In this pilot study of a PSQAR, when compared to the RD the accuracy of a PSQAR for documenting SREs was very good. However, the accuracy of the CMR for SREs, including both nurse and physician documentation, was poor and there was poor agreement between them. The incomplete documentation of SREs in the medical record has significant implications for chart review-based research and quality improvement.

References

1. Hess D. Retrospective studies and chart reviews. Respir Care 2004;49 (10):1171-4.
2. Wright SB, et al. Administrative databases provide Inaccurate data for surveillance of long-term central venous catheter-associated infections. Infect Control Hosp Epidemiol 2003;24(12):946-9.
3. Shahian DM, et al. Comparison of clinical and administrative data sources for hospital Coronary artery bypass graft surgery report cards. Circulation 2007;115(12):1508-10.
4. McCarthy ML, et al. Comparison of Maryland hospital discharge and trauma registry data. J Trauma 2005;58(1):154-61.
5. Mullins RJ, et al. An analysis of Hospital Discharge Index as a trauma data base. J Trauma 1995;39(5):941-8.
6. Shore A, et al. Validity of administrative data for characterizing traumatic brain injury-related hospitalizations. Brain Inj 2005;19(8): 613-21.
7. Davidson SJ, et al. Where’s the beef? The promise and reality of clinical documentation. Acad Emer Med 2004;11:1127-34.
8. Silfen E. Documentation and coding of ED patient encounters: an evaluation of the accuracy of an electronic medical record. Am J Emer Med 2006;24:664-78.
9. Sacchetti A, et al. Procedural sedation in the community emergency department: initial results of the ProSCED Registry. Acad Emer Med 2007;14:41-6.
10. Hogan K, et al. The safety of single-physician procedural sedation in the emergency department. Emerg Med J 2006;23:922-3.
11. Awad IT, Chung F. Factors affecting recovery and discharge following ambulatory surgery. Can J Anaesth 2007;54(3):243-4.