a b s t r a c t

Background: diagnostic errors made by radiology resident physicians may lead to significant morbidity/- mortality and patient dissatisfaction.

Objective: To determine the etiology and disposition associated with radiology discrepancies on emer- gency department (ED) patients.

Methods: We conducted a retrospective electronic chart review of patients presenting to our ED during ”off hours” at the Penn State Hershey Medical Center during October 2013-November 2014 and had a radiology discrepancy, defined as a patient discharged from the ED with a diagnostic interpretation dis- agreement between the initial radiology resident physician read and final radiology attending physician read.

Results: 81,201 images were performed during ”off hours”, with 174 radiology discrepancies (0.214%) identified. Most discrepancies were associated with CT scans (62%). The most common final diagnostic interpretations associated with discrepancies were missed fracture (10.9%), incidental findings of mass or cyst (10.3%), gastrointestinal inflammation (6.3%), and pneumonia (5.7%). 10% of radiology discrepan- cies were instructed to emergently return to the ED. The most common modality associated with ED follow-up was CT scan of the abdomen/pelvis (50%). Of the 17 patients that returned to the ED, 10 had additional diagnostic imaging, 9 received a subspecialist consult, 5 required surgical treatment, 5 required additional medications, and 1 required a medical hospitalization.

Conclusions: Based on our sample, discrepancies were a small percentage of images performed during ”off hours”, and were associated with CT scans, missed fractures, and non-emergent outpatient follow- up. We suggest that ED and radiology departments work collaboratively to monitor their own rates of discrepancies, and subsequent morbidities and mortalities, to improve patient care.

(C) 2019

Introduction

Discrepancies in diagnostic interpretation between radiology resident and attending physicians may lead to significant morbid- ity or mortality, overcrowding in the emergency department (ED), and patient dissatisfaction. [1] During ”off hours” (weekday eve- nings and overnights, weekends, and holidays), diagnostic images requested by ED physicians are often initially interpreted by a radi- ology resident physician, and final interpretations are provided by an attending radiology physician the following morning. ED patients may be discharged based on the initial interpretation by

* Corresponding author at: Department of Emergency Medicine, Penn State Hershey Medical Center, 500 University Drive, PO Box 850, Hershey, PA 17033- 0850, USA.

E-mail addresses: [email protected] (L. Gergenti), rolym- [email protected] (R.P. Olympia).

the radiology resident physician, however clinically significant dis- crepancies between initial and final interpretations may require emergent re-admittance to the ED for further management.

Several published studies have examined the differences in the interpretation of diagnostic imaging amongst radiology physicians at various levels of training and clinical experience. Based on double-blind interpretations of studies utilizing different diagnos- tic modalities at both community and university hospitals, the Radiology Quality Institute reported a blended error rate of 4.4%, with a possible range of errors between 0.8 and 9.2% depending on the Diagnostic modality performed, body location, and subspe- cialty expertise of the radiology physician [2]. In addition, several studies have reported the discrepancy in the interpretation of diag- nostic imaging between radiology trainees and attending physi- cians, specific to various imaging modalities and the clinical impact at an urban teaching hospital ED [1], the interpretation of

https://doi.org/10.1016/j.ajem.2019.02.027

0735-6757/(C) 2019

torso CT scans at a Level I trauma center [3], and errors in fracture diagnoses in an ED [4].

We conducted a study at our institution to determine the rate of discrepancies between resident and attending radiology physi- cians, focusing on all diagnostic imaging modalities, and the effect of these discrepancies on patients presenting to our rural, tertiary care Level-1 Trauma Center ED. The objective of this study was to determine the etiology and disposition associated with radiology discrepancies on ED patients. Our goal was that by identifying the etiology of radiology discrepancies, educational interventions provided to radiology resident physicians may improve the recog- nition of commonly missed ED diagnoses.

Methods

We conducted a retrospective electronic chart review of prospectively identified patients (both pediatric and adult patients) presenting to the ED at the Penn State Hershey Medical Center, a rural, academic teaching hospital and Level-1 Trauma Center, during October 2013-November 2014 and had a radiology discrepancy, defined as a patient discharged from the ED with a dif- ference in diagnostic interpretation between the initial radiology resident physician read and final radiology attending physician read. Data obtained from the electronic chart included patient demographics, imaging modality associated with the discrepancy, initial and final imaging interpretations, and suggested patient dis- position subsequent to the discrepancy.

During ”off hours” at our institution, diagnostic images (plain film x-rays, CT scans, MRI, and ultrasounds) are interpreted by a second, third, or fourth year resident physician (at a four-year radi- ology program) in real-time initially. Final interpretations are pro- vided the following morning by one of the attending radiology physicians assigned to provide final reads from the previous eve- ning. At our institution, ”off hours” are defined as weekday eve- nings and overnights (5 p.m. to 7 a.m.), weekends, and holidays. Initial reads provided by radiology resident physicians include comparisons with previous diagnostic images (if available and appropriate), Diagnostic findings, differential diagnosis, and initial impression. Assigned ED advanced practice clinicians (physician assistants or nurse practitioners) are notified immediately of radi- ology discrepancies by phone, and subsequently decisions are made by the advanced practice clinician, after review of the elec- tronic medical record, to the appropriate disposition (non- emergent follow-up with their outpatient primary care provider, non-emergent outpatient follow-up with a medical/surgical sub- specialist, or emergent return to the ED for additional care). Each patient with a discrepancy in radiology interpretation is docu- mented on the ”call back” list, which we were given access to. We reviewed ED discharge notes from each patient on this list and compared the initial discharge instructions subsequent to the initial radiology diagnosis, with the final radiology diagnosis added as an addendum before the radiology read was marked as complete.

Data entry and analysis was performed using the Resource Elec-

tronic Data Capture system (REDCap; Vanderbilt University, Nash- ville, Tennessee USA). Descriptive statistics were calculated for all response variables. The institutional review board at the Penn State Hershey Medical Center, approved the study.

Results

A total of 172,907 diagnostic exams were performed in our ED during the study period. Diagnostic exams performed during ”off hours” were 81,201 exams; 174 exams were considered discrepan-

Table 1

Patient demographics.

Characteristics Patients

(n = 174)

Age (years) Mean 42.9

SD 21.1

Range (0-88)

Sex (%) Female 67% (n = 116)

Male 33% (n = 58)

Race (%) Caucasian 88% African American 6%

Asian 1%

Other 5%

cies (0.214%). Demographics on the 174 patients associated with discrepancies are shown in Table 1.

The most common diagnostic modality associated with radiol- ogy discrepancies was CT scans (59.8%), followed by X-rays (25.3%), ultrasound (6.3%), MRI (6.3%), and CT angiogram (2.3%). The most common final diagnostic interpretation associated with radiology discrepancies (Table 2) were missed fracture (10.9%) and incidental findings of mass or cyst (10.3%).

Suggested disposition as a result of the radiology discrepancy included non-emergent outpatient follow-up with a medical/surgi- cal subspecialist (67%), non-emergent follow-up with their outpa- tient primary care provider (23%), and emergent return to the ED for additional care (10%). The most common diagnostic modality associated with ED follow-up was CT scan of the abdomen/pelvis (50%). Of the 16 patients that returned to our ED (1 patient was instructed to return to our ED but did not return and was lost to follow up), 10 had additional diagnostic imaging, 9 received a sub- specialist consultation, 5 required surgical treatment, and 5

required additional medications, and 1 required medical hospital- ization (Table 3). Patients asked to return to the ED spent a mean number of 218.7 min (SD 127.09, 95% CI [151.0-286.4]) in the ED before final disposition. There were no significant morbidities or mortalities associated with patients with diagnostic discrepancies who were instructed to return to the ED.

Discussion

The objective of our study was to determine the etiology and disposition associated with radiology discrepancies on ED patients. Based on our sample, most discrepancies were associated with CT scans (modality), missed fractures (diagnosis), and non-emergent outpatient follow-up with a medical/surgical subspecialist (dispo- sition). Furthermore, of the 16 patients (10% of our total sample) that were asked to return emergently to our ED, the most common diagnostic modality associated with discrepancies was CT scan of the abdomen/pelvis (62%), 31% (n = 5) required surgical treatment and 6% (n = 1) required medical Inpatient hospitalization, and there were no mortalities reported in this subset.

In terms of modality, diagnosis, and disposition, our reported data is similar to that reported in other published studies examin- ing diagnostic imaging errors associated with patients seen in EDs. Similar to our data, several published studies have determined that CT scans are the most common modality associated with discrep- ancies in the ED [1,5], therefore urging ED physicians to exercise caution when relying on radiology resident physicians’ interpreta- tions of CT scans, especially those of the abdomen and pelvis, due to their complexity and high error rates [6]. In addition, a recently published review article determined that failure to identify frac- tures was the most common diagnostic error associated with dis- crepancies in the ED [7], accounting for 41-80% of diagnostic errors, and that the most common fractures missed were those of the ankle/ft, lower arm, and hands/fingers [4]. Lastly, two pub-

Table 2

The most common final diagnostic interpretations associated with radiology discrepancies.

Missed diagnosis (n = 174) Site

medications, consultations, and hospitalizations, a small percent- age of patients required emergency surgical interventions and no patients sustained long term morbidity or mortality [1,3].

Errors in the interpretation of diagnostic imaging in the ED set-

Fracture

(N = 19, 10.9%)

Nasal/Facial (N = 4) Foot/Ankle (N = 4)

Long bones of arm/leg (N = 3) Finger (N = 2)

Toe (N = 2) Rib (N = 1)

Pelvis (N = 1) Scapula (N = 1) Vertebrae (N = 1)

ting accounts for a large percentage of malpractice cases. Several

studies have attempted to report rates of discrepancy amongst radiology physicians, at various levels of training and experience, in the interpretation of diagnostic testing [2,5]. A published study from the International Radiology Group (Dallas, Texas) compared disagreement rates amongst radiology attending physicians for 6703 diagnostic exams (not including CT or MRI scans) performed

between 1997 and 2001, and reported an overall error rate of

Mass or Cyst (N = 18, 10.3%) Renal (N = 3) Ovarian (N = 3) Hepatic (N = 1) Prostate (N = 1) Parotid (N = 1)

Supradiaphragmatic (N = 1) Uterus (N = 2)

Vaginal (N = 1) Thymus (N = 1) Colon (N = 1) Pancreas (N = 1) Breast (N = 1) Adrenal (N = 1)

Gastrointestinal inflammation (N = 11, 6.3%)

Pneumonia (N = 10, 5.7%)

Pulmonary Nodule (N = 9, 5.2%)

Gastrointestinal obstruction (N = 8, 4.6%)

Pulmonary Atelectasis (N = 7, 4.0%)

3.48%, with error rates specific to type of modality as general radi- ology (3.03%), diagnostic mammography (3.61%), screening mam- mography (5.79%), and ultrasound (4.07%) [5]. In addition, the Radiology Quality Institute collected data from 6 different studies representing 650,000 diagnostic exams, from both community and university based hospitals, and reported an overall error rate of 4.4%, ranging between 0.8% - 9.2%, depending on the diagnostic modality interpreted [2]. Furthermore, they determined that x- rays, mammograms, and ultrasounds had lower rates of error (3.48%) compared to CT scans of the abdomen and pelvis (7%) [2]. Based on our sample of patients, our rate of discrepancies (0.214%) was lower than that previously reported in the literature. Although these studies highlight the variability of diagnostic inter- pretation amongst radiology physicians, an accepted gold standard

Stone

(N = 6, 3.4%)

Ureteral (N = 3) Gallbladder (N = 2) Renal (N = 1)

”error rate” has yet to be determined. Many studies have not spec- ified whether interpretations from first year resident physicians are included in their error rate, nor if there was a statistically sig-

Bony lesions (axial skeleton, femur, frontal bone) (N = 4, 2.3%)

Lymphadenopathy (neck, pelvis, rectum) (N = 4, 2.3%) False positive findings (N = 13, 7.5%)

The remaining diagnoses included: pyelonephritis (2), abscess (2), bile duct dilation (2), hypodensities in solid organs (2), granulomatous disease (2), soft tissue swel- ling in anterior mediastinum (1), prevertebral soft tissue swelling by upper C spine

pulmonary embolism (1), unable to rule out pulmonary embolism due to increased image noise (1), appendicolith with mildly dilated appendix, could rep- resent early appendicitis (1), missed appendicitis (1), perforated appendix (1) ultrasound unable to rule out appendicitis in pregnant female with increased body habitus (1), likely hemorrhagic ovarian cyst, cannot rule out Ovarian torsion (1), gallbladder wall thickening (1), bladder wall thickening (1), distended gallbladder (1), air in urinary bladder (1), bronchitis (1), fluid collection in colon (1), stable infrarenal AAA (1), focal calcification in bladder wall - concern for malignancy (1), fat containing ventral hernia (1), subchorionic hematoma (1), bilateral inguinal hernias (1), calcified plaque in artery (1), dilation in infrarenal aorta (1), Nutcracker syndrome (1), retained IVC filter fragments (1), encephalomalacia (1), critical stenosis of right subclavian artery (1), chronic volume loss, scarring, and bronchiectasis in left lower lung (1), perinephric edema and hydroureteronephrosis (1), GJ tube incompletely visualized, position cannot be established (1), subdural hematoma (1), L4-L5 foraminal stenosis, occlusion of left vertebral artery (1), splenomegaly (1), buttock subcutaneous tissue edema (1), prevertebral soft tissue thickening in cervical spine (1), decompressed bowel loops (1), acute Inflammatory changes in tonsillar fossa (1), focus of fat extending through fascia into musculature in abdominal wall (1), intermetatarsal bursitis, Achilles tendinosis, calcaneocuboid degenerative changes (1), arthritic change with irregular narrowing and sclerosis involving C2-3 and L2-S1 (1), specular reflectors in cervix are nonspecific and may reflect IUD string (1), thickening of ligamentum flavum with posterior disc osteo- phyte complex (1), IUD is mispositioned (1), focus of fat in LUQ concerning for hematoma or infection (1), disc extrusion at C5-C6 and compression of cord (1), focus of fat on cephalic portion of uterus (1), demyelination in tissues, possible nerve sheath tumor (1), possible small calcified popliteal artery aneurysm (1), Epiploic appendagitis (1), concerning for triceps insertion avulsion injury (1), avascular lesion - likely endometrioma (1), tracheal lucency below vocal cords - cannot exclude underlying malignancy (1), radiodense opacity at base of bladder - likely a phlebolith (1), bony defect in maxillary sinus communicating with oral cavity, possible oroantral fistula or odontogenic sinusitis (1), and tear of peroneus longus tendon (1), metallic object in right lower quadrant adjacent to appendix, could represent a surgical clip - clinical correlation needs to be made with history of right lower quadrant surgeries (1).

lished studies examining diagnostic errors associated with ED patients determined that, although discrepancies may result in a few patients requiring additional interventions, such as imaging,

nificant difference in diagnostic discrepancy rate based on level of radiology resident physician [1,3,5,6]. A 2003 study in the Ameri- can Journal of Roentgenology which included first year radiology resident physicians (after 6 months of formal training), found that the number of radiologic discrepancies decreased with level of training, although it was found to not to be statistically significant [8]. At our institution, first year residents are not included in ”off hours” radiology preliminary reads. Our data did not let us deter- mine level of resident radiology physician between second, third, or fourth year residents. We believe that because some of the insti- tutions in previously published reports either include PGY-1 resi- dents (or do not mention whether or not first year residents are included) in their initial readings during nights or weekends, that it may be a part of the reason their error rates were much higher than our rate of discrepancies found at our institution.

Researchers continue to debate whether radiology attending physicians should be available in-house during ”off-hours” to supervise radiology resident physicians in the interpretations of diagnostic studies performed in the ED. Other studies have stressed the need for additional educational programs, peer review, as well as a comprehensive root cause analysis, so that radiology resident physicians may learn from their errors and adapt educational strategies in order to increase diagnostic accuracy in the ED. [7] With a minimal discrepancy rate based on our sample of patients, our results add to the existing literature that supports senior radi- ology resident physicians working without direct attending physi- cian supervision during ”off-hours”.

We have identified several limitations. Our study was con- ducted in a single rural, academic teaching hospital in central Pennsylvania and may not be generalizable to other EDs and radi- ology departments. Furthermore, we did not record the training level of the radiology resident physician or the experience/training of the advanced practice clinician, whose diagnostic interpretation and suggestion for appropriate disposition, respectively, may have affected the analysis of our data. Also, at our institution, ED physi- cians are available to provide guidance to advanced practice clini- cians regarding discrepancies and Call backs, but advanced practice

Table 3

Description of patients (n = 17) with radiology discrepancies who were instructed to return to the ED for further management (WNL = Within Normal Limits).

Age (yrs.)	Sex	Race	Modality	Location	Initial interpretation	Final interpretation	ED disposition	Additional time spent in ED
4	M	White	XR	Chest	WNL	Patchy opacity in left lower lobe, may	1. Inpatient	558 min
						represent infiltrate	hospitalization
13	F	Black	MRI	Abdomen/	WNL	Complex fluid collection with layered	1. Additional imaging (CT	192 min
				Pelvis		debris and an abnormal enlarged	abdomen/pelvis)
						appendix; perforated appendix	2. Specialist consult
							3. Surgical management
19	F	White	CT	Abdomen/	Dominant follicle or corpus	Urinary tract infection and hemorrhagic	1. Additional imaging	205 min
				Pelvis	luteum in ovary, otherwise	corpus luteum	(Transvaginal US)
					WNL		2. Additional medication
19	F	White	CT	Abdomen/	WNL	Appendix thicker than past study; may	(Antibiotics) 1. Specialist consult	299 min
				Pelvis		represent early appendicitis
27	F	White	XR	Foot	WNL	Step off at base of 2nd metatarsal; non-	1. Additional imaging	25 min
						displaced fracture vs. stress fracture	(Foot XR)
31	F	White	CT	Abdomen/	WNL	Focal inflammatory colitis vs. infectious	1. Specialist consult	228 min
				Pelvis		colitis	2. Additional medication
31	M	White	CT	Abdomen/	WNL	2 cm hypodensity in left kidney	(Pain medication) 1. Additional imaging (CT	404 min
				Pelvis			abdomen/pelvis with
							and without contrast)
35	F	White	CT	Chest	WNL	Emboli in left lower lobe, suggestive of PE	1. Additional imaging	181 min
							(CXR) 2. Additional medication (Anticoagulation)
36	M	White	CT	Abdomen/	Acute diverticulitis	Abscess with perforation to sigmoid	1. Surgical management	255 min
				Pelvis		colon
38	M	White	XR	Hip	WNL	Pelvis fracture	1. Additional imaging (CT	227 min

abdomen/pelvis)

Specialist consult

Additional medication (Pain medication)

38	F	White	CT	Abdomen/	Terminal ileitis	Dilated loops of bowel with moderate	1. Additional	imaging	248 min
				Pelvis		bowel obstruction	(Abdominal XR)
2. Specialist consult
42	F	White	CT	Abdomen/	Calcification posterior to	Stone obstructing Distal ureter	1. Additional	imaging	82 min
				Pelvis	distal ureter; phlebolith vs.		(Abdominal XR)
					ureteral calculus		2. Specialist consult
							3. Surgical management
47	M	White	CT	Neck	WNL	Acute inflammatory changes in left	1. Additional medication		176 min
						tonsillar fossa with mild inflammation	(Steroids, antibiotics)
50	M	White	CT	Abdomen/	WNL	Early partial bowel obstruction	1. Additional imaging		111 min
				Pelvis			(Abdominal XR)
2. Specialist consult 3. Surgical management
65	F	White	CT	Abdomen/	Concern for developing	Ileus vs. partial Small bowel obstruction	1. Additional	imaging	197 min
				Pelvis	abscess		(Abdominal XR)
2. Specialist consult

72 F White CT Abdomen/ Pelvis

Infectious vs. inflammatory colitis

3. Surgical management

Possible ischemic colitis 1. Specialist consult 111 min

74 F White XR Shoulder WNL Incidental finding; granulomatous disease in right lung

Pt did not return to ED

clinicians, who do not receive formal training on the interpretation of diagnostic imaging or the appropriate disposition based on dis- crepancies, are not required to precept with an ED attending physi- cian, and therefore the disposition provided to a patient associated with a discrepancy may be incorrect or inadequate. In addition, we did not analyze the rates of morbidity and mortality, or return to the ED, of patients who were instructed by the advanced practice clinician to follow up non-emergently with either their primary care provider or with a medical/surgical subspecialist. Lastly, we compared the initial diagnostic interpretation of a radiology resi- dent physician with the final interpretation of one radiology attending physician who was assigned to provide final readings that day, and as alluded previously, there are studies that demon- strate variability amongst radiology attending physicians and their final interpretations of diagnostic imaging studies. Therefore, a large multicenter study, including community-based and

academic-based general EDs in a variety of Geographic regions and settings (Inner city, rural, suburban), and analyzing various levels of training of radiology resident physicians and attending physicians, should be conducted in order to accurately determine the etiology and disposition associated with radiology discrepan- cies on ED patients.

Conclusions

Based on our sample, most discrepancies were associated with CT scans (modality), missed fractures (diagnosis), and non- emergent outpatient follow-up with a medical/surgical subspecial- ist (disposition). Furthermore, of the 16 patients that were asked to return emergently to our ED, 5 patients required surgical treat- ment and 1 patient required medical inpatient hospitalization, and there no mortalities reported in this subset. Therefore, educa-

tional interventions based on the etiology of radiology discrepan- cies provided to radiology resident physicians may further improve patient satisfaction for patients evaluated in the ED. Although we support having second, third, and fourth year radiology resident physicians working without direct attending physician supervision during ”off hours” at our institution due to low rates of discrepan- cies, we suggest that ED and radiology departments work collabo- ratively to monitor their own rates of discrepancies, and subsequent morbidities and mortalities, to improve patient care and potentially lessen ED overcrowding and patient dissatisfaction.

Declarations of interest

None.

Funding sources

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

References

1. Friedman SM, Merman E, Chopra A. Clinical impact of diagnostic imaging discrepancy by radiology trainees in an urban teaching hospital emergency department. Int J Emerg Med Jul 16 2013;6(1):24.
2. Radiology Quality Institute. diagnostic accuracy in radiology: defining a literature-based benchmark Available from: http://www. radisphereradiology.com/wp-content/uploads/Diagnostic-Accuracy-in- Radiology.
3. Chung JH, Strigel RM, Chew AR, et al. Overnight resident interpretation of torso CT at a level 1 trauma center: an analysis and review of the literature. Acad Radiol Sept 2009;16(9):1155-60.
4. Hallas P, Ellingsen T. Errors in fracture diagnoses in the emergency department - characteristics of patients and diurnal variation. BMC Emerg Med 2006;Feb16 (6):4.
5. Soffa DJ, Lewis RS, Sunshine JH, et al. Disagreement in interpretation: a method for the development of benchmarks for quality assurance in imaging. J Am Coll Radiol Mar 2004;1(3):212-7.
6. Tieng N, Grinberg D, Li SF. Discrepancies in interpretation of ED body computed tomographic scans by radiology residents. Am J Emerg Med Jan 2007;25 (1):45-8.
7. Pinto A, Reginelli A, Pinto F, et al. Errors in imaging patients in the emergency setting. Br J Radiol 2016;89(1061):20150914.
8. Carney E, Kempf J, Decarvalho V, et al. Preliminary interpretations of after-hours CT and sonography by radiology residents versus final interpretations by body imaging radiologists at a level 1 trauma center. Am J Roentgenol 2003;181:367-73.