a b s t r a c t

Introduction: Despite a relatively high frequency of appendix nonvisualization when using ultrasound to diagnose appendicitis, many studies either fail to report these results or inconsistently analyze outcomes. Objectives: The objective of this study is to determine the most transparent and accurate way of reporting and analyzing ultrasound results for the diagnosis of appendicitis.

Methods: This was an observational cohort study of emergency department patients age 18 years or older who underwent right lower quadrant ultrasonography from September 2010 to October 2011. Patient characteristics, imaging, pathology, and follow-up data were analyzed. Test characteristics were calculated using conventional 2 x 2 contingency table analysis excluding inconclusive ultrasound results and an intention-to-diagnose approach with a 3 x 2 table.

Results: Sixty-five patients were included. Forty-four (68%) patients had a nonvisualized appendix resulting in an overall diagnostic yield of 32%. Twenty-one patients had a visualized appendix (14 [22%] negative and 7 [11%] positive for appendicitis). Using 2 x 2 contingency table analysis, sensitivity and specificity were 100%. Using the 3 x 2 table with and the intention-to-diagnose principle, sensitivity was 70% and specificity was 25%. Three (7%) of 44 patients with a nonvisualized appendix had appendicitis (likelihood ratio = 0.40).

Discussion: We suggest reporting ultrasound results using a 3 x 2 table (including nonvisualized findings) but using the traditional 2 x 2 type of analysis for test characteristic calculations. This approach allows for the determination of diagnostic yield and calculation of likelihood ratios when the appendix is not visualized. This approach to reporting should be considered for all types of diagnostic Ultrasound studies.

(C) 2014

Introduction

The diagnosis of acute appendicitis in the adult population is challenging. Recent studies show that clinicians are increasingly using Computerized tomography as the initial study to diagnose appendicitis [1-3]. Limitations of CT include its cost and exposure to ionizing radiation leading several authors to suggest that ultrasound should be the initial diagnostic study for the evaluation of appendicitis [4-6]. Studies of ultrasound for suspected appendicitis have reported sensitivities from 77% to 99% and specificities from 86% to 92% [7,8]. Sensitivity and specificity, however, are not the only important test characteristics for appendiceal ultrasounds. Diagnostic yield, which is the likelihood that a test will provide the information needed to

? The study was presented at the 2012 American College of Emergency Physicians Research Forum.

* Corresponding author. Mayo Clinic, Rochester, MN 55905. Tel.: +1 507 255 6501;

fax: +1 507 255 6592.

E-mail address: [email protected] (V.R. Bellamkonda).

establish a diagnosis, is an important consideration when choosing a diagnostic test. Bowel gas, appendiceal rupture, or retrocecal location can make the appendix difficult to adequately visualize even by a skilled operator and can significantly decrease diagnostic yield.

In clinical practice, clinicians variably equate not visualizing an appendix on ultrasound to a negative study and, at other times, as inconclusive prompting further testing. Inconclusive results have been reported to occur as frequently as 15% to 35% [5,6,9] in studies of pediatric and adult patients. Inconclusive ultrasound results increase emergency department (ED) length of stay, health care costs, and contribute to a Delay in diagnosis, which increases the risk of perforation [10].

Despite the frequency with which the appendix is not visualized on ultrasound, many studies thus far have either failed to report inconclusive results or inconsistently analyzed outcomes when reporting ultrasound test characteristics. The use of traditional test characteristic reporting has the potential to misrepresent the utility of ultrasound for diagnosing appendicitis. In 2012, the British Medical Journal published Schuetz’s work on the use of 3 x 2 tables to evaluate

0735-6757/$ - see front matter (C) 2014 http://dx.doi.org/10.1016/j.ajem.2013.12.052

M. Fedko et al. / American Journal of Emergency Medicine 32 (2014) 346-348 347

CT angiography with an intention-to-diagnose methodology [11]. We aim to determine the most transparent and accurate way of reporting and analyzing ultrasound results for the diagnosis of appendicitis by

Table 1A

Summary of ultrasound results using traditional 2 x 2 table with evaluable results only

Appendicitis No appendicitis

comparing the intention with diagnose method and the traditional

2 x 2 contingency table analysis when analyzing and reporting ultrasound test characteristics for the diagnosis of appendicitis.

Methods

Visualized positive Visualized negative

7 0 PPV = 1.00 LR+ = infinity

0 14 NPV = 1.00 LR- = 0

Sensitivity = 1.00 Specificity = 1.00

Study design and setting

The study was an observational cohort study using electronic health record data gathered at Saint Mary’s Hospital ED, an academic medical center with approximately 73,000 annual patient visits and a Hospital admission rate of 30%. The center has continuous in-facility surgeon, radiologist, and sonographer availability.

Selection of participants

Consecutive ED patients age 18 years or older who underwent right lower quadrant ultrasonography from September 2010 to October 2011 were eligible for inclusion. We excluded patients with prior appendectomies, known inflammatory bowel disease, pregnan- cy, those who had CT before ultrasound, incomplete medical records, patients who declined research participation, and those without follow-up data available (defined as patients who were not seen within the Mayo Clinic Health Care System within 90 days of the index visit). The Mayo Clinic Institutional Review Board approved the study.

Data collection and outcome measures

Data were collected using a standardized data abstraction form. We abstracted data on patient demographics, ultrasound and CT results, and pathology reports. The true diagnosis of appendicitis was based on a pathology confirmation of a surgical specimen. True negatives for the purposes of this study were identified by a visualized negative appendix with lack of surgical intervention or other suggestive morbidity during the 90 days after ED evaluation. Attending level radiology interpretation of ultrasound studies that reported visualized appendices as negative or positive were classified as test negative or test positive, respectively for our study. Ultrasound interpretations that specifically stated an inability to visualize the appendix or did not report any visualization of the appendix were classified as nonvisualized in our study. Clinical Notes generated from follow-up visits and pathology reports were used to ascertain outcomes.

Statistical analysis

Continuous variables are reported as means, standard deviations, medians, and interquartile ranges (IQR). Categorical variables are summarized as frequency counts and percentages. Statistical analyses were performed using the SAS software package (SAS Institute, Cary, NC). Sensitivity, specificity, positive and negative predictive values, and likelihood ratios were calculated for diagnos- tic tests corresponding 95% confidence intervals (CIs) were calcu- lated. Follow-up information and pathology reports were used as outcome measures.

Results

Abbreviations: PPV, positive predictive value, NPV, negative predictive value, LR+, positive likelihood ratio; LR -, negative likelihood ratio.

ultrasound, 1 chart was incomplete, and 2 declined research participation. The final cohort consisted of 65 patients.

Of the 65 patients, 50 (77%) were female. The median age was 23 (IQR, 20-28) years. There were a total of 44 (68%) patients in whom the appendix was not visualized by ultrasound, 14 (22%) with appendix visualized and reported negative, 7 (11%) with appendix visualized and reported as positive for appendicitis. The overall diagnostic yield was 32% (21/65).

A summary of ultrasound results based on the presence or absence of appendicitis is shown in Table 1. The sensitivity, specificity, positive and negative predictive values, and positive and negative likelihood ratios of the ultrasound results are also shown. The first method based on the traditional 2 x 2 contingency table ignores inconclusive results entirely (Table 1A). Using this method, sensitivity was 100% (7/7; 95% CI, 56-100) and specificity was 100% (14/14; 95% CI, 73-100). The second method, an application of the intention-to-diagnose principle, considers inconclusive results as false negatives when calculating sensitivity and as false positives when calculating specificity [11] (Table 1B). Using this method, sensitivity was 70% (7/10; 95% CI, 35- 92) and specificity was 25% (14/55; 95% CI, 15-39). Table 1C shows a 3

x 2 table displaying nonvisualized results but uses Traditional methods to calculate test characteristics. Among patients with a nonvisualized, 3 (7%) of 44 had appendicitis. The likelihood ratio (defined as the ratio of the probability of a nonvisualized appendix in patients with appendicitis to the probability of a nonvisualized appendix in patients without appendicitis) was 0.40 (ratio of 3/10 to 41/55).

Discussion

When including only evaluable results, both the sensitivity and specificity of ultrasound in our study for the detection of appendicitis were 100% (Table 1A). However, sensitivity decreased to 70% and specificity to 25% using the intention-to-diagnose method (Table 1B). The overall diagnostic yield was only 32%, due to the high proportion of nonvisualized appendices (68%). Most patients (93%) with a nonvisualized appendix on ultrasound did not have appendicitis. Our results underscore the critical importance of reporting all study results when estimating the utility of a diagnostic test.

The intention-to-diagnose approach using a 3 x 2 table for calculating test characteristics was recently described for coronary CT angiography [11]. Using this method, the 2 x 2 table commonly used to calculate specificity and sensitivity is transformed into a 3 x 2

Table 1B

Summary of ultrasound results using 3 x 2 table intention-to-diagnose analysis Appendicitis No appendicitis

There were 95 eligible patients with abdominal pain who under-

Visualized

positive

7 41 PPV = 0.15 LR+ = 0.94

went right lower quadrant ultrasound by radiology during the study

Nonvisualized 3 + 0 41 + 0

period. Thirty patients were excluded: 17 were pregnant, 6 had no follow-up visit documented in the electronic medical record, 1 had inflammatory bowel disease, 3 patients had CT performed before

Visualized negative

3 14 NPV = 0.82 LR- = 1.18

Sensitivity = 0.70 Specificity = 0.25

348 M. Fedko et al. / American Journal of Emergency Medicine 32 (2014) 346-348

Table 1C

Summary of ultrasound results by final outcome, using 3 x 2 table and traditional analysis of test characteristics

	Appendicitis	No appendicitis
Visualized positive	7	0	PPV = 1.00	LR+ = infinity
Nonvisualized	3	41		LR = 0.4
Visualized negative	0 Sensitivity = 1.00	14 Specificity = 1.00	NPV = 1.00	LR- = 0

table resulting in more transparent reporting of all test results. From the 3 x 2 table, calculations of diagnostic yield as well as likelihood ratios can be performed for all results, including inconclusive studies. This method allows for transparent reporting of all results and determination of diagnostic yield. However, using this analysis will lead to an underestimation of test characteristics for evaluable results. When using the 2 x 2 table to report test characteristics, sensitivities, specificities, and likelihood ratios can be calculated only on evaluable results. Furthermore, there is a lack of consensus with regard to reporting inconclusive results with ultrasound for the diagnosis of appendicitis. Some studies exclude these results entirely and do not report the rate of appendix nonvisualization [6,9,12,13]. In these studies, the diagnostic yield and thus utility are unknown and possibly overestimated. Furthermore, no additional understanding can be gained about the likelihood of appendicitis in patients with inconclusive appendices from this type of reporting. Other studies consider nonvisualization to be equivalent to a visualized and negative appendix [9,13]. This method of interpretation also may lead to an inaccurate estimation of sensitivity and specificity for evaluable results and likewise does not permit the calculation of a

likelihood ratio for inconclusive results.

To promote transparent reporting and an accurate interpretation of diagnostic utility, we suggest reporting ultrasound results using the 3 x 2 table to clearly display all outcomes, including inconclusive ones, and reporting a diagnostic yield. However, rather than using the intention-to-diagnose method of analysis, we suggest using the traditional calculation of test results to prevent the underestimation of test characteristics for evaluable results. Furthermore, using this reporting method, likelihood ratios can be calculated for all outcomes. Likelihood ratios can be calculated for tests with more than 2 outcomes as demonstrated by Hayden and Brown [14]. Using the likelihood ratio then allows the practitioner to incorporate the results into clinical decision making. For example, if a patient had a Low pretest probability for appendicitis and a nonvisualized appendix, the clinician may decide to consider observation rather than further

evaluation with a CT.

Our study is limited by its small size and retrospective design. Furthermore, we have a higher rate of inconclusive results (68%) compared with prior studies that reported rates of 15% to 35%; [5,6,9] however, many studies have not specified the rate of inconclusive

results or may have included a nonvisualized appendix as a negative study. Our results underscore the importance of transparent report- ing of all test results. With the increasing use of ultrasound in emergency medicine, a standardized approach to reporting test characteristics will allow the clinician to most accurately estimate the diagnostic utility of a test and incorporate all test results into clinical decision making.

References

1. Broder J, Warshauer DM. Increasing utilization of computed tomography in the adult emergency department, 2000-2005. Emerg Radiol 2006;13(1):25-30.
2. Jones K, Pena AA, Dunn EL, Nadalo L, Mangram AJ. Are negative appendectomies still acceptable? Am J Surg 2004;188(6):748-54.
3. Storm-Dickerson TL, Horattas MC. What have we learned over the past 20 years about appendicitis in the elderly? Am J Surg 2003;185(3):198-201.
4. Sivit CJ, Siegel MJ, Applegate KE, Newman KD. When appendicitis is suspected in children. Radiographics 2001;21(1):247-62.
5. Schuh S, Man C, Cheng A, Murphy A, Mohanta A, Moineddin R, Tomlinson G, et al. Predictors of non-diagnostic ultrasound scanning in children with suspected appendicitis. J Pediatr 2011;158(1):112-8.
6. Park JS, Jeong JH, Lee JI, Lee JH, Park JK, Moon HJ. Accuracies of diagnostic methods for acute appendicitis. Am Surg 2013;79(1):101-6.
7. Balthazar EJ, Birnbaum BA, Yee J. Acute appendicitis: CT and US correlation in 100 patients. Radiology 1994;190:31-5.
8. Wilson EB, Cole JC, Nipper ML, Cooney DR, Smith RW. Computed tomography and ultrasonography in the diagnosis of appendicitis: when are they indicated? Arch Surg 2001;136(6):670-5.
9. Poortman P, Oostvogel HJ, Bosma E, Lohle PN, Cuesta MA, de Lange-de Klerk ES, Hamming JF. Improving diagnosis of acute appendicitis: results of a diagnostic pathway with standard use of ultrasonography followed by selective use of CT. J Am Coll Surg 2009;208(3):434-41.
10. Emil S, Mikhail P, Laberge JM, Flageole H, Nguyen LT, Shaw KS, Baican L, et al. Clinical versus sonographic evaluation of acute appendicitis in children: a comparison of patient characteristics and outcomes. J Pediatr Surg 2001;36(5): 780-3.
11. Schuetz GM, Schlattmann P, Dewey M. Use of 3×2 tables with an intention to diagnose approach to assess Clinical performance of diagnostic tests: meta- analytical evaluation of coronary CT angiography studies. BMJ 2012;345(e6717): 1-10.
12. Franke C, Bohner H, Yang Q, Ohmann C, Roher HD. Ultrasonography for diagnosis of acute appendicitis: results of a Prospective multicenter trial. Acute Abdominal Pain Study Group. World J Surg 1999;23(2):141-6.
13. Chan I, Bicknell SG, Graham M. Utility and diagnostic accuracy of sonography in detecting appendicitis in a community hospital, AJR. Am J Roentgenol 2005;184(6):1809-12.
14. Hayden SR, Brown MD. Likelihood ratio: a powerful tool for incorporating the results of a diagnostic test into clinical decision making. Ann Emerg Med 1999;33(5):575-80.