Article, Emergency Medicine

Prospective validation of a biomarker panel to identify pediatric ED patients with abdominal pain who are at low risk for acute appendicitis

Prospective validation of a biomarker panel to identify pediatric ED patients with abdominal pain who are at low risk for

acute appendicitis?,??,?

David S. Huckins, MD ^a^,?, Harold K. Simon, MD, MBA ^b, Karen Copeland, PhD ^c, Truman J. Milling Jr, MD ^d^,^e, Philip R. Spandorfer, MD, MSCE ^f, Halim Hennes, MD ^g,

Coburn Allen, MD ^h, Joseph Gogain, PhD ⁱ for the APAB Study Group ¹

^a Division of Emergency Medicine, Newton-Wellesley Hospital, Newton, MA

^b Emory University, Children’s Healthcare of Atlanta, Atlanta, GA

^c Boulder Statistics, LLC, Boulder, CO

^d Seton University of Texas Southwestern Clinical Research Institute, University Medical Center at Brackenridge, Austin, TX

^e Dell Children’s Medical Center, Austin, TX

^f Scottish Rite Hospital, Children’s Healthcare of Atlanta, Atlanta, GA

^g Children’s Medical Center, Dallas, TX

^h Dell Children’s Medical Center, Austin, TX

ⁱ Venaxis, Inc, Castle Rock, CO

a r t i c l e i n f o

Article history:

Received 18 September 2015

Received in revised form 8 January 2016

Accepted 30 March 2016

a b s t r a c t

Objectives: The objective of the study is to prospectively validate the diagnostic accuracy of a biomarker panel consisting of white blood cell, C-reactive protein, and myeloid-related protein 8/14 levels in identifying pediatric patients with abdominal pain who are at low risk for appendicitis.

Methods: This prospective observational study enrolled subjects aged 2 to 20 years presenting to 29 US emergen- cy departments with abdominal pain suggesting possible acute appendicitis. Blood samples were analyzed for white blood cell, C-reactive protein, and myeloid-related protein 8/14 levels from which the composite bio- marker panel results were calculated, then correlated with the final diagnosis either positive or negative for acute appendicitis.

Results: A total of 2201 patients were enrolled, with 1887 completing all aspects of the study. Prevalence of ap- pendicitis in this cohort was 25.3%. The biomarker panel exhibited a sensitivity of 97.1% (95% confidence interval [CI], 95.1%-98.2%), negative predictive value of 97.4% (95% CI, 95.8%-98.5%), negative likelihood ratio of 0.08 (95% CI, 0.05-0.13), with a specificity of 37.9% (95% CI, 35.4%-40.4%) for appendicitis. The panel correctly identified 534 (37.8%) of 1410 patients who did not have appendicitis with 14 false negatives (2.9%). Overall, 23.7% (132/557) of computed tomographic scans were done for patients with negative biomarker panel results, including 31.2% (131/420) of patients who had CT but did not have appendicitis.

? Funding: This research was funded by Venaxis, Inc, Castle Rock, CO.

?? Prior presentations: ACEP 2014, Chicago, IL, October 27, 2014.

? Conflict of interest disclosure: David S. Huckins, MD, paid consulting fees by Venaxis, Inc, for design, execution, and analysis of study results, and as member of Clinical Study Steering

Committee and Chair of Publication Committee; reimbursed expenses for travel to FDA meetings and conferences. Harold K. Simon, MD, MBA, served on the Venaxis Clinical Study Steering Committee and as a sponsored trial site investigator. Karen Copeland, PhD, paid consulting fees by Venaxis, Inc, as member of Clinical Study Steering Committee and for study design and analysis of study results. Truman J. Milling, Jr, MD, paid consulting fees by Venaxis, Inc, as Chair of Clinical Study Steering Committee. Philip R. Spandorfer, MD, sponsored trial site inves- tigator. Halim Hennes, MD, sponsored trial site investigator. Coburn Allen, MD, sponsored trial site investigator. Joseph Gogain, PhD, employed by Venaxis, Inc.

* Corresponding author at: Department of Emergency Medicine, Newton-Wellesley Hospital, 2014 Washington St., Newton, MA 02462.

¹ Acute Pediatric appendicitis Biomarker (APAB) Study Group David S. Huckins, MD-Newton-Wellesley Hospital, Newton, MA Harold K. Simon, MD, MBA-Emory University, Children’s Healthcare of Atlanta, Atlanta, GA Truman J. Milling, Jr, MD-Dell Children’s Medical Center, Austin, TX Philip R. Spandorfer, MD, MSCE-Children’s Health Care of Atlanta at Scottish Rite, Atlanta, GA Halim Hennes, MD-UT Southwestern, Children’s Medical Center, Dallas, TX Daniel M. Cohen, MD-Nationwide Children’s Hospital, Columbus, OH Coburn Allen, MD-Dell Children’s Medical Center, Austin, TX Blake Bulloch, MD-Phoenix Children’s Hospital, Phoenix, AZ James Linakis, MD-Hasbro Children’s Hospital, Providence, RI Holly Depinet, MD- Cincinnati Children’s Hospital, Cincinnati, OH Micheal Spohn, MD, FACEP-St Joseph’s Regional Health Center, Bryan, TX Robert Hickey, MD, FAAP, FAHA-Children’s Hospital of Pittsburgh, Pittsburgh, PA Cheryl Vance, MD-University of California, Davis, Sacramento, CA Phyllis Hendry, MD, FAAP, FACEP-University of Florida College of Medicine, Jacksonville, FL Rebecca Andrews-Dickert, MD-Spectrum Health, Grand Rapids, MI Norman A. Paradis, MD-Dartmouth-Hitchcock Medical Center, Lebanon, NH Jeffrey Johnson, MD-South Shore Hospital, Weymouth, MA Alex J. Rogers, MD-University of Michigan Hospital and Health System, Ann Arbor, MI Barry Hahn, MD, FACEP-Staten Island University Hospital, Staten Island, NY Shireen

M. Atabaki, MD, MPH-Children’s National Medical Center, Washington, DC David Spiro, MD-Oregon Health Sciences University, Portland, Oregon Howard A Smithline, MD-Baystate Medical Center, Springfield, MA Sharon E. Mace, MD-Cleveland Clinic, Cleveland, OH Keith Borg, MD-Medical University of South Carolina, Charleston, SC Don Arnold, MD, MPH-Vanderbilt Children’s Hospital, Nashville, TN Tom Abramo, MD-Arkansas Children’s Hospital, Little Rock, AK Kelly Sinclair, MD-Children’s Mercy Hospital, Kansas City, MO Wassam Rahman, MD-All Children’s Hospital, St. Petersburg, FL Milan Nadkarni, MD-Wake Forest Medical Center, Winston-Salem, NC.

http://dx.doi.org/10.1016/j.ajem.2016.03.066

Conclusion: This biomarker panel exhibited high sensitivity and negative predictive value for acute appendicitis in this large prospective cohort. This panel may be useful in identifying pediatric patients who are at low risk for appendicitis and might be followed clinically, potentially reducing the dependence on CT in the evaluation for acute appendicitis.

Introduction

Abdominal pain is the fourth leading cause for emergency depart- ment (ED) visits in the pediatric age group with acute appendicitis being the most common urgent surgical diagnosis [1,2]. Acute appendi- citis, however, can be a difficult diagnosis especially in children. Appen- dicitis is a common pediatric disease with a peak incidence in the second decade of life and a challenging diagnosis, as children often pres- ent with vague abdominal symptoms particularly in the very young or early in the course of disease [3-7]. The difficulty establishing the diag- nosis is manifested by the fact that missed acute appendicitis is the sec- ond leading cause of malpractice judgments against emergency physicians arising from patients in the pediatric age group [8,9]. A re- cent study noted that the diagnosis was missed in 4.8% of pediatric pa- tients with appendicitis at the time of initial ED evaluation [10]. For these reasons, acute appendicitis must be considered in the differential diagnosis of most pediatric and Adolescent patients with nonspecific ab- dominal pain. However, identifying those patients with appendicitis from among the many with abdominal pain from other more common causes can be problematic because, for most patients, the history, phys- ical examination, and standard laboratory tests alone lack adequate sen- sitivity (Se) and Specificity to reliably diagnose or exclude acute appendicitis [3-7]. The accuracy and reproducibility of clinical predic- tion rules, such as the Alvarado score and Pediatric Appendicitis Score (PAS), have also been inconsistent [11-17]. Advanced imaging studies such as ultrasound (US) or computed tomography (CT) are therefore often used to help confirm or rule out the diagnosis [18-21].

There has been a dramatic increase in CT utilization for abdominal

pain in the pediatric age group over the last decade raising concerns about the associated radiation exposure and the potential for Radiation-induced malignancy later in life [18-27]. This has led some authorities to recommend that US be used as the initial diagnostic imag- ing modality of choice for pediatric patients [28,29]. Ultrasound has ex- cellent Sp but has diminished Se compared to CT, particularly early in the course of disease or in settings where the sonographer and/or radi- ologist might have limited experience identifying acute appendicitis with US [30-36]. This often leads to CT being used as either the primary imaging modality or a second imaging procedure for those patients who have an inconclusive initial US [37-39]. Better diagnostic alternatives are needed to exclude the diagnosis in patients who do not have appen- dicitis to minimize radiation exposure from CT scans and limit other dis- advantages of advanced imaging.

We have previously described a biomarker panel with high Se and negative predictive value (NPV) to identify those pediatric ED patients with abdominal pain who are at low risk for acute appendicitis [39]. This investigation is a prospective validation study of the diagnostic ac- curacy of this biomarker panel in an independent cohort of pediatric and adolescent patients presenting to the ED with abdominal pain for whom the clinician considered acute appendicitis to be part of the dif- ferential diagnosis.

Methods

This study was a prospective observational industry sponsored in- vestigation to validate a previously described biomarker panel in pedi- atric and adolescent patients presenting to the ED with abdominal pain and/or other signs and symptoms suggesting acute appendicitis as part of the differential diagnosis. All sites obtained institutional

review board approval before study initiation, and all participating pa- tients provided written consent if older than 18 years or written paren- tal consent and patient assent for those younger than the age of 18 years per hospital specific institutional review board requirements. This study was sponsored by Venaxis, Inc, Castle Rock, CO.

This study was conducted at 29 pediatric, general academic, and community hospital EDs across the United States between January 2013 and January 2014. The study population is a convenience sample of patients aged 2 to 20 years inclusive presenting to participating EDs with abdominal pain suggesting possible acute appendicitis as deter- mined by the treating clinician. Patients were eligible whenever the treating physician felt appendicitis was in their initial differential diagno- sis. Those patients considered by the treating clinician to have a very high probability of acute appendicitis based on the initial clinical examination alone were designated as “high risk” and excluded based on predetermined criteria as outlined below. The study therefore includes patients ranging from low to moderately high likelihood of appendicitis. Patients were approached by study staff, who were hospital-based research physicians, nurses, and physician assistants, based on chief complaint and presenting symptoms during times of study staff avail- ability, primarily weekday and evening hours at the majority of sites. All patients who met inclusion criteria and agreed to participate were enrolled prospectively based on presenting symptoms and clinical as- sessment before any Advanced diagnostic imaging (US or CT). Patients who provided consent/assent had blood samples drawn for testing of the biomarker panel described below. All decisions regarding evalua- tion, other laboratory testing, imaging, and treatment were determined solely by the treating physicians independent of study participation. All treating physicians and care providers were blinded to the biomarker

panel results.

Inclusion criteria were ages from 2 to 20 years inclusive, right lower quadrant or generalized abdominal pain with other signs and symp- toms suggesting acute appendicitis as part of the differential diagnosis, duration of symptoms less than or equal to 72 hours, and the ability to obtain informed consent from the patient (age 18 years or older), or from the parent or guardian (patients younger than 18 years) with as- sent from the patient.

Exclusion criteria were a history of previous appendectomy, meta- static cancer, Bleeding disorder, or active autoimmune disorder. Patients were excluded if they were treated with any immunosuppressive med- ications within the preceding 28 days or systemic steroids within the preceding 14 days. Other exclusions included abdominal trauma, inva- sive abdominal procedures, or prior diagnostic imaging (other than sim- ple radiographs) for abdominal pain within the 2 weeks before study enrollment to avoid selection bias. Patients were also excluded if they were considered to be at high risk for acute appendicitis. High risk was defined as those patients who were considered by the treating physi- cian to have such a high probability of acute appendicitis based solely on the initial history, physical examination, and clinical gestalt, that it would be reasonable to proceed with appendectomy or exploratory lap- arotomy without the need for any diagnostic imaging. Patients were also excluded if they were prisoners of an adult or juvenile detention center or if the patient participated in other research protocols within the prior 30 days.

The biomarker panel under study consists of the combined values of total white blood cell count , plasma C-reactive protein level and plasma myeloid-related protein 8/14 (MRP 8/14 or calprotectin) level in a proprietary mathematical algorithm

expressed as x(WBC) + y(CRP) + z(MRP 8/14) + k = A, as previously described [39].

white blood cell values from each patient were obtained using the standard clinical laboratory process and hematology analyzer at each participating site. Whole blood samples for measurement of CRP and MRP 8/14 levels were drawn into 4-mL EDTA tubes either by direct ve- nipuncture or from an established peripheral intravenous catheter using standardized draw and flushing procedures for blood sampling from existing intravenous catheters. Blood draws for WBC determina- tions and for biomarker testing were required to be no more than 1 hour apart. The CRP and MRP 8/14 concentrations were then measured using the APPY1 Test System which uses a lateral flow sandwich immu- noassay. The concentrations obtained internally in the APPYReader for MRP 8/14 and CRP were combined with the subjects WBC value which is entered into the APPYReader by the operator. The APPYReader then calculates the APPY1 Test result. All sample testing was performed on site at each hospital in real time. Clinical staff were blind to all bio- marker test results. All site personnel performing the APPY1 Test were trained and qualified by the sponsor. Daily dry (electronic) and weekly liquid (procedural) quality control testing was performed at each site to ensure test kit and reader performance.

Standardized case report forms (CRFs) were used to collect demograph- ic, clinical, laboratory, imaging, and treatment data across all sites. Clinical data collected included medical history, comorbid conditions, and preced- ing treatment with antipyretics as well as 19 clinical data points including reported duration of illness, symptoms, signs, and physical findings to spe- cifically include those needed to calculate the Alvarado Score and PAS. Pa- tients who had missing data points necessary for the calculation of these Clinical scores were excluded from the clinical score analysis. Timing and re- sults of subsequent Abdominal ultrasound and/or CT imaging, disposition, and discharge diagnosis for all patients were recorded. Surgical pathology reports for those who had appendectomy, and results of follow-up tele- phone calls at 14 days (+-3 days) for all patients who did not have appen- dectomy at initial presentation were also recorded.

In compliance with International Conference on Harmonization Good Clinical Practice, study site staff transcribed all source data onto protocol CRFs. The APPY1 Test results obtained via a printout from the APPYReader were included in the CRF documentation. Case report forms were verified for accuracy by independent study monitors and transmitted to an independent data management company, and a final study database was transmitted to the sponsor for statistical anal- ysis. In addition, internal data (ie, CRP and MRP 8/14 concentrations) from the APPYReader were downloaded onto SD cards and sent to the data management company for download and storage.

Primary outcome measures were the results of the biomarker panel (the index test) and the presence or absence of appendicitis (the refer- ence standard) for each patient. The presence or absence of appendicitis was defined by the onsite clinical pathologist’s final interpretation of the surgical specimen of the appendix for those patients undergoing appen- dectomy and clinical diagnosis for those who did not. Those patients di- agnosed as negative for acute appendicitis at initial evaluation were contacted by study staff via telephone at 2 weeks (+-3 days) after dis- charge to confirm resolution of symptoms and the absence of subse- quent diagnosis of acute appendicitis. The biomarker panel results were classified as negative less than 4 and inconclusive greater than or equal to 4 as derived in the original study (observed range, 3.0-9.2) [39]. All biomarker panel testing was completed before the determina- tion of the patient’s final diagnosis, and pathologists evaluating the sur- gical specimens were blind to the biomarker panel results.

The primary study end point was the diagnostic accuracy of the bio- marker panel at the predetermined negative cutoff, as measured by the Se and Sp with 95% confidence intervals (CIs) in identifying those patients without appendicitis. Secondary end points included imaging data analysis with potential impact of the APPY1 Test on imaging utilization. Measure- ments, end points, data collection, and reporting of results follow the guide- lines of the Standards for Reporting of Diagnostic Accuracy initiative [40,41].

The study size of 2000 subjects was determined to provide a greater than 80% power of meeting the primary outcome (lower bound of a 2- sided 95% CI for Se not less than 90% and Sp not less than 35%) given true assumed values of Se greater than or equal to 0.944, Sp greater than or equal to 0.4, and prevalence greater than or equal to 0.2. This sample size was obtained through simulation to estimate the predictive probability of success of meeting the primary objective across a range of the expected Se (90%-96%), Sp (35%-43%), and prevalence (20%-26%) values. The subjects were to be collected from up to 30 Clinical sites with no one site contributing more than 20% of the subjects to the study, thus representing a variety of EDs.

Continuous normally distributed variables are expressed as mean (SD); and nonnormally distributed, as median (interquartile range [IQR]). Discrete variables are expressed as counts and percents with CIs as appropriate. A 2 x 2 table of the test outcomes vs diagnosis is pro- vided as well as performance estimates (Se, Sp, NPV, negative likelihood ratio [NLR], positive predictive value [PPV], and positive likelihood ratio) with 95% CI. Data analysis was performed by an independent stat- istician (KC) with JMP Pro (v11; SAS Institute, Cary, NC).

Results

From January 2013 through January 2014, pediatric and adolescent patients presenting to the EDs of 29 participating institutions with symptoms suggesting acute appendicitis as part of their differential di- agnosis were approached for consent to participate. There were 2201 patients enrolled, of whom 314 were excluded for ineligibility, lack of adequate samples, improper sample processing, or lost to follow-up, leaving 1887 patients in the final study sample.

The final study sample of 1887 patients was 55% female with a me-

dian patient age of 12 years (IQR, 8-15). age distribution is presented in Fig. 1. The overall prevalence of appendicitis in the sample was 25.3%, with a prevalence of 34.7% in male patients and 17.7% in female patients. Patient demographics and characteristics are displayed in Table 1. En- rollment, exclusions, Diagnostic pathway, imaging, treatment, and dis- position information for the patient sample are shown in Fig. 2.

The composite biomarker panel exhibited an Se of 97.1% (95% CI, 95.1%-98.2%), an NPV of 97.4% (95% CI, 95.8%-98.5%), an NLR of 0.08

(95% CI, 0.05-0.13), and an Sp of 37.9% (95% CI, 35.4%-40.4%) for acute

appendicitis. The panel correctly identified 534 (37.8%) of 1410 patients who did not have appendicitis with 14 false negatives (2.9%). The corre- lation of the composite biomarker panel results to the diagnosis of acute appendicitis is shown in Table 2.

The overall performance characteristics of the biomarker panel are displayed in Table 3 along with the results of exploratory subgroup analysis of the diagnostic accuracy by duration of symptoms and patient age. Detailed imaging, diagnostic, and treatment data comparing pa- tients with negative vs inconclusive biomarker panel results are shown in Fig. 3. Data regarding the clinical characteristics of the 14 false-negative patients are presented in Table 4.

There were another 14 patients who were diagnosed as having acute appendicitis by imaging alone who did not have surgery within the time frame of the study. Some of these patients were diagnosed as Perforated appendicitis at initial presentation and treated with antibiotics, with or without drainage via interventional radiology, with surgery delayed or deferred. Others were diagnosed as mild early appendicitis and treated with antibiotics alone, with symptoms resolving without the need for sur- gery. Although these patients are included in the primary analysis as pos- itive for acute appendicitis, the definitive diagnosis was not pathologically proven. For this reason, we also performed the primary analysis with these patients both excluded and with these patients classified as nega- tive for acute appendicitis. These results are also shown in Table 3.

Analysis of imaging data revealed that 84.1% (1587/1887) of all pa- tients had imaging after arrival to the ED: 73.4% (1386/1887) had US, 29.5% (557/1887) had CT, and 18.9% (356/1887) had both US and CT. Fifteen patients (15/1887, 0.8%) had US and magnetic resonance

160

140

120

100

N 80

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Age (years)

Total Appendicitis

Fig. 1. Age distribution.

imaging (MRI). Among patients who had imaging but did not have ap- pendicitis, 31.2% (131/420) who had CT and 40% (410/1021) who had US were correctly identified by negative biomarker panel results. For the entire patient population, 23.7% (132/557) of all CT scans and 31% (423/1386) of all US were done for patients with negative biomarker panel results. Data regarding imaging utilization, enrolling institution, and prevalence of appendicitis are displayed in Table 5.

As a basis for comparison, we calculated the Alvarado Score and PAS for all patients as negative predictors using negative cutoffs of less than

5 and less than 4 as suggested by a recent meta-analysis [42]. Based on previous reports in the literature using the 2-marker combination of WBC and CRP as a diagnostic marker for acute appendicitis, we also per- formed a post hoc analysis of this combination [43-48]. Our post hoc analysis used the hospital WBC with the CRP concentration obtained from the APPYReader. We used the combination of normal WBC and CRP less than 0.8 mg/dL and also WBC less than 10 k/uL in combination with a CRP less than 0.8 mg/dL. The 2-marker combination was consid- ered negative when both markers were below the negative cutoff and

Table 1

Patient demographics and clinical characteristics

	Not appendicitis	Appendicitis	Pa	Biomarker negative	Biomarker inconclusive	Pa
Total, N (N/1887%)	1410 (75%)	477 (25%)		548 (29%)	1339 (71%)
Patient data, n (n/N%)
Age, y Median (IQR)	12 (8-15)	11 (9-15)	.8808	13 (10-15)	11 (7-15)	b.0001
<=12	784 (56%)	289 (61%)	.0693	251 (46%)	822 (61%)	b.0001
Sex
Male 555 (39%)		293 (62%)	b.0001	213 (39%)	635 (47%)	.0008
Ethnicity
White	787 (56%)	269 (56%)	b.0001	332 (61%)	724 (54%)	.0008
Black	231 (16%)	31 (7%)		83 (15%)	179 (13%)
Hispanic	320 (23%)	144 (32%)		103 (19%)	372 (28%)
Other/unreported	72 (5%)	22 (5%)		30 (5%)	64 (5%)
Pain durationb 0-12 h	542 (39%)	141 (30%)	b.0001	196 (36%)	487 (36%)	.2701
12-24 h	425 (30%)	195 (41%)		166 (30%)	454 (34%)
24-48 h	310 (22%)	106 (22%)		135 (25%)	279 (21%)
48-72 h	133 (9%)	35 (7%)		49 (9%)	119 (9%)
Imaging US only	732 (52%)	283 (59%)	.0043	325 (59%)	690 (52%)	.0046
CT only	143 (10%)	58 (12%)		39 (7%)	162 (12%)
Both US and CT	277 (20%)	79 (17%)		93 (17%)	263 (20%)
No imaging	246 (17%)	54 (11%)		86 (16%)	214 (16%)
Both US and MRI	11 (b1%)	4 (b 1%)		5 (b1%)	10 (b1%)
Diagnosis
Abdominal pain	694 (49%)		298 (54%)		396 (30%)
Constipation	160 (11%)		72 (13%)		88 (7%)
Gastroenteritis	91 (7%)		18 (3%)		73 (5%)
Ovarian cyst	66 (5%)		33 (6%)		34 (2%)
UTI	47 (3%)		9 (2%)		38 (3%)
Colitis	12 (b1%)		0		12 (1%)
Other	340 (24%)		104 (19%)		235 (18%)
Appendicitis total		477 (100%)	14 (3%)		463 (35%)
Perforated app.		102 (21%)	0		102 (8%)
Institution type Pediatric specialty	936 (66%)	361 (76%)	b.0001	355 (65%)	942 (770%)	.0622
General tertiary care	209 (15%)	40 (8%)		82 (20%)	167 (17%)
Community hospital	265 (19%)	76 (16%)		111 (15%)	230 (13%)

^a P values are based on t tailed tests: Fisher exact test for 2-level categorical variables; Pearson ?² for multilevel categorical variables; Wilcoxon/Kruskal-Wallis test for median age.

^b Pain duration, 2 missing data points.

Subjects enrolled n = 2201

Excluded n = 314

Subjects in Cohort n = 1887

Processing error = 67

CBC >1h before sample = 64 Imaging before consent = 62 LTFU = 58

Unable to obtain sample = 34 Ineligible = 21

Other = 8

Discharged home w/o imaging

n = 246

Subjects to OR w/o imaging

n = 54

Appendicitis n = 2*

No appendicitis n = 244

Appendicitis n = 52

No appendicitis n = 2

Imaging

n = 1587

n = 1386

n = 201

MRI

n = 15

n = 1386

n = 356

n = 557

Home AA+ n = 0 AA- n = 11

Antibiotics AA+ n = 0 AA- n = 0

Home AA+ n = 1* AA- n = 718

Antibiotics AA+ n = 9 AA- n = 0

Home AA+ n = 1* AA- n = 411

Antibiotics AA+ n = 5 AA- n = 0

Surgery AA+ n = 4 AA- n = 0

Surgery AA+ n = 272 AA- n = 15

Surgery AA+ n = 131 AA- n = 10

AA+ = acute appendicitis, AA- = no acute appendicitis, LTFU = Lost to follow up.

*captured on return visit or follow up

Fig. 2. Patient disposition. AA+, acute appendicitis; AA-, no acute appendicitis; LTFU, lost to follow-up. ?Captured on return visit or follow-up.

positive if either marker was elevated. These results are shown in Table 6.

Discussion

This composite biomarker panel uses a multivariate algorithm to combine 3 markers into a single score from which the classification of low risk for acute appendicitis can be determined. This algorithm, devel- oped from a previous sample of 503 subjects, is validated here in this large independent prospective cohort of 1887 pediatric and adolescent patients with abdominal pain who were enrolled based on their initial

Table 2 Biomarker panel results
	No appendicitis	Acute appendicitis	Total
Biomarker panel negative	534	14	548
Biomarker panel inconclusive	876	463	1339
Total	1410	477	1887

presenting symptoms suggesting acute appendicitis as part of their dif- ferential diagnosis [39]. The diagnostic accuracy of the biomarker panel is demonstrated in this study with an Se of 97.1% (95% CI, 95.1%-98.2%), an NPV of 97.4% (95% CI, 95.8%-98.5%), and an Sp of 37.9% (95% CI,

35.4%-40.4%) for acute appendicitis, identifying 37.8% (534/1410) of pa- tients who did not have appendicitis.

Evaluating children with abdominal pain for possible acute appendi- citis has become increasingly dependent on advanced imaging, primar- ily US and CT [20,21,30-33]. The overall imaging rate in this cohort was 84.1% (1587/1887). Although the improved diagnostic accuracy provid- ed by advanced imaging has reportedly reduced the negative appendec- tomy and Perforation rates, this dependence also has potential disadvantages such as increased cost; prolonged ED length of stay; and, in particular, ionizing radiation exposure from CT [18-27,49-56]. Ultrasound is currently recommended by many authorities as the pre- ferred first imaging modality for pediatric patients to limit radiation ex- posure from CT [24,25]. Although US utilization has increased, studies have shown inconsistency in the Se of US for detecting acute appendici- tis with variability noted from institution to institution, with frequency

Table 3

Biomarker panel diagnostic performance characteristics

n = AA+/total	Se (95% CI)	Sp (95% CI)	NPV (95% CI)	NLR (95% CI)	PPV (95% CI)	PLR (95% CI)
All patients (477/1887)	97.1% (95.1-98.2%)	37.9% (35.4-40.4%)	97.4% (95.8-98.5%)	0.08 (0.05-0.13)	34.6% (32.1-37.2)	1.56 (1.49-1.63)
Duration Sx
b12 h (141/683)	97.9% (93.9-99.3%)	35.7% (31.8-39.8%)	98.5% (95.6-99.5%)	0.06 (0.02-0.18)	28.3% (24.5-32.5)	1.52 (1.42-1.63)
12-24 h (195/620)	96.9% (93.5-98.6%)	37.7% (33.3-42.4%)	96.4% (92.3-98.3%)	0.08 (0.04-0.18)	41.9% (37.4-46.4)	1.56 (1.44-1.68)
24-36 h (50/155)	94.0% (83.8-97.9%)	35.2% (26.8-44.7%)	92.5% (80.1-97.4%)	0.17 (0.06-0.53)	40.9% (32.3-50.0)	1.45 (1.24-1.70)
36-48 h (56/258)	94.6% (85.4-98.2%)	45.3% (38.6-52.2%)	96.8% (91.1-98.9%)	0.12 (0.04-0.36)	32.3% (25.6-39.8)	1.73 (1.51-1.99)
48-72 h (35/168)	100.0% (90.1-100.0%)	36.8% (29.1-45.3%)	100.0% (92.7-100.0%)	NA	29.4% (22.0-38.1)	1.58 (1.39-1.80)
Age <= 12 (288/784)	97.9% (95.5-99.0%)	31.3% (28.1-34.6%)	97.6% (94.9-98.9%)	0.07 (0.03-0.15)	34.4% (31.3-37.7)	1.42 (1.36-150)
N 12 (189/625)	95.2% (91.2-97.5%)	46.1% (42.2-50.0%)	97.0% (94.3-98.4%)	0.10 (0.05-0.20)	34.8% (30.8-39.0)	1.77 (1.63-1.91)
AA+ patients w/o path (14) Excluded	97.0% (95.0-98.2%)	37.8% (35.3-40.4%)	97.3% (95.5-98.3%)	0.08 (0.05-0.14)	33.9% (31.4-36.5)	1.56 (1.49-1.63)
Included as AA-	97.0% (95.0-98.2%)	37.5% (35.0-40.0%)	97.4% (95.8-98.5%)	0.08 (0.05-0.13)	33.6% (31.1-36.2)	1.55 (1.49-1.62)

Abbreviations: AA+, positive for acute appendicitis; Sx, symptoms; AA+ patients w/o path, patients diagnosed as positive for acute appendicitis without pathologic confirmation; AA-, negative for acute appendicitis; PLR, positive likelihood ratio.

Biomarker negative n = 548

Biomarker inconclusive n = 1339

Surgery

n = 1

AA+= 1

AA- = 0

Discharge home

n = 85

AA+ = 0

AA- = 85

Surgery

n = 53

AA+= 51

AA- = 2

Discharge Home

n = 161 AA+= 2** AA- = 159

Imaging

n = 462

Imaging

n = 1125

n = 39

n = 162

*Surgery

n = 21

AA+= 12

AA- = 9

Discharge Home

n = 304

AA+ = 0

AA- = 304+

*Surgery

n = 275

AA+= 269

AA- = 6

Discharge Home

n = 415 AA+ = 1** AA- = 415

n = 98

n = 273

n = 5

n = 93

n = 263

n = 10

n = 132

n = 425

Surgery

n = 0

AA+= 0

AA- = 0

Discharge Home

n = 5

AA+ = 0

AA- = 5

Surgery

n = 1

AA+= 1

AA- = 0

Discharge Home

n = 131

AA+ = 0

AA- = 131

*Surgery

n = 145

AA+= 135

AA- = 10

Discharge Home

n = 280 AA+ = 1** AA- = 279

*Surgery

n = 4

AA+= 4

AA- = 0

Discharge Home

n = 6

AA+ = 0

AA- = 6

MRI

n = 10

MRI

n = 5

n = 963

n = 423

Subjects in cohort

n =1887

AA+ = acute appendicitis, AA- = no acute appendicitis, *Includes those diagnosed as AA+ but treated with antibiotics only, **Captured on return visit or follow up, +one Negative appendectomy on follow up

Fig. 3. Diagnosis and disposition by biomarker results. AA+, acute appendicitis; AA-, no acute appendicitis. ?Includes those diagnosed as AA+ but treated with antibiotics only. ??Cap- tured on return visit or follow-up. ⁺One negative appendectomy on follow-up.

Table 4

Patient characteristics with false-negative biomarker results

Pt.	Age/sex	Duration Sx (h)	US	CT	WBC (k/uL)	Biomarker result	Clinical course	Pathology
1	13/M	0-12	I	ND	9.6	3.90	Appy at initial visit	“acute mucosal app.”
2	12/F	0-12	P	ND	10.5	3.86	Appy at initial visit	“acute appendicitis”
3	16/F	0-12	P	ND	7.6	3.55	Appy at initial visit	“acute appendicitis”
4	11/M	12-24	P	ND	7.7	3.80	Appy at initial visit	“acute appendicitis”
5	17/F	12-24	P	ND	9.6	3.89	Appy at initial visit	“transmural acute app.”
6	13/M	12-24	P	ND	9.1	3.71	Appy at initial visit	“acute appendicitis”
7	10/M	12-24	P	ND	5.8	3.44	Appy at initial visit	“acute appendicitis”
8	10/M	12-24	P	ND	6.6	3.40	Rx ATB only, not perf.	None
9	11/M	24-36	I	ND	9.5	3.82	Appy at initial visit	“acute appendicitis”
10	16/F	24-36	N	P	8.4	3.75	Appy at initial visit	“mild appendicitis”
11	14/M	24-36	ND	ND	8.6	3.83	Appy at initial visit	“acute app. w/serositis”
12	14/M	36-48	P	ND	9.1	3.82	Appy at initial visit	“acute appendicitis”
13	13/M	36-48	P	ND	7.3	3.81	Appy at initial visit	“acute appendicitis”
14	15/F	36-48	P	ND	6.3	3.72	Appy at initial visit	“acute appendicitis”

Abbreviations: Pt, patient; I, inconclusive for AA; P, positive for AA; N, negative for AA; ND,not done; Appy, appendectomy; app., appendicitis; perf., perforation.

of use, and with duration of disease [6,30-38]. Another major limitation of US is the large number of nondiagnostic studies [33-38]. For these reasons, the American College of Emergency Physicians recommends US as a modality to confirm acute appendicitis but not to exclude it [57]. Patients who do not have a clear diagnosis after US often proceed to CT [33-38]. In the current study, 26.8% (371/1386) of patients who were initially imaged with US subsequently had CT, as did 32.4% (97/ 299) in our previous study [39]. Although the risk of missed appendicitis clearly outweighs the risk of radiation exposure from a single CT scan for patients with the disease, the same is not necessarily true for the major- ity of these patients who do not have appendicitis.

This biomarker panel may be a useful tool to identify a substantial proportion of patients who do not have appendicitis and might avoid the radiation exposure of CT and other disadvantages of advanced imag- ing at initial presentation. It is difficult, however, to accurately quantify the potential for imaging reduction based on the use of this biomarker panel, as some patients may still require imaging even after negative biomarker panel results to evaluate for diagnoses other than acute appendicitis.

The clinical utility of this biomarker panel is not limited exclusively to imaging reduction. The laboratory turnaround time for the biomarker panel result is less than 30 minutes which can help clinicians focus more quickly on Alternative diagnoses once acute appendicitis is rendered un- likely. This impacts decisions regarding whether (or what type of) diag- nostic imaging is needed, if other diagnostic testing is necessary, and what initial treatment is most appropriate. Some institutions use proto- cols that emphasize Surgical consultation before or in preference to ra- diologic imaging [58]. The identification of a substantial portion of these patients as low risk for acute appendicitis might also reduce re- quests for surgical consultation and the subsequent time demands on surgical staff.

There were 28 patients in this study who had a normal appendix on pathologic examination (negative appendectomy), resulting in a nega- tive appendectomy rate of 5.7% (28/491). The biomarker panel correctly identified 35.7% (10/28) of these patients, similar to the 36.4% (4/11) of patients with negative appendectomies in our previous study [39]. Al- though the negative appendectomy rate in the United States has de- clined substantially during this recent period of increased imaging

utilization from the previously expected 15% to 20% to 3% to 8%, it might be further reduced with use of this biomarker panel [49-53].

The primary safety issue regarding use of this biomarker panel is that some patients with appendicitis might be missed on initial presentation due to false-negative results, delaying the diagnosis and increasing the risk of perforation with its attendant complications and morbidities. There were 14 (2.9%) of 477 patients with appendicitis who were false negative by this biomarker panel (Table 4). However, 8 of 14 false- negative patients had symptoms for less than 24 hours, 6 had symptoms for less than 48 hours, and no patients with perforation had false- negative results. Perforation is uncommon in patients with symptoms less than 24 hours, so there remains a window of opportunity for diag- nosis before perforation is likely [59,60]. Standard clinical practice would dictate that patients for whom imaging, further testing, or treat- ment is deferred at initial evaluation be given instructions to return if symptoms persist or worsen, or alternatively, be admitted for observa- tion. Those few patients with false-negative biomarker results whose symptoms persist or progress should be reevaluated, at which time im- aging could be reconsidered.

The false-negative rate of 2.9% for the biomarker panel in this study is comparable to the 2% to 5% false-negative rate reported in the litera- ture for the current standard of care using high rates of advanced imag- ing [10,61,62]. Additionally, the NPV of 97.4% in this patient sample also reduces the likelihood of appendicitis in biomarker-negative patients to less than 3%, the testing threshold suggested by Ebell and endorsed by Kharbanda, below which observation rather than further diagnostic testing is an acceptable alternative [42,63].

Ten of the 14 biomarker false-negative patients were also correctly diagnosed by US (Table 4). The Se of the biomarker panel alone was 97.1% (95% CI, 95.1%-98.2%), whereas the combination of the biomarker panel and US as currently used had an Se of 99.2% (95% CI, 98.8%-99.9%) even when nondiagnostic US were considered negative if combined with a negative biomarker panel result. This suggests that the biomark- er panel in combination with US could further improve overall NPV while also providing clinicians support in deferring further imaging after inconclusive US results for biomarker-negative patients. This ad- dresses one of the primary limitations of US for this indication, that being the large number of US which are nondiagnostic for acute

Table 5

hospital type, prevalence of appendicitis, and imaging utilization

Hospital type	n	Appendicitis, % (n/N)	CT, % (n/N)	US, % (n/N)	US and CT, % (n/N)	US and MRI, % (n/N)
All hospitals	1887	25.3% (478/1887)	29.5% (557/1887)	73.5% (1386/1887)	18.9% (356/1887)	b 1.0% (15/1887)
Pediatric tertiary care	1297	27.9% (362/1297)	22.0% (285/1297)	78.8% (1022/1297)	16.7% (216/1297)	1.1% (14/1297)
General tertiary care	249	16.1% (40/249)	30.5% (76/249)	73.5% (183/249)	22.1% (55/249)	b 1.0% (1/249)
General community	341	22.3% (76/341)	57.5% (196/341)	53.1% (181/341)	24.9% (85/341)	0.0% (0/341)

Imaging totals more than 100% as those with both US/CT and US/MRI are also included in individual totals for US, CT, and MRI.

Table 6

Comparison of negative predictors

Predictor	n	Se, % (95% CI)	Sp, % (95% CI)	NPV, % (95% CI)	NLR (95% CI)	TN, % (n¹/N¹)	FN, % (n²/N²)
Biomarker panel	1887	97.1 (95.1-98.2)	37.9 (35-4-40.4)	97.4 (95.8-98.5)	0.08 (0.05-0.13)	37.8 (534/1410)	2.9 (14/477)
Alvarado b5	1816	92.8 (90.1-94.8)	43.6 (41.0-46.3)	94.7 (92.7-96.2)	0.16 (0.12-0.23)	43.6 (592/1357)	7.2 (33/459)
PAS b5	1825	94.1 (91.6-95.9)	39.1 (36.5-41.7)	95.2 (93.1-96.7)	0.15 (0.10-0.22)	39.1 (533/1364)	5.9 (27/461)
Alvarado b4	1816	96.7 (94.7-98.0)	26.1 (23.8-28.5)	95.9 (93.4-97.5)	0.13 (0.08-0.21)	26.1 (354/1357)	3.3 (15/459)
PAS b4	1825	97.8 (96.1-98.8)	21.4 (19.3-23.7)	96.7 (94.0-98.2)	0.10 (0.05-0.19)	21.4 (292/1364)	2.2 (10/461)
WBC Nml/CRP b0.8	1886	91.4 (88.5-93.6)	54.7 (52.1-57.3)	95.0 (93.2-96.3)	0.16 (0.12-0.21)	54.7 (771/1410)	8.6 (41/476)
WBC b 10 k/CRP b0.8	1886	96.0 (93.9-97.4)	45.0 (42.4-47.6)	97.1 (95.5-98.1)	0.09 (0.06-0.14)	45.0 (634/1410)	4.0 (19/476)

Abbreviations: TN, true negative; n¹, true negatives identified; N¹, total true negatives; FN, false negative; n², false negatives, N², total appendicitis; Nml, normal; b0.8, less than 0.8 mg/dL;

b10 k, less than 10 k/uL.

appendicitis, which includes 56.9% of US in this cohort [33-37]. In our sample, a single patient in the biomarker-negative group was diagnosed with acute appendicitis by CT after an initial US, such that 132 CT scans (23.7% of the total) were done in biomarker-negative patients with a di- agnostic yield of a single case of acute appendicitis.

The safe use of this test is also dependent on the recognition that this test is not appropriate to use for patients who are considered to be high risk on clinical grounds alone, which was the rationale for exclusion of high-risk patients from the study. The reliability of a negative test in any in- dividual patient is dependent on the pretest probability, just as the NPV in a given population is dependent on the prevalence of disease [64]. For a pa- tient with a very high pretest probability or a population with a very high prevalence of disease, the likelihood that any negative test result is false negative is proportionally higher, making that negative result less reliable. The performance of the clinical prediction rules is shown in Table 6.

By comparison, an Alvarado Score less than 5 was significantly less sen- sitive than the biomarker panel, although the 95% CIs for NPV overlap slightly at the upper and lower bounds in this patient population with a prevalence of appendicitis of 25.3%. A PAS less than 5 also overlapped at the upper and lower bounds of the 95% CIs for both Se and NPV with comparable Sp to the biomarker panel, but with nearly twice as many false negatives. An Alvarado score less than 4 and a PAS less than 4 pro- vided comparable Se and NPV but with much lower Sp, identifying only 26.1% and 21.4% of true-negative patients, respectively, compared to 37.8% for the biomarker panel.

The combination of a normal WBC and CRP less than 0.8 mg/dL also had significantly less Se than the biomarker panel, whereas the combi- nation of WBC less than 10 k/uL and CRP less than 0.8 mg/dL resulted in substantial overlap of the 95% CIs for Se and NPV. However, the difficulty in using fixed cutoffs for WBC and CRP is highlighted by the variety of reference ranges across clinical sites in our study. Only 6 of the 29 sites had a single normal WBC range for children ages 2 to 20 years, whereas the other sites had between 2 and 8 different reference ranges for WBC based on age and/or sex. The upper normal limits of WBC ranged from 8.8 k/uL (for 19- to 20-year-old males) to 18 k/uL (for chil- dren younger than 3), with a median of 13 k/uL. The upper normal limits of CRP also varied across sites, ranging from 0.3 to 1.2 mg/dL depending on the assay used. Therefore, applying fixed cutoffs reported in the liter- ature to clinical practice is limited by the problematic translation of cor- responding values for WBC and CRP to specific practice sites where different reference ranges and different assays may be in use. In addi- tion, as noted in the limitations, our CRP values were obtained from the APPYReader, and it is not clear how these levels correlate to the va- riety of currently used commercial assays.

A potential advantage of the 3 biomarker panel is that the result is not dependent on fixed cutoffs of individual markers that may have dif- ferent standards or different assays between sites or potentially variable assessments of clinical findings in the application of Clinical scores. The biomarker panel is an objective and reproducible laboratory test based on the overall contribution of the 3 markers and their relationship in a specific patient. The amount each marker contributes to the overall test result varies from patient to patient. In our cohort, the contribution of WBC ranges from 12% of the test score in some patients to 92% in

others; CRP, from 0% to 51%; and MRP 8/14, from 8% to 72%. The variable contribution across patients leads to a potentially more robust test as compared to tests using fixed cutoffs for each marker (such as in the WBC-CRP comparison). Furthermore, the biomarker panel provides a single uniform standardized test result that is independent of site- specific reference ranges and can be used regardless of site of care.

Any potential benefits of this biomarker panel are dependent on ap-

propriate use by clinicians. It must be emphasized that this test is a 1- way binary test. The clinical utility is based solely in the negative test correctly and safely identifying patients who are unlikely to have acute appendicitis and might be followed clinically without the need for initial CT imaging. A result above the negative cutoff has no clinical value as the Sp and PPV are low and the test is not a reliable positive pre- dictor. With an inconclusive result, further imaging, testing, or treat- ment should be based entirely on clinical assessment. Any benefits associated with imaging reduction in the biomarker negative popula- tion might be lost if clinicians inappropriately pursue imaging driven by inconclusive results.

The strengths of this study are the large independent prospective pa- tient cohort from multiple institutions across the United States with a broad ethnic and geographic distribution, representing a de novo pa- tient population for whom this test has direct clinical relevance. In addi- tion, plasma samples were run in real time on site as would be done in normal clinical practice by hospital staff at the enrolling institution. Cli- nicians caring for the patients were unaware of the results of the bio- marker panel so that diagnostic testing and Treatment decisions were unaffected by test results, and all included patients had direct individual follow-up for confirmation of the diagnosis.

This study has several limitations. The study cohort is a convenience

sample across a nonhomogeneous patient population. The study sites are heavily weighted toward pediatric academic tertiary care centers for which the patient population might not be representative of the gen- eral pediatric and adolescent population. Differing institutions, Hospital systems, and referring facilities may also vary greatly on pretransfer im- aging with US or CT scans, thereby excluding some patients from subse- quent enrollment. Details on this population were not tracked, so there is a possibility that the study cohort does not fully represent the general pediatric population presenting with abdominal pain.

The study sample was collected during times of study staff availabil- ity, resulting in a cohort collected primarily during weekday and eve- ning hours with underrepresentation of patients presenting overnight or on weekends. However, the sample represents a very large cohort of pediatric patients across various practice settings and locations around the country. The study had an appendicitis rate of 25.3% consis- tent with previous studies, suggesting that the cohort is representative of the full range of low- to moderately high-risk patients with potential acute appendicitis. Those patients perceived as highest risk were ex- cluded, so no Se or Sp conclusions can be made for this specific subset. There were 14 patients diagnosed as acute appendicitis based on clinical and imaging data without subsequent pathologic confirmation. However, analysis of the data with these patients included as either pos- itive or negative for acute appendicitis or fully excluded did not materi-

ally affect the primary results.

A small number of patients were lost to follow-up. The final confir- mation or exclusion of appendicitis for these patients is uncertain, and data from these patients are not included in the analysis. If these pa- tients represented a disproportionate number of missed cases of appen- dicitis, this could affect the study results. There also exists the possibility that some patients might have appendicitis with an extremely long Delay in diagnosis beyond the follow-up period, but this would be viewed as unusual, infrequent, and outside of the anticipated screening for which this test would be used.

The WBC values were obtained from each individual institution’s clinical laboratory and were not centralized. However, all laboratories represented were under regulatory supervision and Clinical Laboratory Improvement Amendments certified.

The imaging rates for CT and US in this study may differ from actual

imaging rates for all patients presenting to the ED with a concern for ap- pendicitis. First, CT and US rates vary by institution. Therefore, the rates in this study are representative of the institutions in the study. Second, those at highest risk, defined as those not needing imaging for the diag- nosis, were excluded from participation potentially increasing our im- aging rates. Lastly, those with imaging before enrollment were also excluded, potentially decreasing our imaging rates. However, the prev- alence of appendicitis in this cohort is consistent with previous reports so the estimates of imaging should approximate actual imaging ranges. The PAS and Alvarado score results are limited in that the individual components of these scores were assessed by a single clinical provider or investigator, then collected and collated by the Research staff without measurement of agreement. As such, any potential interobserver vari-

ability cannot be determined.

The WBC-CRP comparison is limited as a post hoc analysis. In addi- tion, this analysis used the raw CRP concentrations obtained from the APPY1 System, not the clinically used hospital commercial CRP assays, and it is uncertain how these values correlate.

In conclusion, this composite biomarker panel exhibited high Se and NPV in this prospective cohort of pediatric and adolescent patients pre- senting to the ED with abdominal pain suggesting possible acute appen- dicitis. If these results can be confirmed, this biomarker panel may help clinicians identify patients who are unlikely to have acute appendicitis, presenting the option of Clinical monitoring and follow-up in lieu of fur- ther diagnostic testing, and potentially decreasing the dependence on radiologic imaging. Further study is needed to assess the impact of this biomarker panel on imaging utilization in clinical practice.

References