a b s t r a c t

Objective: The CHOKAI and STONE scores are clinical prediction rules to predict Ureteral stones in patients pre- senting with renal colic. Both systems contribute to reducing diagnostic radiation exposure; however, few studies have compared the two scoring systems. Therefore, we aimed to compare these systems and assess their diag- nostic accuracy for Ureteral stones.

Methods: This was a Multicenter prospective observational study performed between 2017 and 2018, including patients aged N15 years with renal colic and suspected with ureteral stones. We calculated the CHOKAI and STONE scores of each patient based on their medical interviews and physical and laboratory findings. Primary outcome was differences in the area under the receiver operating characteristic curve in each model, and second- ary outcome was diagnostic accuracy at the optimal cut-off point.

Results: Of the 124 patients included, 84 were diagnosed with ureteral stones. The area under the curve of the CHOKAI score was 0.95, showing a sensitivity of 0.93, specificity of 0.90, positive likelihood ratio of 9.3, and neg- ative likelihood ratio of 0.079, at an optimal cut-off point of 6. The area under the curve of the STONE score was 0.88, showing a sensitivity of 0.68, specificity of 0.90, positive likelihood ratio of 6.8, and negative likelihood ratio of 0.36, at an optimal cut-off point of 9. Thus, the area under the curve was significantly higher for the CHOKAI score than for the STONE score (p = 0.0028).

Conclusions: The CHOKAI score has a diagnostic performance superior to that of the STONE score in this population.

(C) 2019

1. Introduction

Acute onset of renal colic from ureterolithiasis is mainly character- ized by back, flank, or lower abdominal pain, which are non-specific symptoms. Non-contrast helical computerized tomography (NCCT) has become the standard Diagnostic modality for ureteral stones be- cause of its high sensitivity and specificity [1,2]. NCCT is a very useful modality for the detection of ureteral stones; however, radiation

* Corresponding author.

E-mail addresses: [email protected] (H. Fukuhara), [email protected] (A. Sugiura), [email protected] (M. Nakane).

exposure from repetitive CT imaging is a concern because ureterolithiasis is a recurrent disease, which relapses in 15% and 30-50% of patients within 1 and 10 years, respectively [3-7].

To reduce CT radiation exposure, Moore et al. developed the STONE score, which helps in predicting the presence of ureteral stones [8]. It consists of the following five categories for a total of 13 possible points: sex, timing, origin, nausea/vomit, and occult blood in urine. This score classifies patients into three risk categories: low-, intermediate-, and high likelihood of ureteral stones. In addition, they developed a low- dose CT protocol using the STONE score for patients with intermediate- and high likelihood of developing ureteral stones to re- duce radiation exposure [9]. The STONE score is a good screening tool for ureteral stones; however, several limitations have been indicated.

https://doi.org/10.1016/j.ajem.2019.07.018

0735-6757/(C) 2019

One of these is the “race” factor. Wang et al. pointed out the possibility that race (black vs non-black) was not a reliable predictor of stone vs no stone [10]. Furthermore, race also does not work well in Japan, where most inhabitants are mongoloid, as it is ethnicity-specific. Another lim- itation is that the STONE score does not include the presence or absence of hydronephrosis on Ultrasonography imaging, which is consid- ered as the first-line modality for the screening of ureteral stones [11- 13]. To resolve these two limitations, another clinical prediction rule for ureteral stones, named “CHOKAI score,” was developed [14]. It con- sists of the following seven categories: sex, timing, age, nausea/vomit, occult blood in the urine, history of Kidney stones, and hydronephrosis, yielding a score of 0-13 (Table 1). In a single-center study, the CHOKAI score was found to be more effective for detecting ureteral stones than the STONE score; however, this study had participant selection bias be- cause approximately 80% of the eligible patients were excluded; there- fore, the usefulness of the CHOKAI score was not established [14]. Therefore, additional studies are needed to verify the usefulness of the CHOKAI score.

We hypothesized that the CHOKAI score would perform better in predicting ureterolithiasis in the Japanese population. The aim of our study was 1) to compare the diagnostic performance of the STONE and CHOKAI scores and 2) to re-evaluate the diagnostic performance of the CHOKAI score.

1. Methods
   1. Study design and patient selection

This was a multicenter prospective observational validation study. From November 2017 to September 2018, six hospitals enrolled pa- tients aged >=15 years who presented with back, flank, or lower abdom- inal pain and were admitted to the emergency or urology departments. Exclusion criteria were as follows: 1) declining to participate in the study, 2) Abnormal vital signs (body temperature N 37.5 ?C or systolic blood pressure ?90 mmHg), 3) insufficient medical interview data (du- ration of pain, history of kidney or upper ureteral stones, and gastroin- testinal symptoms such as nausea or vomiting), 4) insufficient examination (lack of US, urinalysis, and definitive radiological diagno- sis), and 5) active malignancy. This study was approved by the compe- tent institutional review boards (approval no. 212). All participants provided written informed consent.

Table 1

CHOKAI and STONE score categories.

Factors Indication Points

CHOKAI score STONE score

Nausea or vomiting None 0 0

Nausea 1 1

Vomiting 1 2

Hydronephrosis No 0 -

Yes 4 -

Occult blood in the urine Negative 0 0

Positive 3 3

History of kidney stone No 0 -

Yes 1 -

Sex Female 0 0

Male 1 2

Age >=60 years 0 -

	b60 years	1	-
Timing of pain reduction	N24 h	0	0
	6-24 h	0	1
	b6 h	2	3
Ethnicity	Black	-	0
Total points	Non-black	- 0-13	3 0-13

Methods of measurement

Before starting this study, each researcher was instructed about the study protocol, used forms, and data collection techniques. Physicians or nurses selected the candidates based on their cardinal symptoms or interview sheet. They then distributed a questionnaire inquiring about the duration of pain, nausea or vomiting, prior history of kidney or upper ureteral stones, and informed consent of patients or attending guardians. After a medical examination conducted by emergency physi- cians, urologists, or medical interns, they conducted a Urine dipstick or standard urinalysis for the detection of hematuria and point-of-care ul- trasonography (POCUS) examination for the detection of hydronephrosis using a curviLinear probe of one of the following ma- chines: Hitachi prosound ?6, ?7 (Hitachi Aloka Medical Ltd., Mitaka, Tokyo), Vivid S5 (GE Healthcare Japan, Hino, Tokyo), Vivid i (GE Healthcare Japan), CX50 (Philips Japan, Ltd., Minato-Minami, Tokyo), or Fazone M (Fujifilm Medical Co., Ltd., Nishi-Azabu, Tokyo). POCUS was conducted by well-trained urologists, emergency physicians with or without Fellowship training in ultrasound, ultrasonographically expe- rienced clinicians, or ultrasonographically inexperienced medical in- terns. When ultrasonographically inexperienced medical interns conducted POCUS as the primary examiners, an emergency physician with or without fellowship training always checked the findings. The presence or absence of hydronephrosis was judged by the primary ex- aminer. In case the primary examiner was unable to decide the presence or absence of hydronephrosis, a secondary senior examiner such as a well-trained urologist or an emergent physician judged the same.

Radiological examinations such as NCCT or kidney-ureter-bladder (KUB) radiography were ordered at the discretion of the provider at the time. If these radiological modalities were not ordered, US alone could be used. CT was an indispensable diagnostic tool in the final diag- nosis in this study; however, it could be omitted in the following ure- teral stone cases: when well-trained urologists analyzed the KUB findings or when well-trained emergency physicians or urologists per- formed US and detected ureteral stones located at the pelviureteric junction. Based on CT (KUB radiography or US) or physical and labora- tory findings, medical specialists provided consultation or the treating physician determined the final diagnosis. When the emergency physi- cian judged that consultation was not required, they made the final di- agnosis. After making a definitive diagnosis, emergency physicians, urologists, or emergency medicine residents calculated the CHOKAI and STONE scores and then entered the obtained definitive diagnosis and CHOKAI and STONE scores in a specific form.

Outcome measures

The primary outcome was differences in the area under the curve (AUC) between the CHOKAI and STONE scores. AUC was calculated using the receiver operating characteristic (ROC) analysis. The second- ary outcome was to determine the optimal cut-off point using the ROC analysis, where the maximum sensitivity and specificity are obtained. In addition, sensitivity, specificity, positive likelihood ratio (LR+), and negative likelihood ratio at the optimal-cut off point were calcu- lated. A value of P b 0.05 was considered statistically significant. Data analysis was performed using R version 3.3.1 (R Foundation for Statisti- cal Computing, Vienna, Austria).

Sample size

The AUC value of the CHOKAI scoring system was set to 0.93, based on findings of a previous study [14], and set the ratio of the presence to absence of ureteral stone to 79:17; these provided the estimated sample size of 99, which is required to estimate the AUC value with 95% confi- dence interval within the +-0.05 range expected in the verification study, using the Hanley and McNeil’s method [15].

1. Results

Table 2

Demographic characteristics of patients and outcomes.

Characteristics of patients

Of the 154 patients eligible to participate, 30 were excluded; thus,

Stone group (n

= 84)

No stone group (n

= 40)

All (n = 124)

the remaining 124 patients were included in the final analysis (Fig. 1). Table 2 shows patient characteristics and outcomes. All patients [men: 72/124 (58%) patients] were Japanese, with a mean age of 51 years (range: 17-87 years). Of the 124 included patients, 84 were di- agnosed with ureteral stones, of whom 82 were diagnosed using CT, 1 was diagnosed using KUB radiography, and 1 was diagnosed using US. POCUS was conducted by the following physicians: well-trained urologists in 56 (45%) patients, emergency physicians with fellowship

training in 20 (16%), emergency physicians without fellowship training	6-24	10 (12)	2 (5.0)	12 (10)
in 18 (15%), ultrasonographically experienced clinicians in 29 (23%), and ultrasonographically inexperienced medical interns in 1 (1.0%).	N24 Nausea or vomiting (%)	27 (32)	27 (68)	54 (43)

Number of patients (%) 84 (68) 40 (32) 124 (100)

Median age (range) 51 (20-77) 50 (17-87) 51

(17-87)

Sex (%)

Female	33 (39)	19 (48)	52 (42)
Male	51 (61)	21 (52)	72 (58)
Ethnicity (%) Japanese	84 (100)	40 (100)	124 (100)
pain duration, h (%) b6	47 (56)	11 (27)	58 (47)

Main results

None	25 (30)	27 (68)	52 (42)
Nausea	38 (45)	7 (17)	45 (36)
Vomiting	21 (25)	6 (15)	27 (22)
History of kidney stone (%)

CI, 0.85-0.97), specificity was 0.90 (95% CI, 0.76-0.97), LR+ was 9.3

The diagnostic performance of the CHOKAI and STONE scores was	No	48 (57)	36 (85)	84 (68)
determined using the ROC analysis. AUC values of the CHOKAI and	Yes	36 (43)	4 (15)	40 (32)
STONE scores were 0.95 (95% CI, 0.91-0.99) and 0.88 (95% CI, Occult blood in the urine (%)
0.82-0.94), respectively, which were significantly different (p =	No	15 (18)	33 (83)	48 (39)
0.0028, Fig. 2). The ROC analysis also revealed the optimal cut-off	Yes	69 (82)	7 (17)	76 (61)
point of 6 in the CHOKAI score and 9 in the STONE score.	Hydronephrosis (%)
Of the 40 patients who were not diagnosed with ureteral stones, 36 were classified correctly (classified as less than the cut-off point) in both	No Yes Diagnostic modality (%)	22 (26) 62 (74)	37 (93) 3 (7.0)	59 (48) 65 (52)
scoring systems. As for patients diagnosed with ureteral stones, 78 were	CT	82 (98)	40 (100)	122 (98)
classified correctly in the CHOKAI score and 57 in the STONE score.	KUB radiography	1 (1.0)	0 (0.0)	1 (1.0)
Characteristics of the CHOKAI and STONE scores at the optimal cut-off	US	1 (1.0)	0 (0.0)	1 (1.0)
point were as follows: for the CHOKAI score, sensitivity was 0.93 (95%	Disposition (%)

(95% CI, 3.7-23), and LR- was 0.079 (95% CI, 0.036-0.17) at the optimal cut-off point of 6 and for the STONE score, sensitivity was 0.68 (95% CI, 0.57-0.78), specificity was 0.90 (95% CI, 0.76-0.97), LR+ was 6.8 (95%

CI, 2.6-17), and LR- was 0.36 (95% CI, 0.26-0.50) at the optimal cut-

off point of 9 (Table 3). Specificity was similar between the CHOKAI and STONE scores; however, sensitivity, LR+, and LR- differed. Of the 40 patients who were not diagnosed with ureteral stones, 4 had a

Admit	4 (5.0)	11 (27)	15 (12)
Discharge	80 (95)	29 (73)	109 (88)

Abbreviations: CT, computed tomography; KUB, kidney-ureter-bladder; US, ultrasonography.

CHOKAI score of >=6 and were diagnosed with Ovarian torsion (n = 1), suspected gallbladder cancer (n = 1), and unknown diagnosis (n = 2), and 4 had a STONE score of >=9 and were diagnosed with ileus (n

Flow chart

Patients consenting to participate

n = 151 (98%)

Patients excluded for not providing

consent (n = 3, 2.0%)

Patients >=15 years with back, flank, or lower abdominal pain

between November 1, 2017, and September 30, 2018 (n = 154)

n = 2 (29%)

n = 5 (71%)

Patients excluded due to (n=7)

Active malignancy

Abnormal vital signs

Patients included

n=124 (81%)

Diagnosed with ureteral stones n = 84 (68%) Not diagnosed with ureteral stones n = 40 (32%)

Patients excluded from analysis (n=20)

Lack of US examination n = 5 (25%)

Lack of radiography n = 6 (30%)

Lack of urinalysis n = 9 (55%)

Fig. 1. Flow chart of patient enrollment.

Table 4

p = 0.0028

CHOKAI vs. STONE score

1.0

Differential diagnosis of patients not diagnosed with stones.

Sensitivity

0.6

0.8

Important alternative findings	Case
Unknown	15
Diverticulitis	5
Pleurisy	2
Acute lumbar pain	2
Urinary retention	2
Acute pancreatitis	1
Acute cholelithiasis	1
Acute scapulodynia	1
Acute appendicitis	1
Anisakiasis	1
Constipation	1
Suspected gallbladder cancer	1
Ileus	1
Kidney cyst bleeding	1
Lumbar compression fracture	1
Ovarian torsion	1
Pyelonephritis	1
Renal hemorrhage	1
Slipped disc	1
Total	40

0.0 0.2

0.0 0.2

0.4

0.4

0.6 0.8 1.0

1 - Specificity

CHOKAI score AUC, 0.95 (95% CI, 0.91-0.99)

STONE score AUC, 0.88 (95% CI, 0.82-0.94)

Fig. 2. Comparison of AUC under the ROC curve between the CHOKAI and STONE scores at the optimal cut-off point. Abbreviations: AUC, area under the curve; CI, confidence interval; ROC, receiver operating characteristic.

= 1), kidney cyst bleeding (n = 1), acute lumbar pain (n = 1), and un- known diagnosis (n = 1) (Table 4).

1. Discussion

We found that the CHOKAI score had superior diagnostic perfor- mance than the STONE score in this population. In addition, our study showed that the optimal cut-off point was 6 in the CHOKAI score and 9 in the STONE score.

Evaluation of hydronephrosis was found to be a key factor in the CHOKAI score because of the 13 points, hydronephrosis weighted 4 points. Various studies have emphasized the importance of hydronephrosis in the diagnosis of ureteral stones [11-13,16]. Daniels et al. reported using STONE PLUS, which combined the STONE score and US findings, that US findings improved the detection of ureteral stones in low- and intermediate-risk patients [16].

However, US findings may lead to a misdiagnosis because US is a user-dependent modality, and its diagnostic accuracy of hydronephrosis changes according to the Experience level [11]. In fact, there were 2 cases in which urologists initially changed the diagnosis from the absence of hydronephrosis (initial US was performed by a medical intern and an emergency physician without fellowship train- ing) to the presence of hydronephrosis in the present study. It is possi- ble to change the judgement about hydronephrosis as described above; however, we were able to improve the diagnostic accuracy of hydronephrosis via education and training, which led to the enhanced accuracy of the CHOKAI score [11].

Table 3

Diagnostic accuracy of the CHOKAI and STONE scores at the optimal cut-off score.

Score (optimal cut-off score) CHOKAI (6) STONE (9) Point estimation (95% CI)

Sensitivity 0.93 (0.85-0.97) 0.68 (0.57-0.78)

Specificity 0.90 (0.76-0.97) 0.90 (0.76-0.97)

LR+ 9.3 (3.7-23) 6.8 (2.6-17)

LR- 0.079 (0.036-0.17) 0.36 (0.26-0.50)

Abbreviations: LR+, positive likelihood ratio; LR-, negative likelihood ratio; CI, confi- dence interval.

US is a useful tool for the prediction of ureteral stones; however, Sternberg et al. found that hydronephrosis as an isolated finding on US did not accurately predict the presence or absence of ureteral stones, and they emphasized the need for radiological Imaging techniques such as CT or KUB radiography because stone size or location constitutes an important information for ureterolithiasis management [17]. However, cumulative radiation exposure from repeated CT and the cost burden or length of stay at the emergency department during the diagnosis of ureteral stones must be considered [2,5,6,18,19]. Sternberg et al. also re- ported that a scoring system such as the STONE score could help guide the choice of imaging [17]. To reduce radiation exposure associated with the diagnosis of ureteral stones, the European Association of Urol- ogy (EAU) and American Urological Association (AUA) recommended using low-dose CT, which had almost the same sensitivity and specific- ity as NCCT in the detection of ureteral stones [20-23]. Low-dose CT is a useful modality in the diagnosis of ureteral stones; however, there is a lack of evidence regarding the criteria for use of low-dose CT. Moore et al. provided useful criteria for the use of low-dose CT with the STONE score; [9] however, the use of low-dose CT for the evaluation of kidney stones remains low, reaching only 7.6% in 2015-2016 [24]. Ideal criteria for diagnostic tests had LR+ of N10 or/and LR- of b0.1 be- cause they generated large and often conclusive changes from pretest to posttest probability [25,26]. In our study, the CHOKAI score nearly reached this condition, with an LR+ of 9.3 (95% CI, 3.7-24) and LR- of 0.079 (95% CI, 0.036-0.17) at the optimal cut-off point of 6. Therefore, the CHOKAI score could be a useful screening tool for the diagnosis of ureteral stones and a useful criterion for the use of low-dose CT. Consid- ering this result, for patients complaining of renal colic and having a value of >=6 points in the CHOKAI score, low-dose CT is likely to be a more appropriate diagnostic tool than NCCT, although further study is needed to confirm this. Furthermore, the CHOKAI score might allow cli- nicians to defer CT scan for patients with 10-13 points because in this study, all patients not diagnosed with ureteral stones had CHOKAI scores of 0-9 points.

There are several limitations to this study. First, this study included only Japanese patients. Thus, the properties of the CHOKAI scores may be different in the other populations. Second, this study had a small sim- ple size. Of the 40 patients who did not have ureteral stones, only few had a life-threatening condition such as diverticulitis or acute pancrea- titis. Therefore, the screening power of the CHOKAI score for another important conditions, such as Abdominal aortic aneurysm and pneumo- nia, is unclear. Third, this study was conducted from Monday to Friday during the working hours of Research staff and does not necessarily re- flect the whole population of patients in the emergency or urology

departments. Fourth, this study did not compare the diagnostic accu- racy between the CHOKAI score and STONE PLUS. STONE PLUS classified hydronephrosis into none, mild, and moderate-severe without weighting points. As the hydronephrosis measurement differed, a com- parison between the CHOKAI score and STONE PLUS was difficult.

1. Conclusion

The CHOKAI score was a good Prediction tool for ureteral stones and proved more useful than the STONE score in this population. US training for the detection of hydronephrosis could enhance the accuracy of the CHOKAI score.

Source of support

No funding was provided for this study.

Declaration of Competing Interest

No competing financial interests exist.

Acknowledgments

We wish to thank Osamu Ichiyanagi for critical reading of the man- uscript, Takashi Toyohara, Shinta Suenaga, Yoshinori Yashiro, Shinichi Midorikawa, Akihiko Abe, Masaaki Tsukigi, Jin Suzuki, Tomohiro Shibasaki, Shigemitsu Horie, Tomoyuki Kato, Takuya Yamanobe, Hisashi Kawazoe, Sei Naito, Hayato Nishida, Sakurai Toshihiko, Hidenori Kanno, Yuta Kurota, Mayu Yagi, Takafumi Narisawa, Michinobu Ozawa, Masaki Ushijima, Yuki Takai, Yuya Kuboki and Suguru Fujita for help in collecting and organizing data, Akiko Abe for her work as nurse abstrac- tor, StaGen Co., Ltd. for help in statistical advice on this study, and Editage (www.editage.jp) for English language editing.

References

1. Fwu CW, Eggers PW, Kimmel PL, Kusek JW, Kirkali Z. Emergency department visits, use of imaging, and drugs for urolithiasis have increased in the United States. Kidney Int 2013;83:479-86. https://doi.org/10.1038/ki.2012.419.
2. Ghani KR, Roghmann F, Sammon JD, et al. Emergency department visits in the United States for upper urinary tract stones: trends in hospitalization and charges. J Urol 2014;191:90-6. https://doi.org/10.1016/j.juro.2013.07.098.
3. Berrington de Gonzalez A, Darby S. Risk of cancer from diagnostic X-rays: estimates for the UK and 14 other countries. Lancet 2004;363:345-51. https://doi.org/10. 1016/S0140-6736(04)15433-0.
4. Chen TT, Wang C, Ferrandino MN, Scales CD, Yoshizumi TT, Preminger GM, et al. Ra- diation exposure during the evaluation and management of nephrolithiasis. J Urol 2015;194:878-85.
5. Elkoushy MA, Andonian S. Lifetime radiation exposure in patients with recurrent nephrolithiasis. Curr Urol Rep 2017;18:85. https://doi.org/10.1007/s11934-017- 0731-6.
6. Ferrandino MN, Bagrodia A, Pierre SA, Scales Jr CD, Rampersaud E, Pearle MS, et al. Radiation exposure in the acute and short-term management of urolithiasis at 2 ac- ademic centers. J Urol 2009;181:668-72. https://doi.org/10.1016/j.juro.2008.10.012.
7. Gottlieb M, Long B, Koyfman A. The evaluation and management of urolithiasis in the ED: a review of the literature. Am J Emerg Med 2018;36:699-706. https://doi. org/10.1016/j.ajem.2018.01.003.
8. Moore CL, Bomann S, Daniels B, Luty S, Molinaro A, Singh D, et al. Derivation and val- idation of a clinical prediction rule for uncomplicated ureteral stone-the STONE score: retrospective and prospective Observational cohort studies. BMJ 2014;348: g2191https://www.bmj.com/content/348/bmj.g2191.
9. Moore CL, Daniels B, Singh D, Luty S, Gunabushanam G, Ghita M, et al. Ureteral stones: implementation of a reduced-dose CT protocol in patients in the emergency department with moderate to high likelihood of calculi on the basis of STONE score. Radiology 2016;280:743-51. https://doi.org/10.1148/radiol.2016151691.
10. Wang RC, Rodriguez RM, Moghadassi M, et al. External validation of the STONE score, a clinical prediction rule for ureteral stone: an observational multi- institutional study. Ann Emerg Med 2016;67:423-32. https://doi.org/10.1016/j. annemergmed.2015.08.019.
11. Herbst MK, Rosenberg G, Daniels B, Gross CP, Singh D, Molinaro AM, et al. Effect of Provider experience on clinician-performed ultrasonography for hydronephrosis in patients with suspected renal colic. Ann Emerg Med 2014;64:269-76. https://doi. org/10.1016/j.annemergmed.2014.01.012.
12. Kartal M, Eray O, Erdogru T, Yilmaz S. Prospective validation of a current algorithm including bedside US performed by emergency physicians for patients with acute flank pain suspected for renal colic. Emerg Med J 2006;23:341-4. https://doi.org/ 10.1136/emj.2005.028589.
13. Smith-Bindman R, Aubin C, Bailitz J. Ultrasonography versus computed tomography for suspected nephrolithiasis. N Engl J Med 2014;371:1100-10https://doi:10.1056/ NEJMoa1404446.
14. Fukuhara H, Ichiyanagi O, Midorikawa S, Kakizaki H, Kaneko H, Tsuchiya N. internal validation of a scoring system to evaluate the probability of ureteral stones: the CHOKAI score. Am J Emerg Med 2017;35:1859-66. https://doi.org/10.1016/j.ajem. 2017.06.023.
15. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:29-36. https://doi.org/10.1148/ra- diology.143.1.7063747.
16. Daniels B, Gross CP, Molinaro A, Singh D, Luty S, Jessey R, et al. STONE PLUS: evalu- ation of emergency department patients with suspected renal colic, using a clinical prediction tool combined with point-of-care limited ultrasonography. Ann Emerg Med 2016;67:439-48. https://doi.org/10.1016/j.annemergmed.2015.10.020.
17. Sternberg KM, Pais Jr VM, Larson T, Han J, Hernandez N, Eisner B. Is hydronephrosis on ultrasound predictive of ureterolithiasis in patients with renal colic? J Urol 2016; 196:1149-52. https://doi.org/10.1016/j.juro.2016.04.076.
18. Sodickson A, Baeyens PF, Andriole KP, Prevedello LM, Nawfel RD, Hanson R, et al. Re- current CT, cumulative radiation exposure, and associated radiation-induced cancer risks from CT of adults. Radiology 2009;251:175-84. https://doi.org/10.1148/radiol. 2511081296.
19. Brenner DJ, Hall EJ. Computed tomography-an increasing source of radiation expo- sure. N Engl J Med 2007;357:2277-84.
20. Fulgham PF, Assimos DG, Pearle MS, Preminger GM. Clinical effectiveness protocols for imaging in the management of ureteral calculous disease: AUA technology as- sessment. J Urol 2013;189:1203-13. https://doi.org/10.1016/j.juro.2012.10.031.
21. Moore CL, Daniels B, Ghita M, Gunabushanam G, Luty S, Molinaro AM, et al. Accuracy of reduced-dose computed tomography for ureteral stones in emergency depart- ment patients. Ann Emerg Med 2015;65:189-98. https://doi.org/10.1016/j. annemergmed.2014.09.008.
22. Turk C, Petrik A, Sarica K, Seitz C, Skolarikos A, Straub M, et al. EAU guidelines on di- agnosis and conservative management of urolithiasis. Eur Urol 2016;69:468-74. https://doi.org/10.1016/j.eururo.2015.07.040.
23. Zilberman DE, Tsivian M, Lipkin ME, Ferrandino MN, Frush DP, Paulson EK, et al. Low dose computerized tomography for detection of urolithiasis-its effectiveness in the setting of the urology clinic. J Urol 2011;185:910-4. https://doi.org/10.1016/j.juro. 2010.10.052.
24. Weisenthal K, Karthik P, Shaw M, et al. Evaluation of kidney stones with reduced- radiation dose CT: progress from 2011-2012 to 2015-2016-not there yet. Radiology 2018;286:581-9. https://doi.org/10.1148/radiol.2017170285.
25. Jaeschke R, Guyatt GH, Sackett DL. Users’ guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? The Evidence-Based Medicine Working Group. JAMA 1994; 271:703-7. https://doi.org/10.1001/jama.1994.03510330081039.
26. McGee S. Simplifying likelihood ratios. J Gen Intern Med 2002;17:646-9. https://doi. org/10.1046/j.1525-1497.2002.10750.x.