Original Contribution

Diagnostic performance of a pocket-sized Ultrasound device for quick-look cardiac imaging^?,??,?

Bruce J. Kimura MD ^a,?, Glynn W. Gilcrease III MD ^a, Brian K. Showalter RDCS ^a,

James N. Phan RDCS ^a, Tanya Wolfson ^b

^aDepartment of Cardiology, Scripps Mercy Hospital, 4060 Fourth Ave #206, San Diego, CA 92103, USA

^bComputational and Applied Statistics Laboratory, University of California, San Diego, CA, USA

Received 8 July 2010; revised 12 July 2010; accepted 12 July 2010

Abstract

Background: Although pocket-sized, simplified ultrasound devices have emerged to enable subjective point-of-care assessment, few data on their cardiac application exist. We sought to examine the Image quality and the accuracy of subjective diagnosis of video loops obtained from a pocket-sized ultrasound device for 2 significant Cardiac abnormalities, Left ventricular systolic dysfunction and left Atrial enlargement, obtained from a single, quick-look view.

Methods: Parasternal left ventricular long-axis images acquired with a miniaturized commercially available device (Acuson P10) were reviewed using subjective criteria for left ventricular systolic dysfunction and left atrial enlargement and were compared with M-mode measurements of left atrial systolic diameter and E-point septal separation from a fully featured echocardiograph in 78 inpatients referred for standard echocardiography. Interpretive confidence and image quality were evaluated with each interpretation.

Results: Of 78 inpatient studies, 19% of pocket ultrasound and 13% of standard studies were technically limited (P = NS). Of 61 technically adequate studies, subjective interpretation of pocket ultrasound images had a sensitivity, specificity, and accuracy of 79%, 52%, and 64% for left atrial diameter more than 4 cm; 47%, 98%, and 82% for E-point septal separation more than 1 cm of; 83%, 62%, and 74% for either abnormality; and 92%, 82%, and 87% for either abnormality when interpretive confidence was present (n = 23). The pocket ultrasound image quality scores were significantly lower than the standard echocardiograph (P b .001).

Conclusion: The pocket-sized device provided adequate imaging for screening of 2 significant cardiac entities. Subjective interpretation of a single parasternal view may help identify patients with cardiac disease.

(C) 2012

Abbreviations: PLAX, parasternal long-axis; EPSS, E-point-to- septum separation.

^? Presented at the CHEST meetings, Philadelphia, 2008.

^?? No author has a conflict to disclose.

^? In the spirit of full disclosure, Dr Bruce Kimura’s spouse is employed

by Merck & Co., Inc, a pharmaceutical company.

* Corresponding author. University of California, San Diego. Tel.: +1 (619) 297 0014; fax: +1 (619) 297 1014.

Introduction

Technologic advancements in the last decade have resulted in miniaturized hand-carried ultrasound devices, some of which are now palm or pocket-sized. With the intent of providing rapid “point-of-care” diagnostics through unfettered portability [1], their use is predicated upon

0735-6757/$ – see front matter (C) 2012 doi:10.1016/j.ajem.2010.07.024

subjective recognition of easily-seen disease targets that do not require precise quantitation. The early recognition of cardiac abnormalities such as left ventricular dysfunction or left atrial enlargement in the assessment of unexplained hypotension or dyspnea could potentially be life-saving, result in immediate referral for echocardiography, and also have long-term prognostic implications [2-4]. In theory, left ventricular systolic dysfunction and left atrial enlargement are ideal targets for initial detection with pocket-sized ultrasound devices, in that these entities can signify the presence of a variety of cardiac disorders and are often obvious with “quick-look” viewing of a single parasternal long-axis video loop.

Although pocket-sized devices are now commercially available, their bedside use and accuracy remain undefined. It is unknown whether their reliance on operator subjectivity coupled with reduced screen size and func- tionality will limit diagnostic accuracy to only certain ultrasound targets and clinical applications. Therefore, we sought to investigate the relative accuracy of subjective interpretation of images obtained from a pocket-sized ultrasound device for the diagnosis of left ventricular systolic dysfunction and left atrial enlargement as com- pared with conventional measurements of these entities from standard echocardiography.

Methods

Study design and patient population

Data was obtained from inpatients referred for standard Echocardiographic examination to the echo laboratory of a 300-bed Tertiary medical center. A full-time registered cardiac sonographer (B.K.S.) was trained to perform and record a 2-second video loop of the parasternal left ventricular long-axis (PLAX) image on all inpatients for whom he had been assigned to perform a standard echocardiogram, using first a pocket-sized ultrasound device (Acuson P10; Siemens Healthcare Ultrasound, Mountain View, CA) and then using a conventional, fully featured echocardiograph (Philips iE33; Philips Healthcare, Andover, MA). Both instruments were equipped with low-frequency 3-MHz phased-array cardiac transducer probes. Because the study’s diagnostic criteria require a native mitral valve, 3 patients with mitral prostheses were excluded. The study was approved by the Scripps Institutional Review Board for Scripps Mercy Hospital, San Diego, CA.

Subjective interpretation of images from the pocket ultrasound device

Images of the standard echocardiographic PLAX view obtained from the pocket ultrasound device were displayed on a laptop echo viewing station using device-specific software (Acuson P10 viewer; Siemens Healthcare Ultra-

sound). An echocardiographer-cardiologist (B.J.K.) subjec- tively interpreted the images for left atrial enlargement, defined as being present when the left atrial anterior- posterior diameter appeared larger than the overlying Ascending aorta throughout the cardiac cycle, and left ventricular systolic dysfunction, defined as being present when the anterior leaflet of the mitral valve did not encroach upon the left ventricular outflow tract in diastole. In addition, the technical quality of the PLAX video loop was judged on a quantitative scale as follows: 0, no image; 1, only cardiac motion detected; 2, chambers and valves grossly resolved; 3, endocardium and wall thickness seen, but incomplete; and 4, greater than 90% of endocardium and valve motion seen (Figs. 1 and 2). A study was considered “technically-limited” when its quality score was less than 2. Interpretive confidence, defined as “feeling 95% certain” for the accuracy of the interpretation of left atrial size or left ventricular function, was asked of the echocardiographer as being present or absent at the time of each interpretation.

Standard echocardiographic measurements

Video-looped images of PLAX obtained from the standard echocardiogram on each patient were reviewed by the same echocardiographer on a desktop echo viewing station using commercially available echo reader software (QLAB; Philips Healthcare). This analysis was undertaken after all pocket ultrasound image interpretations had been completed and in random order, so as to avoid bias in study quality assessment. Quantitative M-mode measurements taken by the sonographer at the time of the standard study

Fig. 1 Example of a good quality examination (quality score, 4), with diagnostic confidence present, of low ejection fraction and left atrial enlargement. Note that the anterior leaflet of the mitral valve does not encroach upon the left ventricular outflow tract (LVOT), that is, a large (N1cm) EPSS is present (double-headed arrow) and that the left atrial diameter (dashed line) appears larger than the aortic diameter (dotted line), even at its minimum, in diastole.

Fig. 2 Example of a lesser quality examination (quality score, 3), with presence of diagnostic confidence for a normal left ventricular ejection fraction and left atrium (LA) size, that addresses the robust nature of the diagnostic criteria. E-point-to-septum separation is small (solid line) as the anterior MItral leaflet encroaches on the left ventricular outflow tract, and the LA diameter (dashed line) is smaller than the aortic diameter (dotted line) in diastole.

were reviewed to determine the presence of left atrial enlargement or left ventricular systolic dysfunction as follows. Left atrial enlargement was present when left atrial diameter, at its maximum in end-systole, was greater than

4.0 cm [5], and left ventricular systolic dysfunction was present when the mitral valve’s anterior leaflet E-point-to- septum separation (EPSS) was greater than 1.0 cm in early diastole, which approximates a left ventricular ejection fraction of less than 55% by regression analyses [6,7]. Using the same scale as for the pocket ultrasound quality assessment, the quality of the PLAX image from the standard echocardiograph was quantified. Technically limited studies obtained from the standard echocardiograph, defined by a quality score of less than 2 were not considered in the analysis of diagnostic accuracy.

Statistical analysis

The McNemar test was used to assess the difference in the proportion of technically limited studies between the pocket-

Table 1 Diagnostic Accuracy for LV dysfunction and Left Atrial Enlargement

sized device and standard echocardiograph. The Wilcoxon paired-sample, rank-sum test was used to assess the difference in overall image quality scores between the pocket device and standard echocardiograph. Sensitivity, specificity, and accuracy and their confidence intervals were calculated for the pocket ultrasound subjective interpretation for left atrial size greater than 4.0 cm and EPSS greater than

1.0 cm, for the presence of either abnormality, and for the subgroup of interpretations with interpretive confidence. A P value of less than .05 was considered significant.

Results

Of 78 inpatient studies, 29% were portable studies performed in the intensive care unit. There were 17 technically limited studies (quality score less than 2), 19% of all pocket ultrasound and 13% of all standard studies (P = .13). As expected, the quality score (mean +- SD) of the pocket ultrasound device was significantly less, 2.4 +- 1.1, than the standard echocardiograph, 2.8 +- 1.1 (P b .001), on this difficult-to-image patient population.

Of the 61 remaining technically adequate studies, 28 (48%) had left atrial size greater than 4.0 cm, 19 (31%) had

EPSS greater than 1.0 cm, 27 (44%) lacked either abnormality, and 12 (20%) had both abnormalities present. Subjective interpretation of P10 images had sensitivity, specificity, and accuracy of 79%, 52%, and 64%, respec- tively, for left atrial size greater than 4.0 cm and 47%, 98%, and 82%, respectively, for EPSS greater than 1.0 cm of (Table 1). As the presence of either abnormality should potentially “trigger” a referral for echocardiography and further cardiac evaluation, the sensitivity, specificity, and accuracy of the pocket device for either abnormality being present was 83%, 62%, and 74%. Interpretive confidence was felt to be present in technically adequate pocket Ultrasound studies in 34 (54%) of 61 of left atrial size and in 32/61(52%) of 61 of left ventricular function interpretations and in 23 (38%) of 61 for both evaluations. The sensitivity, specificity, and accuracy for either abnormality when interpretive confidence was present were 92%, 82%, and 87%. Of these 23 technically adequate, confident interpretations for both entities, there were still 3 errors (2 false positive and 1 false negative), all of which occurred in the assessment of left atrial size on studies with lesser quality scores.

SENS [95% CI] SPEC [95% CI] ACC [95%CI]
LA N4 cm (n = 61)	79%	[59-92]	52%	[34-69]	64% [51-76]
EPSS N1 cm (n = 61)	47%	[24-71]	98%	[87-100]	82% [70-91]
Either (n = 61)	83%	[66-93]	62%	[41-80]	74% [61-84]
Either, confident (n = 23)	92%	[61-100]	82%	[48-98]	87% [66-97]
“Either” represents the accuracy for either left atrium (LA) greater than 4 cm or EPSS greater than 1 cm being present. “Either, confident” represents a subset of the “Either” analysis when only studies with interpretive confidence were considered (see “Methods“). Sensitivity (SENS), specificity (SPEC), and accuracy (ACC) with 95% confidence interval values listed as a percentage in brackets.

Discussion

This study describes the accuracy of a pocket-sized ultrasound device in an inpatient environment using expert data acquisition and subjective interpretation. In this setting, the pocket-sized device provided adequate imaging for subjective detection of 2 significant cardiac entities in a difficult patient population, but did not have the image quality of a state-of-the-art echocardiograph.

The advent of hand-carried ultrasound devices has created a new paradigm for ultrasound use. In cardiovascular applications, as opposed to formal echocardiography, “point-of-care” or hand-carried cardiovascular ultrasonogra- phy can quickly diagnose life-threatening disease such as Pericardial tamponade, guide procedures such as venous or arterial access, and screen for “silent” Disease states such as early atherosclerosis [8], left ventricular dysfunction [9], and Abdominal aortic aneurysm [10]. As a burgeoning field that requires specific demands of ultrasound equipment such as portability, rapid boot time, and simplified user interfaces, hand-carried ultrasound has created new users in emergency medicine who are quick to apply diagnostic ultrasound at the bedside without the need for detailed quantitation. The specific imaging applications performed with pocket-sized ultrasound are currently undefined and will be critical in the determination of clinical accuracy and utility of these devices. A poorly conceived, inaccurate or complex protocol may be difficult to apply at the bedside and result in diagnostic errors that may be inappropriately attributed to the device or the operator.

This study used a limited, single-view imaging protocol for detection of left atrial enlargement and left ventricular systolic dysfunction, both entities signifying the presence of a wide variety of underlying cardiac abnormalities and having Prognostic importance. fluid management, early triage, and formal cardiac evaluation could be influenced by the immediate recognition of either of these entities at the bedside. Left atrial enlargement is a global echocardiograph- ic sign of cardiac disease [11], signifying persistent elevation of left atrial pressures from left ventricular ischemia, left- sided valvular disease, or chronic atrial fibrillation, and as such had a high incidence of 48% in our current study of echocardiographic referrals. Left atrial size viewed from the parasternal long-axis window strongly relates to the presence of significant cardiac findings on comprehensive echocardi- ography [12,13]. Similarly, left ventricular systolic dysfunc- tion, present in 31% of our inpatient referrals, can also represent the end result of multiple disorders that could benefit from a more detailed cardiac evaluation and therapeutic approach. In noisy emergency department, these entities are difficult to detect on physical examination, but can be evaluated in the same, single parasternal long-axis ultrasound image. Unlike endocardial and valvular defini- tion, their subjective diagnostic criteria use echogenic and easily discernable interfaces that remain identifiable often despite technically difficult imaging. Two prior studies have

shown the capability of novice users to recognize left ventricular systolic dysfunction [14] and left atrial enlarge- ment [12] using subjective diagnostic criteria similar to the ones used in the current study.

The reduced accuracy noted in the current study likely represents both the limitations of the pocket-sized device and the difference when comparing subjective diagnostic criteria to quantitative measures. The subjective diagnostic criteria are based on comparative landmarks on the image and are related to the criterion standard M-mode measure- ments but do not represent a direct estimation of them. The difference creates a substantial gray zone of indeterminate or “borderline” interpretations. We mitigated this effect by stratifying the data by interpretive confidence, present in more than 50% of the pocket ultrasound studies, which resulted in improvement in overall accuracy to 87%. This analysis of interpretive confidence suggests that only 10% to 15% of error is potentially unexplained, some of which could be attributed to device or Operator characteristics and the rest to each specific subjective criterion. For instance, an error due to the left atrial subjective criterion error could occur in smaller patients, where the left atrium could appear larger than the aorta, but may not measure greater than 4 cm. A prior study from our laboratory has validated the subjective criterion when compared with left atrial volumes indexed to body size [13]. Similarly, our data suggest that the subjective criterion for left ventricular systolic dysfunction may be more appropriate for detecting lower ejection fractions. However, in this 2-target imaging protocol, the high specificity for left ventricular dysfunc- tion is complemented by the increased sensitivity of the diagnostic criterion for left atrial enlargement. To further simulate clinical practice, we assessed the diagnostic accuracy of the detection of either left ventricular systolic dysfunction or left atrial enlargement, since detection of either abnormality with the pocket device would prompt referral for a comprehensive echocardiogram that would delineate all abnormalities. The inclusion of interpretive confidence and the presence of either abnormality in our analysis may best estimate the actual accuracy and optimal use of these pocket-sized devices in practice.

Our study has several limitations. First, only expert use of the device was performed, both in data acquisition by a registered sonographer and in case interpretation by an echocardiographer. However, countering this effect was the high prevalence of studies that were performed in the intensive care unit on the most technically difficult patients to image. We specifically designed subjective diagnostic criteria for comparison with conventional parasternal M- mode measurements, and therefore, the disease prevalences reported reflect abnormalities from this view only. Finally, few studies have investigated relationships between interpretative confidence, image quality, and subjective accuracy with User experience, which would have implications regarding the learning and early adoption of hand-carried ultrasound.

The miniaturization of an ultrasound device to pocket or palm size truly heralds the ultrasonic stethoscope [1,15,16]. The diagnostic application of these devices must be differentiated from formal echocardiography by their reduced accuracy, their need for simplified imaging proto- cols, and their intent to be rapidly deployed on multiple organ systems during physical examination. The current study validates a limited cardiac imaging protocol appropri- ate for these small devices that could be easily learned and incorporated into the initial Bedside examination in the emergency department and provides improved accuracy for significant, evidence-based cardiac finding.

References

1. Kimura BJ, DeMaria AN. Hand-carried ultrasound: evolution, not revolution. Nat Clin Pract Cardiovasc Med 2005;2(4):217-23.
2. The SOLVD Investigators. Effect of enalapril on mortality and the development of heart failure in asymptomatic patients with reduced left ventricular ejection fractions. N Engl J Med 1992;327:685-91.
3. Benjamin EJ, D’Agostino RB, Belanger AJ, et al. Left atrial size and the risk of stroke and death: the Framingham Heart Study. Circulation 1995;92:835-41.
4. Tsang TS, Barnes ME, Gersh BJ, et al. Prediction of risk for first age-related cardiovascular events in an elderly population: the incremental value of echocardiography. J Am Coll Cardiol 2003; 42:119-205.
5. Lang RM, Bierig M, Devereaux RB, et al. Recommendations for chamber quantification: a report from the American Society of Echocardiography’s Guidelines and Standards Committee and the Chamber Quantification Writing Group, developed in conjunction with the European Association of Echocardiography, a Branch of the

European Society of Cardiology. J Am Soc Echocardiogr 2005;18: 1440-63.

1. Massie BM, Schiller NB, Ratshin RA, Parmley WW. Mitral-septal separation: new echocardiographic index of left ventricular function. Am J Cardiol 1977;39(7):1008-16.
2. Silverstein JR, Laffely NH, Rifkin RD. Quantitative estimation of left ventricular ejection fraction from mitral valve E point to septal separation and comparison to magnetic resonance imaging. Am J Cardiol 2006;87:137-40.
3. Kimura BJ, Fowler SJ, Nguyen DT, et al. Detection of early carotid arterial atherosclerosis by briefly trained physicians using a hand-held ultrasound device. Am J Cardiol 2003;92(2):239-40.
4. Vourvouri EC, Schinkel AF, Roelandt JR, Boomsma F, Sianos G, Bountioukos M, et al. Screening for left ventricular dysfunction using a hand-carried cardiac ultrasound device. Eur J Heart Failure 2003;5 (6):767-74.
5. Bruce CJ, Spittell PC, Montgomery SC, et al. Personal ultrasound imager abdominal aortic aneurysm screening. J Am Soc Echocardiogr 2000;13:674-9.
6. Douglas PS. The left atrium: a biomarker of chronic diastolic dysfunction and cardiovascular risk. J Am Coll Cardiol 2003;42: 1206-7.
7. Kimura BJ, Fowler SJ, Fergus TS, et al. Detection of left atrial enlargement using hand-carried ultrasound devices: Implications for bedside examination. Am J Med 2005;118(8):912-6.
8. Kimura BJ, Kedar E, Weiss DE, Wahlstrom CL, Agan DL. A hand- carried ultrasound sign of cardiac disease: the left atrium-to-aorta diastolic ratio. Am J Emerg Med 2010;28(2):203-7.
9. Kimura BJ, Amundson SA, Willis CL, et al. Usefulness of a hand-held ultrasound device for bedside examination of left ventricular function. Am J Cardiol 2002;90(9):1038-9.
10. Popp RL. The physical examination of the future: echocardiography as part of the assessment. ACC Curr Rev 1998;7:79-81.
11. Roelandt JRTC. A personal ultrasound imager (ultrasound stetho- scope): a revolution in the physical cardiac diagnosis! Eur Heart J 2002;23:523-7.