| Home | E-Submission | Sitemap | Contact us |  
top_img
Sleep Med Res > Volume 7(2); 2016 > Article
Byun, Song, Cha, and Shin: Reliability of Manual and Automatic Scoring of Single Channel Nasal Airflow Device (ApneaLink) in Determining Moderate or Severe Obstructive Sleep Apnea Syndrome

Abstract

Background and Objective

ApneaLink (AL) is a single-channel portable monitoring device that identifies apnea or hypopnea from nasal airflow and is commonly used for screening and diagnosis of obstructive sleep apnea (OSA). This study aimed to evaluate the reliability of automatic and manual scoring of AL in patients with moderate or severe OSA.

Methods

This was a retrospective single-center study. Consecutive patients who underwent AL and confirmatory standard polysomnography (PSG) for evaluation of OSA within a 3-month interval were included. Sensitivity, specificity and receiver operating characteristic curves for AL were calculated for automatic and manual scoring with the apnea-hypopnea index (AHI). Pearson correlations and Bland-Altman plots were computed.

Results

Eighty-one patients were included. Overall, their mean age was 56 ± 10 years, and 80% were male. A correlation was found between the AL and PSG AHI and the apnea index but not with the hypopnea index. AL showed high sensitivity (82.7%) for moderate OSA with a good area under the curve (AUC) value of 0.710. Compared with automatic scoring methods, manual scoring provided higher AUC and specificity as well as positive and negative predictive value.

Conclusion

sThis study shows a high level of agreement in the AHI between AL and PSG performed within 3 months. The result of manual scoring with AL was more in agreement with overnight PSG than the result of automatic scoring was.

INTRODUCTION

Obstructive sleep apnea (OSA) is associated with cardiovascular, cerebrovascular, and metabolic disease [1-3]. Treatment of OSA with continuous positive airway pressure (CPAP) can ameliorate high blood pressure [4] and insulin resistance [5]. Because OSA is a prevalent disorder affecting 4.5% and 3.2% of Korean men and women, respectively [6], adequate screening for the disease is a major health issue.
The gold standard for diagnosis of OSA is overnight polysomnography (PSG). However, PSG studies are expensive and labor intensive [7]. Portable monitoring devices allow screening for disordered breathing during sleep with lower cost and more convenience; however, their reliability has always been a concern. Practice parameters for the use of portable monitoring devices published in 2003 discourage the use of type 4 (one- or two-channel devices including oxygen saturation or airflow) to identify an apnea-hypopnea index (AHI) of greater or less than 15. However, recent findings suggest that a single-channel portable monitoring device as a home sleep test can be effective in diagnosing OSA and making related therapeutic decisions [8,9].
ApneaLink (AL) is a type 4 single-channel monitoring device. The pressure transducer in AL is connected to a nasal cannula inserted into the user’s nostrils, and it records nasal airflow. AL software then automatically analyzes the data and produces a 1-page report that contains the AHI, apnea index (AI) and hypopnea index (HI). Apnea and hypopnea can be measured either automatically or manually. Several validation studies reported that AL can identify patients with AHI ≥ 15 with high sensitivity and specificity. Manual scoring of the data produced more accurate results than the automatic analysis. However, only a few reports have evaluated the use of AL, and no studies have evaluated manual versus automatic scoring of the device in Korean patients with OSA.
This study aimed to evaluate the accuracy of automatic and manual scoring of AL compared with standard PSG. We 1) evaluated agreement of respiratory events between the two devices, 2) determined the accuracy of AL in identifying patients with OSA with AHI ≥ 15 or 30, and 3) compared accuracy between automatic and manual scoring of AL.

METHODS

Patients

This retrospective single-center study was performed at Kyung-Hee University Hospital at Gangdong between March 2008 and January 2015. Consecutive patients who were suspected to have OSA according to their symptoms (snoring with or without witnessed apneas and/or somnolence) and who underwent AL for evaluation of OSA and confirmatory standard PSG within a 3-month interval were included. Those who had any therapy during PSG or underwent split-night studies were excluded. This study was approved by the Ethics Committee at Kyung-Hee University Hospital at Gangdong.

Polysomnography

PSG was performed using a digital polygraph system (Grass-Telefactor Twin version 2.6; Astro-Med Inc., West Warwick, RI, USA). The data were manually scored according to the American Academy of Sleep Medicine Manual for the Scoring of Sleep and Associated Events, Version 2.0 [10]. Apnea was scored if there was a drop of ≥ 90% in the oronasal thermal sensor signal for ≥ 10 seconds. Hypopnea was scored if the peak signal excursions of the nasal pressure sensor dropped by ≥ 30% for ≥ 10 seconds with ≥ 3% oxygen desaturation from the pre-event baseline or with event-associated arousal.

ApneaLink

Signals from the recorder were automatically analyzed based on default settings using AL software (version 9.30; ResMed Corporation, San Diego, CA, USA). The AL default settings for apneas and hypopneas were used as in previous studies [11-14]. Apnea was defined as a decrease in airflow by 80% of baseline for 10 to 100 seconds, and hypopnea was defined as a decrease in airflow by 50% of baseline for 10 to 100 seconds. The software also provides manual editing, which enables the operator to edit or delete events or to insert new ones. The results of automatic scoring were reviewed and manually corrected by a technologist trained in PSG reading. The AHI from automatic scoring (AL-AHIa) and manual scoring (AL-AHIm) was used for this study.

Statistical Analysis

Values were expressed as the mean ± standard deviation (SD). A Bland-Altman plot, which represents observed differences between paired measurements, was used to assess the agreement between PSG-AHI and AL-AHIa or AL-AHIm. The limits of agreement have been defined as ± 2 SD. The degree of association among the respiratory event variables (AI, HI, and AHI) between PSG and AL (with automatic or manual scoring) was evaluated using the Pearson correlation coefficient (r).
Receiver operating characteristic (ROC) curves were plotted to assess the diagnostic accuracy of automatic and manual scoring of the AL using AHI cutoff points of 15 for moderate OSA and 30 for severe OSA. A greater area under the curve (AUC) represents a better instrument. Sensitivity, specificity, positive predictive values and negative predictive values (NPV) of AL were calculated for the cutoff points. Statistical analyses were conducted using SPSS version 18.0 (SPSS Inc., Chicago, IL, USA).

RESULTS

Clinical Features and Demographics

Initially, 98 consecutive patients who underwent AL followed by PSG within 3 months were considered for inclusion. Seven patients whose AL raw data were unavailable and ten patients who had split-night PSG were excluded. Eventually, a total 81 patients were included in the analysis.
Overall, the mean age was 56 ± 10 years; 80% were male, with a mean body mass index (BMI) of 26.3 ± 3.2 kg/m2. The mean interval between the AL and PSG studies was 27.7 ± 19.2 days. Patient characteristics and the results of PSG studies are shown in Table 1.

Agreement of PSG-AHI vs. AL-AHIa and AL-AHIm

PSG-AHI ranged from 0 to 81.8/hour, with a mean value of 24.2 ± 19.4/hour. AL-AHIa and AL-AHIm ranged from 2 to 75/hour and 4 to 72/hour, with a mean of 24.7 ± 15.3/hour and 23.2 ± 14.6/hour, respectively. PSG identified moderate to severe OSA (AHI ≥ 15) in 52 of 81 (64.2%) patients. AL showed moderate to severe OSA in 61 (75.3%) patients with the automatic scoring and 60 (74.1%) patients with the manual scoring. Nine patients with moderate to severe OSA were misclassified as false-negative by AL by either automatic or manual scoring. A Bland-Altman plot of the data showed very good agreement between the PSG-AHI and AL-AHIa or AL-AHIm, with mean difference values of -0.511 and 0.995, respectively (Fig. 1).
PSG-AHI was significantly correlated with AL-AHIa and AL-AHIm. The AI also showed a significant correlation between PSG and AL-AHIa or AL-AHIm; however, the HI did not (Fig. 2).

Accuracy of Automatic and Manual Scoring of AL for Moderate and Severe OSA Prediction

ROC curves with PSG-AHI cutoff values of 15 and 30 indicated an AUC measurement of 0.648 and 0.704 for automatic scoring of AL and 0.710 and 0.791 for manual scoring, respectively (Fig. 3). For moderate OSA, the sensitivity of AL was high, but the specificity was low. For severe OSA, the specificity of AL was high, but the sensitivity was low. Positive and NPV were higher with manual scoring for both moderate and severe OSA (Table 2).

DISCUSSION

This study shows a high level of agreement in the AHI between AL, an unattended type of home sleep test device, and PSG performed within 3 months. AL demonstrated a high level of sensitivity (> 80%) for moderate OSA and a high level of specificity for severe OSA. Manual scoring of the AL data provided more reliability than automatic scoring in identifying both moderate and severe OSA. Our data support the use of AL for screening moderate OSA and emphasize the need for manual correction of its data.
The PSG-AHI and AL-AHI were strongly correlated with each other for both automatic and manual scoring. Agreement in the AHI between AL and PSG has been demonstrated in several previous studies [9,11,15,16]. Our data show that AL can reliably predict actual the AHI in patients with sleep apnea in a real clinical setting. Unlike the AI, which is also highly correlated between PSG and AL, the HI was discordant between the two tests. Because AL uses only a pressure cannula to score hypopnea, respiratory events consisting of a 30–50% drop in nasal pressure with more than 3% desaturation or arousals may not be scored as a hypopnea. Additionally, the pressure cannula tends to misclassify apneas as hypopneas [17]. Therefore, the AHI from AL should be interpreted cautiously in accordance with clinical findings in hypopnea-dominant OSA patients.
AL identified patients with moderate OSA with more than 80% sensitivity. This result is in line with previous studies, as summarized in Table 3. However, for severe OSA, the sensitivity of AL was less than 50%, which was lower than in previous studies that simultaneously recorded AL and PSG [11,12,15,16]. However, one study that performed PSG within 16 weeks after AL reported a similar sensitivity value to our study [12]. The low sensitivity that we observed for severe OSA may have occurred because we also performed AL and PSG at different time points (within 3 months). The specificity and AUC value of AL in our study were much lower than in previous reports (meaning, more data were false positive), regardless of whether OSA was moderate or severe. This result may also be caused by the different time points of AL and PSG. Lower BMI in our patients may also have caused the difference. Because it is better to “screen in” than to “screen out” cases for screening purposes, AL may be useful in screening patients with moderate OSA who require CPAP therapy.
Diagnostic performance of the manual analysis of AL was superior to the performance of automated scoring, which is in line with previous reports [8,12]. Compared with automatic scoring methods, manual scoring provided a higher AUC and specificity as well as positive and negative predictive value. Discordance between manual and automatic scoring was prominent in hypopnea events. Additionally, abrupt changes in airflow amplitude, as in movements or sighs, can be misclassified as hypopneas with automatic scoring. Visual inspection of the data can correct such errors and provide more accurate AHI data.
Although this study was the first to determine the usefulness of AL for screening suspected OSA in Korean patients, several limitations should be noted. As this was a retrospective single-center study and not all patients with clinically suspected OSA underwent AL and follow-up PSG study, there is a risk of selection bias. We used the AHI from PSG as a reference value; however, in-laboratory PSG performed within 3 months does not represent sleep-disordered breathing at home during AL. Night-to-night variability and home-lab environmental differences may have affected the number of respiratory events. Moreover, because AL does not record electroencephalography, total sleep time can be overestimated, which can result in a difference in the AHI for the same respiratory events. Even with these limitations, our study shows how reliable AL is in making clinical decisions for evaluating and treating suspected OSA patients.

Conclusion

A single-channel portable monitoring device (AL) identified patients with moderate OSA with high sensitivity. Manual correction of automatic scoring data provided more reliable AHIs for determining both moderate and severe OSA. For patients with symptoms of sleep-disordered breathing, AL can be used as a reliable tool for screening patients with moderate OSA with manual scoring. Those with an AL-AHIm of 15 or more should be further evaluated and considered as CPAP candidates.

Notes

Conflicts of Interest
The authors have no financial conflicts of interest.

REFERENCES

1. Gottlieb DJ, Yenokyan G, Newman AB, O’Connor GT, Punjabi NM, Quan SF, et al. Prospective study of obstructive sleep apnea and incident coronary heart disease and heart failure: the sleep heart health study. Circulation 2010; 122: 352-60.
crossref pmid pmc
2. Peppard PE, Young T, Palta M, Skatrud J. Prospective study of the association between sleep-disordered breathing and hypertension. N Engl J Med 2000; 342: 1378-84.
crossref pmid
3. Loke YK, Brown JW, Kwok CS, Niruban A, Myint PK. Association of obstructive sleep apnea with risk of serious cardiovascular events: a systematic review and meta-analysis. Circ Cardiovasc Qual Outcomes 2012; 5: 720-8.
crossref pmid
4. Montesi SB, Edwards BA, Malhotra A, Bakker JP. The effect of continuous positive airway pressure treatment on blood pressure: a systematic review and meta-analysis of randomized controlled trials. J Clin Sleep Med 2012; 8: 587-96.
crossref pmid pmc
5. Iftikhar IH, Hoyos CM, Phillips CL, Magalang UJ. Meta-analyses of the association of sleep apnea with insulin resistance, and the effects of CPAP on HOMA-IR, adiponectin, and visceral adipose fat. J Clin Sleep Med 2015; 11: 475-85.
crossref pmid pmc
6. Kim J, In K, Kim J, You S, Kang K, Shim J, et al. Prevalence of sleep-disordered breathing in middle-aged Korean men and women. Am J Respir Crit Care Med 2004; 170: 1108-13.
crossref pmid
7. Flemons WW, Douglas NJ, Kuna ST, Rodenstein DO, Wheatley J. Access to diagnosis and treatment of patients with suspected sleep apnea. Am J Respir Crit Care Med 2004; 169: 668-72.
crossref pmid
8. Masa JF, Duran-Cantolla J, Capote F, Cabello M, Abad J, Garcia-Rio F, et al. Efficacy of home single-channel nasal pressure for recommending continuous positive airway pressure treatment in sleep apnea. Sleep 2015; 38: 13-21.
crossref pmid pmc
9. Masa JF, Duran-Cantolla J, Capote F, Cabello M, Abad J, Garcia-Rio F, et al. Effectiveness of home single-channel nasal pressure for sleep apnea diagnosis. Sleep 2014; 37: 1953-61.
crossref pmid pmc
10. Berry RB, Budhiraja R, Gottlieb DJ, Gozal D, Iber C, Kapur VK, et al. Rules for scoring respiratory events in sleep: update of the 2007 AASM Manual for the Scoring of sleep and associated events. Deliberations of the sleep apnea definitions task force of the American academy of sleep medicine. J Clin Sleep Med 2012; 8: 597-619.
pmid pmc
11. Erman MK, Stewart D, Einhorn D, Gordon N, Casal E. Validation of the apneaLink for the screening of sleep apnea: a novel and simple single-channel recording device. J Clin Sleep Med 2007; 3: 387-92.
pmid pmc
12. Nigro CA, Dibur E, Aimaretti S, González S, Rhodius E. Comparison of the automatic analysis versus the manual scoring from ApneaLink-TM device for the diagnosis of obstructive sleep apnoea syndrome. Sleep Breath 2011; 15: 679-86.
crossref pmid
13. Oktay B, Rice TB, Atwood CW Jr, Passero M Jr, Gupta N, Givelber R, et al. Evaluation of a single-channel portable monitor for the diagnosis of obstructive sleep apnea. J Clin Sleep Med 2011; 7: 384-90.
pmid pmc
14. Ng SS, Chan TO, To KW, Ngai J, Tung A, Ko FW, et al. Validation of a portable recording device (ApneaLink) for identifying patients with suspected obstructive sleep apnoea syndrome. Intern Med J 2009; 39: 757-62.
crossref pmid
15. Chen H, Lowe AA, Bai Y, Hamilton P, Fleetham JA, Almeida FR. Evaluation of a portable recording device (ApneaLink) for case selection of obstructive sleep apnea. Sleep Breath 2009; 13: 213-9.
crossref pmid
16. Ragette R, Wang Y, Weinreich G, Teschler H. Diagnostic performance of single airflow channel recording (ApneaLink) in home diagnosis of sleep apnea. Sleep Breath 2010; 14: 109-14.
crossref pmid
17. Farré R, Rigau J, Montserrat JM, Ballester E, Navajas D. Relevance of linearizing nasal prongs for assessing hypopneas and flow limitation during sleep. Am J Respir Crit Care Med 2001; 163: 494-7.
crossref pmid

Fig. 1.
Bland-Altman plot of PSG AHI data and ApneaLink AHI. (A) with automatic scoring and (B) with manual scoring of the ApneaLink. PSG: polysomnography, AHI: apnea-hypopnea index, SD: standard-deviation.
smr-2016-00129f1.gif
Fig. 2.
Correlation of AHI, Apnea index, Hypopnea index between PSG and ApneaLink. Automatic scoring of ApneaLink is presented in upper row and manual scoring in lower row. AHI: apnea-hypopnea index, PSG: polysomnography.
smr-2016-00129f2.gif
Fig. 3.
Receiver-operator characteristic curve from the ApneaLink (AL) AHI with automatic versus manual scoring. (A) for moderate OSA (PSG-AHI ≥ 15) and (B) for severe OSA (PSG-AHI ≥ 30). PSG: polysomnography, AHI: apnea-hypopnea index, OSA: obstructive sleep-apnea, AL-AHIa: AHI-automatic scoring, AL-AHIm: AHI-manual scoring.
smr-2016-00129f3.gif
Table 1.
Demographic characteristics and result of polysomnography
Age years 55.8 ± 10.2
Sex (male) 65 (80.2)
BMI kg/m2 26.3 ± 3.2
Questionnaire score
 BDI 10.4 ± 7.1
 ESS 5.9 ± 3.3
 PSQI 14.1 ± 7.3
PSG result
 TST min 315.5 ± 45.9
 N1 sleep% 26.0 ± 14.3
 N2 sleep% 38.6 ± 11.2
 N3 sleep% 18.8 ± 10.7
 REM sleep% 16.5 ± 8.0
 WASO% 18.4 ± 14.3
 MinO2% 81.5 ± 8.3
 PLMI /hr 5.8 ± 15.1
 Apnea Index 14.6 ± 16.6
 Hypopnea Index 9.7 ± 10.2
 AHI 24.2 ± 19.4
 RDI 35.6 ± 17.7
 OSA severity
  Normal (AHI < 5) 10 (12.3)
  Mild (AHI ≥ 5 –< 15) 20 (24.7)
  Moderate(AHI ≥ 15 –< 30) 28 (34.6)
  Severe (AHI ≥ 30) 23 (28.4)

All values are given as the mean ± standard deviation or number (percentage).

BMI: body-mass index, BDI: Beck depression inventory, ESS: Epworth Sleepiness Scale, PSQI: Pittsburg sleep quality index, TST: total sleep time, WASO: Wake after sleep onset, MinO2%: minimum oxygen saturation, PLMI: periodic limb movement index, AHI: apnea-hypopnea index, RDI: respiratory disturbance index, OSA: obstructive sleep-apnea.

Table 2.
Sensitivity, specificity, PPV, NPV, and AUC of AHI derived by the ApneaLink device for diagnosing moderate (PSG AHI ≥ 15–< 30/hr) and severe OSA (PSG AHI ≥ 30/hr)
AHI Sensitivity
Specificity
PPV
NPV
AUC
Auto Manual Auto Manual Auto Manual Auto Manual Auto Manual
≥ 15–< 30 82.7 82.7 37.9 41.4 70.5 71.7 55 57.1 0.648 0.71
≥ 30 37.5 45.8 80.7 86 45 57.9 75.4 79 0.704 0.791

AHI: apnea-hypopnea index, OSA: obstructive sleep-apnea, PPV: positive predictive value, NPV: negative predictive value, AUC: area under the curve, PSG: polysomnography.

Table 3.
Summary of previous reports that evaluated Sens, Spec, PPV, NPV, and AUC of AL for identifying moderate or severe OSA
Interval between PSG and AL Scoring Pts. No. Age Male BMI Sens Spec PPV NPV AUC
AHI ≥ 15/hr
 Simultaneous [11] Auto 59 57.3 49% 32.6 90.9 94.6 90.9 94.6 0.977
 Simultaneous [12] Auto 50 48.7 64% 32.2 87.5 88.9 0.944
 Simultaneous [13] Auto 102 54.7 76% 29.5 92 88.5
 Simultaneous [15] Auto 90 49.6 77% 29.3 76.7 83
 Within 2 weeks [11] Auto 39 - - - 84.6 92.3 84.6 92.3 0.923
 Within 8 weeks [13] Auto 131 59.1 73% 28 73.1 84.7
 Within 16 weeks [16] Auto 53 45.1 45% 35.9 73.7 85.3 73.7 85.3 0.924
 Simultaneous [15] Manual 90 49.6 77% 29.3 72.1 97.9
AHI ≥ 30/hr
 Simultaneous [12] Auto 50 48.7 64% 32.2 88.2 93.9 0.954
 Simultaneous [15] Auto 90 49.6 77% 29.3 88.5 95.3
 Within 16 weeks [16] Auto 53 45.1 45% 35.9 55.6 95.5 71.4 91.3 0.924
 Simultaneous [15] Manual 90 49.6 77% 29.3 84.6 98.4

BMI: body-mass index, Sens: sensitivity, Spec: specificity, PPV: positive predictive value, NPV: negative predictive value, AUC: area under the curve, OSA: obstructive sleep-apnea, AHI: apnea-hypopnea index, PSG: polysomnography, AL: ApneaLink, Auto: automatic scoring.

TOOLS
PDF Links  PDF Links
PubReader  PubReader
ePub Link  ePub Link
Full text via DOI  Full text via DOI
Download Citation  Download Citation
  E-Mail
  Print
Share:      
METRICS
0
Crossref
0
Scopus
3,993
View
88
Download