1.

INTRODUCTION

Sleep is an important biological activity to most creatures. With a sufficient sleep, living organisms have a chance to recover from the day’s activities and maintain a normal physiological balance. While sleeping, several physiological changes occur. For example, an organism’s sensory awareness and metabolic rate will decrease, muscle activities will be inhibited. As a result, the organism will produce less metabolic waste than they can be eliminated¹. Thus, sleeping is an important way for organisms to restore the normal function to various systems, which allows them for better performance during wakeful activities.

An effective and healthy sleep is closely related to the healthy life of human beings. Some people may suffer from poor sleep quality, and accurate classification of sleep status and a detailed analysis of sleep quality have become vital prerequisite for doctors to assist in improving sleep. According to scientists’ research on the human sleep process, human sleep activity is an irregular biological state². Sleep can be divided into a series of repeated stages, the cycle varies from person to person, between 60 and 90 minutes. The series of stages can be divided into wakefulness, NREM sleep stage and REM sleep stage. NREM sleep can be further subdivided into four stages: I, II, III, and IV. This is the R&K standard which is widely used to classify sleep stages around the globe³.

One of the authoritative references on sleep scoring and staging is the AASM Manual for the Scoring of Sleep and Associated Events: Rules, Terminology and Technical Specifications. According to this manual, patients are required to wear PSG with the help of an experienced doctor when assessing sleep stage. Typically, there are three to six electrodes for collecting EEG signals, three electrodes for EMG signals and another two electrodes for EOG signals in addition to pulse oximeters, breathing detectors⁴. After the data are collected, an experienced doctor will complete the final sleep stage classification based on the EEG curve characteristics, combined with other supporting features, such as patients’ physical performance, moving events, respiratory behaviors. Although PSG is currently the key standard for sleep staging, the PSG device is large and complicated to use. Patients are equipped with dozens of signal sensors on the body when they are sleeping. Their sleep quality will also be affected by the equipment, which introduces additional variables beyond themselves. Raw data must be subjected to long-term manual monitoring and analysis by doctors, resulting in potential errors and deviations from actual values during staging analysis.

The surge in the occurrence of sleep problems has spurred extensive research in sleep science. Sleep staging is a widely researched and multidisciplinary application that encompasses various fields, including medicine, sensor technology, and computer science. The traditional method of sleep staging involves manual marking by experienced doctors, which is a time-consuming and expensive process. Consequently, several research studies and efforts have been conducted to solve this problem. There are a variety of automatic sleep staging systems been developed, including those based on EEG^5-7, those based on all PSG signals^8-10, and those based on pre-feature extraction based on EEG and EOG signals¹¹

In this paper, we build an automatic sleep staging model using deep learning. We only take patients’ respiratory signal collected via PSG and basic physical data as inputs. Firstly, a low-pass filter is implemented to reduce high-frequency noise from the original signal. Next, we extract features from patients’ respiratory signal in time domain and frequency domain simultaneously using short-time Fourier transform, wavelet packet processing and convolutional neural network. Finally, we use a transformer encoder is used to combine the features extracted above with additional metadata from patients, including their gender, age, and weight. This model is a comprehensive application of convolutional neural network, transformer encoder, Word2vec to unveil hidden features within the signal data.

2.

METHODOLOGY

Preprocessing

Because of some reasons when collecting the raw data, the signal is full of high-frequency noise. Firstly, we designed a Butterworth low-pass filter to remove the noise. The comparison is shown in figure 1.

Figure 1.

Signal before (up) filtering and after (down) filtering

After the signal is filtered, we divide the signal into 30 seconds part and save it with the label annotated by experienced doctor provided by the database.

Feature extraction

The short-time Fourier transform (STFT) is a method of extracting partial frequency features in a signal. Firstly, it uses a fixed window function to divide a long continuous time-domain signal into several shorter segments of the same length. Then, for each segment, a Fourier transform is performed separately. Finally, overlap these segments and generate a spectrogram of the entire signal.

Given a time-domain signal x[n], a window w[n], and a hop size H. The calculation process is as follows.

Continuous wavelet transform (CWT) is also a method of extracting and compressing information from the original time-domain signal. It uses several wavelet functions to convolve the original signal, decomposes the complex information carried by the original signal into several basic waves, and achieves the original signal. To achieve the purpose of analyze the signal in time-domain and frequency-domain simultaneously.

Given a mother wavelet which is a continuous function in both the time-domain and the frequency-domain ψ(t) to generate child wavelet in the further calculation.

By shifting and translating the mother wavelet, and convolving with the original signal. We can get:

We use STFT and CWT to extract features in frequency-domain from the original signal, and save them in a 224x224 color map as shown in figure 2.

Figure 2.

Signal features (left: STFT, right: CWT)

Model

In general, a person’s respiratory signal is related to his physical status and basic information such as age, weight, and gender. Some traditional deep learning model may only take one feature into account, for example, time-domain signal series, while ignoring some other details. Inspired by the model introduced by Minh¹³. Our model can accept the timedomain signal, frequency-domain features, and other meta data simultaneously. There will be auto-encoder to mine potential relationship between the features extracted separately by different model. The model design is shown in figure 3:

Figure 3.

Model structure

Feature extraction module This module aims to extract feature vector from the inputs separately. We use 1D Resnet-34 to extract features from the time-domain signal and two independent 2D Resnet-34 to process the STFT feature map and the CWT feature map. For the meta data part, an auto encoder Word2vec is used to extract features. These four feature extraction components will generate four 64-dimensional features.

Transformer encoder module This module aims to generate encodings contained in the inputs and between the inputs. For each part of the input, the multi-head attention module will weigh the relevance from the four feature vectors and extract information from them. The feed-forward module will perform additional processing of the outputs from the multi-head attention module, such as normalization and residual connects.

Model output module This module aims to integrate the feature vector produced from the transformer encoder module and give a prediction. In this module, we use several full-connect layers and dropout layers, ReLU and SoftMax as the activation functions, to generate the result.

3.

RESULTS

Datasets The MIT-BIH Polysomnographic dataset consists of 130 hours long respiratory signal records of 18 patients. The recordings are digitized at a sampling frequency of 250Hz. There are a total of 16 different labeled data in the database tag. In our experiments we choose 6 common labels related to sleep staging. The labels and their meaning are shown in the following table 1.

Table 1.

Labels and their meanings

Before training, we found that there is a data imbalance in the original database. We use the following strategies to enhance the database data to get a better performance:

After removing partially overlapping signal records, the data is divided into training set and test set at a ratio of 85:15.

Precision and recall matrixes represent how well the model performs.

F1-score represents the harmonic mean of the precision score and recall score.

The experiment result is shown in figure 4.

Figure 4.

Model performance (6-categories)

To compare with other sleep staging models in table 2, we combine label 3 and label 4 as label 3. And the prediction result is shown below in figure 5.

Figure 5.

Model performance (5-categories)

Table 2.

Performance comparison between our model with some other sleep staging model

4.

DISCUSSION AND CONCLUSION

In this paper, we assume that a person’s sleep depth can be reflected from respiratory signal. To prove this concept, we build a multi-module neural network and use the patient’s respiratory signal data collected from PSG during sleep from the MIT-BIH Polysomnographic dataset. And our model achieved an accurate score of 81.96%. By deploying our automated sleep staging system, the characteristics of respiratory signal can be fully mined, and get a similar effect of the model using EEG, EOG, ECG and many other signals collected by PSG.

Our experiments demonstrate the feasibility of performing an automatic sleep staging using only a patient’s respiratory signal. This model implies that in some non-emergency medical scenarios, we can complete sleep staging without PSG. This will eliminate the influence carried by the complicated monitoring equipment. Our automated sleep staging system shows great potential for use in future scenarios, such as home-based medical treatment. The model can be deployed in some IoT devices, and enables people to monitor their sleep at home without the help of the doctors. Moving forward, we believe that our automated sleep staging system will provide people more accessible, cost-effective, and personalized sleep analytics.