KEYWORDS: Binary data, Nonlinear optics, Complex systems, Telecommunications, Optical communications, Modulation, Signal generators, Digital signal processing, Neurons, Signal to noise ratio
We investigated the problem of overfitting of artificial neural networks (ANNs) which are used for digital nonlinear equalizers in optical communication systems. ANN-based digital signal processing (DSP) can be used in the time domain or the frequency domain to compensate for the optical nonlinearity. We have reported the advantages of ANN over Volterra series transfer function (VSTF) in terms of the computational complexity. However, when pseudo-random binary sequence (PRBS) data is used to evaluate the ANN-based nonlinear equalizers, the ANNs can potentially learn the repeated PRBS patterns, resulting in overestimation of the equalization performance. In this paper, we clarify that the overfitting of the ANNs hardly occurs or requires much more neuron units than used for common nonlinear equalization, if we employ multi-level modulation signals including simple 4-ary pulse-amplitude modulation (PAM4). In our study, we compared binary signal and 4-level signal generated by PRBS data. White Gaussian noise (WGN) was added to the signals. The ANN-based nonlinear equalizers were trained using a least mean squares (LMS) algorithm, to attempt to “compensate” for the noise. The signal quality was evaluated using the error vector magnitude (EVM). When the overfitting occurs, the EVM is improved by the nonlinear equalization. The results revealed that, in the case of 4-level signal, the influence of the overfitting was suppressed in comparison with the case of the binary signal.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.