Deep-learning end-to-end autoencoder for the joint mitigation of chromatic dispersion and Kerr nonlinearity in optical communication systems

• Applicant: Professor Dr.-Ing. Stephan ten Brink
• Subject Area: Communication Technology and Networks, High-Frequency Technology and Photonic Systems, Signal Processing and Machine Learning for Information Technology
• Term: since 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 460943258

In this project, we seek to improve the spectral efficiency of optical communication systems by using methods from artificial intelligence, and, as a result, increase reach and data rates. We put emphasis on the interpretability of the learnings -- facilitated by using the idea of "architectural templates" -- to enable the derivation of insights about the optical channel and its capacity. At first glance, optical fibers promise a seemingly infinite bandwidth, a static propagation environment combined with low noise and small attenuation coefficients. However, in the previous decades, the exponential growth of data-rates and the fast progress in circuit design have pushed the occupied bandwidth and sampling rates towards an operation point where even the seemingly perfect optical fiber is dominated by nonlinearity that cannot be neglected nor compensated easily anymore. From an engineering perspective, this opens up an exciting field of research to mitigate such impairments. At the same time, deep learning-driven communications has become a promising and active research topic, in particular in the wireless domain. It has been shown that end-to-end learning of transmitter and receiver in a joint manner via an "autoencoder" allows to find new signal constellations and even waveforms for (almost) arbitrary channels that are not restricted to linear scenarios, and that have not been accessible via classic methods before. We, thus, are attracted by the challenges of signaling across the nonlinear optical fiber and the conceptual simplicity of the end-to-end learning framework. Based on our previous results in wireless and optical communications using neural networks, we seek to come up with novel architectural templates and learning concepts tailored to the optical fiber channel, for increasing spectral efficiency, reach and data rates over single wavelength channels, as well as over wavelength division multiplex-based systems. Also, we intend to study the relationship of the novel architectural templates (with learned signal constellations and waveforms) to Eigenvalue-based optical communications using the nonlinear Fourier transformation, to find further ideas for jointly compensating chromatic dispersion and fiber nonlinearities.

• DFG Programme: Research Grants