XX Month, XXXX
XX Month, XXXX
XX Month, XXXX
XX Month, XXXX
XX Month, XXXX

Deep Learning-Aided 6G Wireless Networks: A Comprehensive Survey of
Revolutionary PHY Architectures

Burak Ozpoyraz1, Graduate Student Member, IEEE,
A. Tugberk Dogukan1, Graduate Student Member, IEEE,
Yarkin Gevez1, Graduate Student Member, IEEE,
Ufuk Altun1, Graduate Student Member, IEEE,
Ertugrul Basar1, Senior Member, IEEE

Communications Research and Innovation Laboratory (CoreLab), Department of Electrical and Electronics Engineering, Koç University, Sariyer 34450, Istanbul, Turkey (e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
Corresponding author: Ertugrul Basar (e-mail: ebasar@ku.edu.tr).
This work is supported by TUBITAK under Grant Number 121C254.
DEEP LEARNING-AIDED 6G WIRELESS NETWORKS: A COMPREHENSIVE SURVEY OF REVOLUTIONARY PHY ARCHITECTURESOZPOYRAZ et al.

Deep learning (DL) has proven its unprecedented success in diverse fields such as computer vision, natural language processing, and speech recognition by its strong representation ability and ease of computation. As we move forward to a thoroughly intelligent society with 6G wireless networks, new applications and use cases have been emerging with stringent requirements for next-generation wireless communications. Therefore, recent studies have focused on the potential of DL approaches in satisfying these rigorous needs and overcoming the deficiencies of existing model-based techniques. The main objective of this article is to unveil the state-of-the-art advancements in the field of DL-based physical layer methods to pave the way for fascinating applications of 6G. In particular, we have focused our attention on four promising physical layer concepts foreseen to dominate next-generation communications, namely massive multiple-input multiple-output systems, sophisticated multi-carrier waveform designs, reconfigurable intelligent surface-empowered communications, and physical layer security. We examine up-to-date developments in DL-based techniques, provide comparisons with state-of-the-art methods, and introduce a comprehensive guide for future directions. We also present an overview of the underlying concepts of DL, along with the theoretical background of well-known DL techniques. Furthermore, this article provides programming examples for a number of DL techniques and the implementation of a DL-based multiple-input multiple-output by sharing user-friendly code snippets, which might be useful for interested readers.

Deep learning, 6G, massive multiple-input multiple-output (MIMO), multi-carrier (MC) waveform designs, reconfigurable intelligent surfaces (RIS), physical layer (PHY) security.

[style=tree, type=, title=Nomenclature, nogroupskip, nonumberlist]

Emerging applications of 6G wireless networks.

§ INTRODUCTION

Starting with the 3gpp (3rd Generation Partnership Project) Release-15 standard in March 2017, the brand new 5th generation (5g) wireless communication technology has made significant progress worldwide [1], [2]. 3GPP completed the Release-17 Stage-2 freeze in June 2021 and is planning the next freeze in March 2022 [3]. As these releases have been proceeding, the deployment process of 5G has attained a high level of maturity. According to a recent GSA report, 176 operators from 72 countries had launched commercial 5G networks as of August 2021 [4]. However, the recent advancements in the implementation of 5G have exposed the drawbacks and limitations of this technology despite being nominated as the primary enabler of Internet-of-Everything (ioe) systems [5]. Even though 5G has introduced more flexibility and efficiency to wireless networks by utilizing new technologies such as millimeter-wave (mmwave) communication and multi-numerology orthogonal frequency division multiplexing (ofdm), the pledged revolutionary mobile communication perspective has remained a pipe dream thus far [5], [6]. Moreover, although 5G systems are ready to support data rate-hungry enhanced mobile broadband services and ultra-reliable low-latency communications, it is questionable whether they can support future IoE applications [6]. Therefore, researchers have been investigating sophisticated communication technologies and developing intriguing concepts for 6th generation (6g) wireless networks, which are projected to come into life after 2030. Even if cellular communication systems have advanced to a new level with the development of 5G, expectations of the intelligent information society of 2030 and beyond will be remarkably compelling. 6G will be the essential provider of a highly digital community by connecting everything, enabling almost unlimited wireless coverage, establishing full-vertical networks, and supporting holographic and high-precision communications for tactile and haptic applications [7]. In addition, new IoE services ranging from extended reality applications and flying vehicles to space travel and brain-computer interfaces will come to reality with 6G networks, making 6G the true master of IoE [5], as illustrated in Fig. <ref>. However, these new and fascinating applications will bring strict and very challenging requirements such as ultra-high reliability, low latency, substantially high data rate, high energy and spectral efficiency, and dense connectivity. Fig. <ref> compares the specifications of 5G and 6G, elucidating the 6G expectations [8]. The developments and expectations on the way toward these 6G technologies in the literature have also been comprehensively reviewed and discussed by some other research groups .

Comparison of 5G and 6G specifications [8].

The stringent demands of 6G have driven researchers to look for sophisticated physical layer (phy) techniques. Beyond using more spectrum in upper 100 GHz bands, such as high-frequency mmWave and terahertz bands, 6G might incorporate numerous emerging technologies such as scalable artificial intelligence (ai), index modulation (im) [12], [13], cell-free massive multiple-input multiple-output (mimo) systems [14], [15], reconfigurable intelligent surfaces (ris) [6], [16], [17], PHY security [18], [19], advanced waveforms [20], [21], satellite and non-terrestrial networks [22], [23], and wireless power transfer [24], [5], as shown in Fig. <ref>. Even though all these technologies will significantly impact 6G, AI might be the leading innovator of 6G networks, supporting a completely new outlook for wireless networks, particularly in the PHY.

The aforementioned current technologies optimize several network functions by using model-based techniques that provide characteristics of the policy involved. These techniques, yet, can be too sophisticated to be implemented physically in terms of run time, or they might contain too much abstraction to work in a broad environment. Conversely, AI-based solutions may adjust to dynamically changing scenarios and localized characteristics by absorbing knowledge of the target communication environment. AI can be implemented in a highly configurable infrastructure with a wide range of network flexibility. AI has the ability to forecast certain limits and handle vast amounts of data so that AI-integrated 6G will assist in processing the final volume of metadata with less resources and computational burden.

AI has a long and successful history, dating back to approximately 1940s when neural networks (nn) first appeared in intelligent systems. Machine learning (ml) and deep learning (dl) approaches have been proven to be competitive frameworks over the years for challenging tasks such as computer vision, robotics, and natural language processing, in which building a concrete mathematical model is relatively difficult. For example, although it is almost mission impossible to develop an analytical model or robust algorithm for detecting handwritten digits (digit recognition) or different objects in an image (image segmentation), DL techniques can accomplish these tasks with a performance exceeding human level. On the other hand, communication technologies hinge on numerous mathematical models and theories requiring expert knowledge, such as information theory and channel modeling. Information signals stream from optimal transmitter designs with modulation, coding, and signaling schemes over a range of mathematically defined channel models to be reliably detected at the receiver, in which each block is optimized individually. In addition to this well-defined PHY architecture, the ability of secure transmission along with eliminating various hardware imperfections makes communications a complicated and mature field. Therefore, ML and DL techniques must reach lofty goals to outperform existing technologies and provide discernable advantages. Applying DL approaches to optimize the building blocks separately, ranging from modulation and coding schemes (mcs) to symbol detection and channel estimation algorithms, would not produce much of a change [25]. End-to-end (e2e) optimization of communication systems as a whole in complex scenarios, such as unknown channel models and ultra-high mobility conditions, is the most fascinating concept that we expect to see in 6G, and we call this PHY revolution. Switching from the block structure of current PHY communications, where each block is responsible for a different signal processing task, to E2E optimization using DL methods represents a radical paradigm shift that will entirely transform 6G and beyond wireless communications.

An overview of emerging PHY techniques that can dominate the 6G era.

Four leading research directions in 6G on which this comprehensive survey article investigates the DL solutions.

Researchers have been studying the use of ML and DL techniques to overcome the drawbacks of current systems, particularly in the past couple of years. However, despite being exceptionally promising, DL methods experience some difficulties that restrict their viability in challenging communication scenarios. The highly-parameterized architecture of deep NNs (dnns) necessitates massive datasets to learn an appropriate mapping from features to desired outcomes, which increases the computing complexity and complicates the training process. Thus, utilizing solely data-based DNNs as a black box and leaving all predictions to model weights is questionable, at least for now, since datasets regarding communications have not reached a satisfactory level yet. Due to the limits of model-based and data-based methods, model-driven DNNs have emerged, which combine DNNs' powerful learning and mapping abilities with expert knowledge to get the most potential advantage. The two most common model-driven DNN strategies are deep unfolded networks, where DNN layers replicate iterations of an existing iterative algorithm, and hybrid networks, where DNNs help conventional models and enhance efficiency [26]. As the DL literature on communication technologies has progressed, the data scarcity problem has also become a thing of the past, paving the way for entirely data-based DL systems.

§.§ Contributions

In this comprehensive survey article, we investigate DL applications from the perspective of four prominent and innovative 6G concepts in the PHY, including massive MIMO systems, advanced multicarrier (mc) waveforms, RIS-empowered systems, and PHY security, as illustrated in Fig. <ref>. We thoroughly present the overview of the up-to-date literature on DL for PHY design, reveal the progress made so far, and compare the considered models in terms of their key characteristics. Moreover, considering the limited coverage of and overwhelming need for the implementation and programming steps of DL concepts in the literature, we exhibit example implementations by providing simple code snippets to enlighten readers interested in this field. In this regard, we also hope that our article might be helpful for wireless researchers who want to have hands-on experience on DL-based wireless system design. To our knowledge, this is the only survey in the literature that focuses on 6G from the perspective of PHY architectures and presents a complete guideline to PHY researchers.

In particular, we first shed light on DL approaches that address the challenges in massive MIMO detection and channel estimation in order to overcome the drawbacks of traditional methodologies. Combining DL-based detection and channel estimation schemes, we present entirely intelligent large-scale MIMO receiver frameworks. Furthermore, we analyze how to replace the conventional signal processing blocks in a classical massive MIMO transmitter with DL structures to provide optimal transmission schemes with low complexity. Our long-term objective for intelligent massive MIMO systems is to integrate DL-based transmitters and receivers to develop innovative E2E communication systems even in unknown channel models. As a glaring enhancement for MIMO systems, IM techniques have attracted huge interest recently. Therefore, we also examine how to improve the efficiency of the existing IM techniques and present DL-empowered IM 2.0 solutions. We also provide an implementation example for a DL-based IM system with simplified code snippets to guide interested readers on programming a DL-based MIMO system in Python.

For MC communication systems, we primarily focus on the DL-based transceiver design to improve various performance metrics. In most studies, NNs have been designed to be implemented in the receiver side of various MC communication systems with the purpose of outperforming conventional algorithms. Data-driven DL-based receiver designs are introduced to perform channel estimation and signal detection tasks. Furthermore, by using expert knowledge, model-driven NNs have been presented to further improve the performance and decrease the computational complexity. Recent studies have focused on transceiver designs optimizing the transmitter and receiver pipeline as a whole by benefiting from the E2E approach, which is put forward as a promising direction. Moreover, an extensive literature review is provided to amend the weaknesses of OFDM, which is by far the most popular waveform in current standards. Particularly, DL techniques utilizing deep unfolding stand out as promising solutions to combat major problems of OFDM.

On the side of RIS-assisted communication systems, we investigate DL-based studies that present novel frameworks to overcome the existing drawbacks of earlier RIS-based systems. We primarily introduce DL approaches constituting intelligent channel estimation and signal detection processes. We conclude that employment of DL approaches requires less pilots and training overhead than conventional techniques. Furthermore, we focus on emerging DL-based NNs eliminating model dependency for phase configuration in RIS-assisted schemes. The presented passive beamforming designs prove that DL-based processes, which unveil a direct mapping between phase shifts and received signal strength, provide robustness in different channel environments with reduced computational complexity. We also review the recent studies that present standalone operations employing fully passive elements to suit the passive nature of RISs and reduce the training overhead. Besides, due to the growing interest in non-orthogonal multiple access (noma) solutions for next-generation communication systems, we examine DL-based systems improving clustering and power allocation performance of RIS-assisted NOMA designs. Thereby, we discuss that the utilization of DL approaches provides enhanced flexibility and robust performance under dynamic states and the number of users compared to traditional methodologies.

Organization of the article.

Finally, we focus on DL-assisted PHY security approaches and investigate the existing studies in the literature. DL is implemented to PHY security schemes from three main perspectives based on the attack type: spoofing, jamming, and eavesdropping. Against spoofers, DL is mostly used at receivers to improve authentication accuracy. Especially, DL improves the legitimate/illegitimate classification problem of the authentication test and brings robustness against unknown and varying environments. DL is also implemented for anti-jamming purposes. However, most of the anti-jamming methods employ reinforcement learning to avoid jammed frequencies and apply DL for the purpose of better Q-functions approximation. Moreover, various secure communication systems implemented DL algorithms in their system models to improve the secrecy rate against eavesdroppers. Particularly, we reveal that these studies employ data-driven models to generate coding or beamforming schemes which minimize the leaked information to eavesdropper.

§.§ Organization

The organization of this article is given in Fig. <ref>. In Section II, we briefly discuss the structure of DL and introduce basic DL techniques from both theoretical and programming perspectives. We begin our extensive literature review with emerging MIMO technologies in Section III and provide an implementation example for a DL-based MIMO system model. Section IV presents  DL solutions for sophisticated MC waveforms. In Section V, we delve into DL-aided RIS communication systems. We discuss how DL methods can enable secure communications in Section VI. Finally, we conclude the article with Section VII by providing our future perspectives.

§ AN OVERVIEW OF DL TECHNIQUES

In this section, we will discuss the general structure of DL models and explain the training procedures of NNs to provide a quick overview of DL techniques. We will emphasize fundamental concepts and parameters that form DL models to facilitate understanding of proposed methods in the literature. We will also dig into the details of commonly used DL techniques, namely DNN, convolutional NNs (cnn), and recurrent NNs (rnn), by providing their theoretical background and motivations. In addition, we will discuss the applications of each technique in wireless networks. We will conclude this section with an image classification task using a simple CNN architecture and the well-known CIFAR10 dataset [27], [28].

§.§ DL Architecture at a Glance

DL is a sub-branch of ML, with multiple stacked processing layers, that enable predictions, classifications, or other decisions by learning data representations. Given raw data, in contrast to classical ML methods that strongly depend on domain-expert features, DL models generate non-linear input-output mappings to execute actions on a goal objective. Fig. <ref> illustrates the relationship between DL, ML, and AI. Each layer in an NN architecture transforms a representation to a higher level, where each transformation unlocks the learning of a more sophisticated characteristic of raw data. A model may be shallow or deep depending on the number of layers, the depth of an NN, imitating either a simplistic or complicated function of raw input data by being sensitive to vital features and insensitive to irrelevant changes such as environment or nearby objects. Aside from the input and output layers, each layer is a hidden layer consisting of hidden units known as neurons. During the representation learning process, hidden neurons in a layer calculate a weighted sum of their preceding layer's inputs and transfer the result to the next layer through a non-linear activation function, corresponding to one iteration of forward-propagation. Starting with the input layer, which passes the weighted sum of raw input data to the first hidden layer, each hidden layer handles its computation. In the end, the output layer produces the desired outcomes, which might be probabilities, category scores, or any other metric. DL researchers have identified several activation functions to introduce a non-linearity effect into forward-propagation, such as the rectified linear unit (relu), leaky ReLU, tanh, sigmoid, and more. The most popular one is the ReLU function and defined as $f(a) = \text{max}(a, 0)$ [29], [30].

Demonstration of the relationship between DL, ML, and AI.

When an NN completes a forward-propagation iteration, the algorithm computes a loss function by comparing the output scores with desired labels. This loss function represents a model error, which is subsequently utilized to update internal parameters, also known as model weights. The update of these adjustable weights corresponds to the training of the DL model. During training, the algorithm calculates the derivative of the loss function with respect to each weight to observe the amount of resultant error when weight was increased or decreased. In the opposite direction of forward-propagation, the algorithm computes weight gradients using the chain rule [31], which obtains derivatives going backward, called backpropagation. DL model updates its weights by changing them in the reverse direction of calculated gradients to decrease the error value after backpropagation [29]. The update rule is one of the most critical aspects of DL applications since it highly impacts overall learning performance and training time. Thus, researchers have developed different optimizers, including momentum [32], root mean square propagation, stochastic gradient descent (sgd) [33], adaptive moment estimation (adam) [34], and many more. Recently, Facebook Research has published a new update algorithm, called MADGRAD [35], and the DL world is debating whether this new technique can replace the ADAM optimizer. Fig. <ref> demonstrates a single run of a training procedure including forward-propagation, loss calculation, and backpropagation for a single hidden layer DNN, where the given variables will be defined shortly. A typical DL model takes hundreds of millions of these iterations to ensure that the model observes a good deal of instances and that the model error converges to a reasonable value depending on the application. A testing procedure follows training to examine whether the model can generalize to unseen occurrences, also known as the compositionality of DL models. The difference between training and test loss values represents model variance and indicates that the model is overfitting, in which DL model memorizes rather than learning. On the other hand, the difference between training and the Bayes error values corresponds to model bias and reveals that the model is underfitting, which necessitates more training. It is a common mistake to investigate model bias by comparing training error with zero error. Indeed, the Bayes error, the known optimal value that can be human-level performance or other values, should be the benchmark for bias, which is not necessarily zero.

Computation graph of a single hidden layer DNN including forward-propagation, loss calculation, and backpropagation.

We have provided a brief introduction to DL structures and training procedures, including insights of forward-propagation and backpropagation so far. In the following, we will go over the theoretical background of three frequently used DL architectures: DNN, CNN, and RNN, and present the implementation of a simple image classification task.

Three common DL architectures: (a) DNN, (b) CNN, (c) RNN.

§.§ Theoretical Foundations of DL

We will begin by analyzing DNNs, which are by far the most common DL structures used in DL applications. A DNN consists of multiple stacked layers, with each neuron connected to all neurons in the preceding and the following layers creating a fully connected structure. The number of neurons in the input and output layers equals the number of features in the raw input data and the number of categories, classes, or other targets according to the application, respectively. Fig. <ref>(a) depicts the architecture of a DNN with two hidden layers and three classes. Each link between two neurons represents a real-valued weight that forms the basis of the learning mechanism. It is critical to carefully initialize model weights to optimize learning performance and avoid vanishing or exploding gradients. A widespread initialization method is the random generation of weight matrices with uniform or Gaussian distributed elements. Another popular approach is Xavier initialization [36], developed by Xavier Glorot and Yoshua Bengio, in which randomly generated weights between layers $[l-1]$ and $[l]$ are scaled by $\sqrt{2 / (n^{[l-1]} + n^{[l])}}$, where $n^{[l]}$ is the number of neurons in the $l^{\text{th}}$ layer. The forward-propagation of the DNN given in Fig. <ref>(a) for a single instance is given as follows:

\begin{equation} \label{DNN_FP}
\begin{aligned}
\text{Hidden-1: } \mathbf{z}^{[1]} &= \mathbf{W}^{[1]} \mathbf{x} + \mathbf{b}^{[1]}, \quad \mathbf{a}^{[1]} = g^{[1]}(\mathbf{z}^{[1]}),
\\
\text{Hidden-2: } \mathbf{z}^{[2]} &= \mathbf{W}^{[2]} \mathbf{a}^{[1]} + \mathbf{b}^{[2]}, \quad \mathbf{a}^{[2]} = g^{[2]}(\mathbf{z}^{[2]}),
\\
\text{Output: } \mathbf{z}^{[3]} &= \mathbf{W}^{[3]} \mathbf{a}^{[2]} + \mathbf{b}^{[3]}, \quad \hat{\mathbf{y}} = g^{[3]}(\mathbf{z}^{[3]}).
\end{aligned}
\end{equation}

In (<ref>), $\mathbf{x}$ and $\hat{\mathbf{y}}$ represent $(4 \times 1)$-dimensional raw input data and $(3 \times 1)$-dimensional class scores, respectively, while $\mathbf{W}^{[l]}$, $\mathbf{b}^{[l]}$, $\mathbf{z}^{[l]}$, $g^{[l]}$, and $\mathbf{a}^{[l]}$ correspond to $(n^{[l]} \times n^{[l-1]})$-dimensional weight matrix, $(n^{[l]} \times 1)$-dimensional bias vector, $(n^{[l]} \times 1)$-dimensional weighted sum, the activation function, and $(n^{[l]} \times 1)$-dimensional layer output of the $l^{\text{th}}$ layer, respectively. The activation function $g$ is non-linear since using linear activation functions eliminates the effect of stacking multiple layers and results in the same performance as the model with a single output layer, as indicated in (<ref>) where the activation function is $g(x) = x$:

\begin{equation} \label{DNN_FP_LinearActivation}
\begin{split}
\hat{\mathbf{y}} &= \mathbf{W}^{[3]}(\mathbf{W}^{[2]}(\mathbf{W}^{[1]} \mathbf{x} + \mathbf{b}^{[1]}) + \mathbf{b}^{[2]}) + \mathbf{b}^{[3]}
\\
&= \bar{\mathbf{W}} \mathbf{x} + \bar{\mathbf{b}},
\end{split}
\end{equation}

where $\bar{\mathbf{W}}=\mathbf{W}^{[3]} \mathbf{W}^{[2]} \mathbf{W}^{[1]}$ and $\bar{\mathbf{b}}=\mathbf{W}^{[3]} \mathbf{W}^{[2]} \mathbf{b}^{[1]} + \mathbf{W}^{[3]} \mathbf{b}^{[2]} + \mathbf{b}^{[3]}$. As we discussed earlier, ReLU is a suitable option for hidden layers, while the output layer employs the softmax activation function, given by (<ref>), to get output scores:

\begin{equation} \label{softmax}
\mathbf{softmax}(\hat{y}_{i}) = \frac{e^{\hat{y}_{i}}}{\sum_{c=1}^{3} e^{\hat{y}_{c}}}.
\end{equation}

Here, $\hat{y}_{i}$ is the score of $i^{\text{th}}$ class. Yet, a DL network's output layer need not always employ a softmax activation function. This is dependent on the work that should be handled using the DL technique, in fact. The addressed problem that is being handled shapes the selection of the output's layer activation function. While any regression problem best fits with linear activation function, classification problems require different types of activation functions at the output layer, such as sigmoid and softmax.

When the algorithm obtains the output scores by forward-propagation, it calculates the loss value depending on these scores. The loss function might vary according to the application, and we will use the categorical cross-entropy loss function for this classification model:

\begin{equation} \label{categorical_cross_entropy_loss}
L(\mathbf{y},\hat{\mathbf{y}}) = - \sum_{c=1}^{3} y_{c} \text{log}(\hat{y}_{c}),
\end{equation}

where $\mathbf{y}$ is the one-hot vector of the ground truth labels consisting of all zeros except one in the index of the correct class. The next step is backpropagation, in which the gradients are computed by taking the derivative of the loss function with respect to the weights and biases. (<ref>) provides the calculation of the gradients of the first hidden layer's weight matrix and bias vector, using the well-known chain rule:

\begin{equation} \label{chain_rule}
\begin{aligned}
\frac{\partial L}{\partial \mathbf{W}^{[1]}} &=
\frac{\partial L}{\partial \hat{\mathbf{y}}}
\frac{\partial \hat{\mathbf{y}}}{\partial \mathbf{z}^{[3]}}
\frac{\partial \mathbf{z}^{[3]}}{\partial \mathbf{a}^{[2]}}
\frac{\partial \mathbf{a}^{[2]}}{\partial \mathbf{z}^{[2]}}
\frac{\partial \mathbf{z}^{[2]}}{\partial \mathbf{a}^{[1]}}
\frac{\partial \mathbf{a}^{[1]}}{\partial \mathbf{z}^{[1]}}
\frac{\partial \mathbf{z}^{[1]}}{\partial \mathbf{W}^{[1]}},
\\[5pt]
\frac{\partial L}{\partial \mathbf{b}^{[1]}} &=
\frac{\partial L}{\partial \hat{\mathbf{y}}}
\frac{\partial \hat{\mathbf{y}}}{\partial \mathbf{z}^{[3]}}
\frac{\partial \mathbf{z}^{[3]}}{\partial \mathbf{a}^{[2]}}
\frac{\partial \mathbf{a}^{[2]}}{\partial \mathbf{z}^{[2]}}
\frac{\partial \mathbf{z}^{[2]}}{\partial \mathbf{a}^{[1]}}
\frac{\partial \mathbf{a}^{[1]}}{\partial \mathbf{z}^{[1]}}
\frac{\partial \mathbf{z}^{[1]}}{\partial \mathbf{b}^{[1]}}.
\end{aligned}
\end{equation}

The final stage of the single training pass of the DNN model performs the update of the model weights and biases. Using the SGD optimization method, the algorithm updates the weight matrix and the bias vector of the $l^{\text{th}}$ layer by changing them in the opposite direction of their gradients by their gradients scaled by a learning constant $\alpha$:

\begin{equation} \label{SGD_update}
\begin{aligned}
\mathbf{W}^{[l]} &:= \mathbf{W}^{[l]} - \alpha \frac{\partial L}{\partial \mathbf{W}^{[l]}}, \\
\mathbf{b}^{[l]} &:= \mathbf{b}^{[l]} - \alpha \frac{\partial L}{\partial \mathbf{b}^{[l]}}.
\end{aligned}
\end{equation}

The second well-known DL architecture is CNN, also known as ConvNets. A CNN architecture presumes similarity between various areas of a 2D feature vector and aims at capturing correlations between these parts, as shown in Fig. <ref>(b) [29], [30], [37]. The first key idea underlying the motivation of the CNN architecture is local connections. In contrast to DNNs, where each hidden unit links with all hidden units of the former and latter layers, CNNs connect a single unit in a convolutional layer to a local patch in the previous layer. Raw input data of a CNN model consists of multiple 2D feature vectors corresponding to channels. For example, a $28 \times 28$ RGB image comprises three 2D feature vectors representing the pixels in the RGB color channels. It is most likely that local regions of an RGB image are correlated, and statistics are comparable, which means a pattern appearing in an area might also come out of another region, such as two ears of a cat in different parts of an image [29], [37]. Therefore, it is logical to utilize these spatial relationships and link a patch of input data to a hidden unit rather than a fully connected structure [37].

The second major characteristic that forms the CNN baseline is weight sharing. Each local patch in the 2D feature vector connects to a distinct hidden unit in the following convolutional layer, where all links use the same weight matrix. The motivation behind weight sharing is to decrease the number of learnable parameters. The example given in [37] describes this motivation perfectly. Let us assume that we have $482 \times 415$ input data with three channels, where the input size is $600090$. Considering the dense connection of DNNs, the parameter size is approximately $600$K for a single hidden unit in the next layer, increasing to more than $600$M for $1000$ hidden units. Many parameters increase the training time substantially, crack the available memory, and necessitate powerful regularizers to prevent overfitting. On the other hand, a $40 \times 40$ shared weight matrix spanning local patches provides $4800$ learnable parameters, allowing $125$ hidden layers with $1000$ units each against a single hidden layer in the dense connection. As a result, the number of parameters and model size reduce significantly along with the enhancement of the generalization capacity, preventing memorizing data rather than learning.

Windowing procedure and the convolution operation of a CNN model.

The primary operation of CNN forward-propagation is convolution, which links local regions to hidden units. Specifically, a CNN model calculates the weighted sum of a patch in the 2D feature vector by the convolution operation to pass the patch's features to a single hidden unit of the next layer. A single patch represents the window of the CNN forward-propagation whose size determines the size of the shared weight matrix, and CNN travels across all windows to move on to the following layer. These windows might, and most probably, intersect, in which the shifting size corresponds to the stride of the current convolutional layer. A simple example of this windowing procedure and the convolution operation is given in Fig. <ref>. In this example, the $5 \times 7$ input data $\mathbf{X}$ and the $2 \times 3$ output data $\mathbf{Y}$ both include a single channel; however, they might differ depending on the model architecture. When there are more channels in the input data, the $3 \times 3$ shared weight spans other channels in the same way as shown in Fig. <ref>. The dimensions of the input and output data may dispute according to window size and stride value. Therefore, amending a padding layer composed of all zeros to the input data equalizes the dimensions, if necessary.

A pooling layer follows the convolutional layer in a typical CNN model architecture, which computes a single statistic, such as the average or the maximum, of the input patches [37]. Pooling layers enable the CNN model to become robust to minor distortions without requiring additional learnable parameters [29], [37]. The stride value of a pooling layer is generally as large as the window size. The initial stages of a CNN model are typically composed of stacked convolution, non-linear activation function, and pooling, followed by further convolutional and dense layers [29]. The loss calculation and backpropagation procedures are the same as that of DNNs, where the algorithm computes the gradients of the weights by taking the derivative of the loss with respect to the weights.

The DL techniques that we have examined so far suffer from a common shortcoming of being memoryless, where each layer's output depends only on its input and the corresponding weights without considering the past. This is where RNNs come into play for applications including sequential data, such as speech recognition and natural language processing, thanks to their ability to utilize previous parts. RNNs process an input sequence with correlations between the elements one sample at a time, and the outputs not just depend on the input and weights but also the past information, as given in Fig. <ref>(c) [29], [30], [38]. The architecture of an RNN might vary according to the application. For example, each layer might yield an output, like in Fig. <ref>(c), for a sequence generation application such as image captioning or a single outcome for a sequence classification such as assessing the sentiment of a product review if it is positive or negative [38]. For a sequential input data $\mathbf{X}$, such as a sentence or a speech, a typical RNN's hidden layer performs the following forward-propagation:

\begin{equation} \label{RNN_forward_propagation}
\begin{aligned}
\mathbf{s}^{(t)} &= \text{tanh}(\mathbf{W}_{x} \mathbf{x}^{(t)} + \mathbf{W}_{s} \mathbf{s}^{(t-1)} + \mathbf{b}_{s}),
\\
\mathbf{h}^{(t)} &= \mathbf{W}_{h} \mathbf{s}^{(t)} + \mathbf{b}_{h}.
\end{aligned}
\end{equation}

Here, $\mathbf{s}^{(t)}$ is the internal state of the model at time $t$, which forms the memory unit by containing the information about the past elements, $\mathbf{W}_{x}$ and $\mathbf{W}_{s}$ are the weights that connect the $t^{\text{th}}$ input sample $\mathbf{x}^{(t)}$ and the previous state $\mathbf{s}^{(t-1)}$ to the current state, respectively, $\mathbf{b}_{s}$ is the bias of $\mathbf{s}^{(t)}$, and $\mathbf{b}_{h}$ is the bias of the layer output $\mathbf{h}^{(t)}$. Similar to the weight sharing procedure in a CNN, the hidden layers of an RNN share the weights and biases, which prevent overfitting and decrease the model complexity [30].

The training procedure of an RNN resembles that of DNNs and CNNs, where the RNN model serves as a multi-layer feed-forward NN with each layer corresponding to a time step, shared weights across these time steps, and single or multiple outputs according to the application. Following the loss calculation, the RNN model is trained via the backpropagation through time (bptt) algorithm, which is the SGD unfolded in time [30], [38]. However, the BPTT algorithm is challenging since it causes gradient vanishing that inhibits information delivery or exploding that paralyzes the training. The reason for this is repeated multiplication with the weight matrix during the backward pass. Researchers have suggested several solutions for this problem, such as initializing weights from a previously trained model, gradient clipping that prevents gradients from exceeding a threshold, and sophisticated optimization techniques like ADAM [38]. As a powerful alternative to these solutions, the Long Short-Term Memory (lstm) networks are designed to address gradient vanishing or exploding issues along with providing longer memorization than a classical RNN by enhancing the network with explicit memory. Let us consider a language model aiming at predicting the upcoming word based on the formers [39]. RNNs perform well in anticipation when the gap between relevant information and the prediction point is narrow:

\begin{align*}
\textit{The }
\underbrace{\textit{\fcolorbox{orange}{table_red}{clouds} are in the \underline{\textcolor{red}{\textbf{sky}}}.}}_{\textbf{{\normalsize \textcolor{table_green}{Small Gap}}}}
\end{align*}

Unfortunately, RNNs can not find the relevant information as the distance increases, and it is possible for the distance becomes considerably large. In such cases, it is beneficial to employ LSTM networks to learn long-term dependencies:

\begin{align*}
\textit{I'm from }
\underbrace{\textit{\fcolorbox{orange}{table_red}{France} and I'm good at speaking fluent \underline{\textcolor{red}{\textbf{French}}}.}}_{\textbf{{\normalsize \textcolor{table_red}{Large Gap}}}}
\end{align*}

According to our comprehensive research, DNNs are by far the most popular DL architectures used in the PHY design of wireless communications. Specifically, DNNs are utilized for signal detection, channel estimation, peak-to-average power ratio (papr) reduction in MC waveform designs, and passive beamforming designs in RIS-based systems. CNN-based models also have an immense role in the literature of DL-based PHY designs. There are use-cases like modulation recognition, signal classification, removing cyclic prefix (cp) in MC waveform designs, and anti-spoofing & anti-jamming in PHY security schemes.
It is also worth mentioning that a CNN does not have to utulize 2D feature vectors.  There are several studies using 1D CNNs in the literature. Particularly, the literature in wireless communication widely benefits from 1D CNNs in numerous tasks such as network management, traffic analysis so on [40],[41], [42].

RNNs find a moderate pace in DL-based PHY methods, with certain applications for MC waveform designs under high mobility and for anti-eavesdropping PHY security schemes. Considering future wireless networks, DNNs, CNNs, and RNNs have the potential to be utilized for modeling multi-attribute mobile data, spatial mobile data analysis, and temporal data modeling, respectively [30]. Once we understand the theoretical background and major applications of the most popular DL architectures, we may now delve into DL programming with Python. Despite presenting an image classification application is not the primary focus of this work, we provide a simple example here to illustrate the basics of DL programming.

§.§ Autoencoders and Generative Adversarial Networks

We also elaborate and present the studies pursuing end-to-end learning schemes in this survey. Since the end-to-end learning concept shows a significant contribution to performance enhancement in wireless communication schemes, several studies have been published using this still maturing autoencoder approach.
An autoencoder is a kind of neural network that is trained to pass its input to the output the same. The network can be divided into two components:

\begin{equation} \label{encoder}
\begin{aligned}
\mathbf{h}=\mathbf{f(x),}
\end{aligned}
\end{equation}

\begin{equation} \label{decoder}
\begin{aligned}
\mathbf{\hat{x}}=\mathbf{g(h).}
\end{aligned}
\end{equation}

In (<ref>), f(x) function defines the encoding process where x is the input data to be converted to h code. Also, g(h) function in (<ref>) defines the decoding process converts this h code to the output $\mathbf{\hat{x}}$ with the g function. The simple architecture of an autoencoder is shown in Fig.  <ref>(a). An autoencoder is not particularly effective if it simply learns the set $\mathbf{g(f(x))=x}$ all the time. Conversely, autoencoders are made so that they cannot be perfect at copying. They are typically regulated so that they can only duplicate input that closely resembles the training data and only with some degree of accuracy. The model frequently learns beneficial features of the data because it must decide which aspects of the input should be duplicated. The concept of an encoder and a decoder has been expanded by modern autoencoders to include stochastic mappings in addition to deterministic functions.

An autoencoder scheme (a) mapping  the input x to the output x where function f and g  stands for encoder and decoder, respectively (b) from wireless communication perspective

Through a sequence of effective parametric linear algebra operations, autoencoders in wireless communication, as shown in Fig.  <ref>(b), enable the physical layer transmitter and receiver to take the form of essentially unconstrained mappings. By attempting to reduce BER corresponding reconstruction loss, deep learning allows us to arrive at a solution to the whole communications system design challenge. This approach is capable of being applied over a wide range of channels and impairment models to provide better-suited solutions. Tailor-made waveforms may accomplish innovative and unparagoned performance under challenging channel conditions by learning physical layer information (encoder, decoder) and representation solutions in this end-to-end manner.

A generative adversarial network (GAN) resembles an inside-out autoencoder in appearance. Yet, unlike autoencoders compressing input data, GANs convert their low-dimensional input into high-dimensional data at the inner network. Using two neural networks in competition with one another (adversarial), GANs are computational structures that produce new, artificial instances of data that can be misinterpreted for genuine data. They are extensively utilized in the creation of images, videos, and voices.
The neural network named "Generator" creates new data instances while they are authenticated by another neural network named "Discriminator" as shown in Fig. <ref>. Each sample of data that is reviewed by the discriminator is evaluated to determine if it is a part of the training dataset or not.
The primary distinction between Autoencoders and GANs is the way that learn. Autoencoders may be thought of as addressing a semi-supervised learning issue since they aim to reproduce a picture with the least amount of loss possible. In other respects, GAN handles problems involving unsupervised learning. The training period for the two approaches could be treated as the most significant distinction. The required training time for GAN is longer; nevertheless, autoencoders provide less credible results (such as decreased image quality). GANs were thus considered and found to be far more stable.

Labels and descriptions of the categories in the CIFAR10 dataset.

table_white $0$
table_white Airplane
table_white $5$
table_white Dog

table_white $1$
table_white Automobile
table_white $6$
table_white Frog

table_white $2$
table_white Bird
table_white $7$
table_white Horse

table_white $3$
table_white Cat
table_white $8$
table_white Ship

table_white $4$
table_white Deer
table_white $9$
table_white Truck

§.§ Programming for DL: A Simple Classification Example

We have analyzed the theoretical foundations of DNNs, CNNs, and RNNs so far. The programming stage of a DL application is as important as building the DL-based system since the feasibility of the considered method requires thorough examination via extensive computer simulations. Thus, we will provide the coding stage of a basic CNN architecture using Python and Keras [43]. We will build a CNN-based image classifier model with training using the CIFAR10 dataset [27], [28]. CIFAR10 consists of $60000$ $(32 \times 32)$ RGB images including $10$ categories as given in Table <ref>. $50000$ images correspond to the training data while the rest $10000$ images construct the test data. We will begin with calling the necessary modules:

A generative adversarial network scheme.

from tensorflow.keras.preprocessing.image import img_to_array
from tensorflow.keras.datasets.cifar10 import load_data
from tensorflow.keras.layers import Conv2D, MaxPooling2D
from tensorflow.keras.layers import Dense, Flatten
from tensorflow.keras.models import Sequential
from PIL import Image
import numpy as np

Here, the img_to_array module converts an image to the corresponding 3D pixel array, load_data module fetches the CIFAR10 dataset, Conv2D, MaxPooling2D, Dense, and Flatten represent the convolutional and pooling layers of a CNN, dense layer of a DNN, and flattening layer which squashes its input, respectively. The Sequential module creates a DL model by adding the layers consecutively. The Image module is useful to process an image and numpy is the well-known scientific computing module. Now, we can import the CIFAR10 data and preprocess it by normalizing each pixel to (0-255) range:

(x_train, y_train), (x_test, y_test) = load_data()
x_train = x_train.astype("float32") / 255
x_test = x_test.astype("float32") / 255

The astype function converts data to "float32" data type. The first ten images in the training data and their labels with corresponding descriptions are given in Fig. <ref>. After preprocessing and manually observing the data, we can now build the CNN model. We use a simple architecture to keep the model complexity and training time moderate, which can be created as follows:

First ten images in the CIFAR10 training data with their labels and corresponding descriptions.

model = Sequential([
Conv2D(32, 3, input_shape=(32, 32, 3), activation="relu"),
Conv2D(32, 3, activation="relu"),
MaxPooling2D(2, 2),
Conv2D(64, 3, activation="relu"),
Conv2D(64, 3, activation="relu"),
MaxPooling2D(2, 2),
Dense(64, activation="relu"),
Dense(10, activation="softmax")
model.compile(loss="sparse_categorical_crossentropy", optimizer="adam", metrics=["accuracy"])

We use sparse_categorical_crossentropy loss function to calculate the loss value which performs exactly the same as the typical categorical cross-entropy loss function given in (<ref>). However, in this case the ground truth label is a single digit from 0 to 9 representing the correct label instead of a one-hot vector. Subsequently, we train our CNN model to observe train/test loss and accuracy values:

history = model.fit(x_train, y_train, validation_split=0.1, batch_size=128, epochs=50)

Here, the validation_split represents the percentage of the train data to be utilized as validation data to observe the loss and accuracy changes of the unseen data during the training. The batch_size is the number of images that train the model in a single iteration, where a single epoch is composed of training all batches. The history variable includes the loss and accuracy values corresponding to the train and validation data for each epoch, which is shown in Fig. <ref>. The graph indicates that the train loss and accuracy decrease and increase smoothly, where the train accuracy converges to nearly $97\%$. Therefore there is almost no bias and the model does not underfit. However, the validation loss rises after almost the tenth iteration and the validation accuracy converges to approximately $72\%$. Thus, there is a considerable variance and the model strongly overfits to the training data. This can also be observed by evaluating the trained model on the test data:

The loss and accuracy graphs of the train and validation data.

_, accuracy = model.evaluate(x_test, y_test)

$\gg$ Accuracy: 71.10

As a result, this model requires a hyperparameter tuning to improve its generalization ability, which is out of our scope. As the last operation, we can try explicit images to predict their categories:

image = Image.open('/../image')
image = image.resize((32, 32))
img_array = img_to_array(image)
img_array = img_array.reshape(-1, 32, 32, 3)
c = np.argmax(model.predict(img_array))

Here, we first import the image and resize it to make it suitable for our model. Thereafter, we convert the image to the corresponding 3D pixel array. Since we feed the model with a train data including 50000 instances, the input of the model has the dimension of (N, 32, 32, 3). Thus, we reshape the 3D pixel array to have 1 as the first dimension and predict the class of the image by taking the maximum index of the output array. Fig. <ref> shows the predictions of our model on four explicit images that the model has never seen before. The model correctly recognizes the cat and the truck, while it classifies the given dog and frog as a truck and a ship, which supports the low validation accuracy in Fig. <ref>.

We have covered the basics of DL architectures, introduced an overview of the three mostly used DL techniques: DNN, CNN, and RNN, and provided the programming stage of an image classification task. In the following, we will begin our investigations of DL applications on PHY technologies with MIMO systems.

Predictions of the trained CNN model on the unseen images.

§ DL FOR EMERGING MIMO SYSTEMS

Next-generation wireless communication technologies are expected to bring forth a variety of improvements such as extremely high data rate and spectral efficiency. The brand new mobile communication technology, 5G, has attracted the attention of both academic and industrial circles with the promise of a maximum of 20 Gb/s peak data rate along with other major improvements in latency and energy/spectrum efficiency. The key enabler of a fully intelligent world, 6G, will introduce even higher specs such as a peak data rate of at least 1 Tb/s and a user-experienced data rate of 1 Gb/s [5]. These significant improvements are possible with the use of multiple antennas at both transmitter and receiver sides as in early standards, which is known as the MIMO technology. MIMO systems can provide spatial diversity gain where the transmitted symbols are received from multiple paths and the fading effect is mitigated. Alternatively, spatial multiplexing gain is obtained by MIMO systems where the capacity of the channel increases linearly with the minimum of transmit and receive antennas [44]. Scaling up the conventional MIMO systems with a few hundred antennas that simultaneously serve many users, massive multi-user (mu) MIMO technology has become a breakthrough. Massive MU-MIMO utilizes all the benefits of the conventional MIMO while eliminating the MU interference to further enhance the transmission performance, which can be seen in Fig. <ref>. Massive MIMO provides 10 times or more capacity compared to the conventional MIMO via its powerful spatial multiplexing ability. In addition, it enhances the energy efficiency on the order of 100 times via the ability of sharp beamforming to the small regions with a large number of antennas [45].

An example massive MU-MIMO scenario with a large antenna array supporting multiple users.

Although massive MIMO technology brings immense benefits, it comes at some prices. The most shining feature of massive MIMO, spatial multiplexing, requires reliable channel state information (csi). During the downlink transmission of conventional MIMO systems such as in the Long Term Evolution (4G) standard, the base station (bs) transmits a pilot sequence so that the users estimate the channel responses and feed them back to the BS. Nevertheless, this method is not suitable for massive MIMO systems since each user would have to estimate a few hundred channel responses which gives rise to a huge feedback overhead. Another reason is that pilots should be orthogonal between the antennas and this requires a considerable amount of orthogonal resources [45]. Another challenge of massive MIMO technology is the number of required radio frequency (rf) chains. When each transmit antenna at the BS is activated to transmit a different symbol, the number of required RF chains equals the number of transmit antennas and this increases the hardware cost. As a promising solution to the inherent problems of MIMO mentioned above, IM appears as a strong candidate for next-generation wireless communication technologies. Along with decreasing the need for the massive number of RF chain requirements, IM techniques provide high energy and spectral efficiency by conveying additional information bits utilizing the indices of the building blocks of a communication system [12]. One of the most critical shortcomings of IM techniques is the detection complexity. The more bits transmitted via IM, the more complex the detector becomes. The optimum detector is the maximum likelihood detector (mld) and in some cases, it is impossible to use it because of its unacceptable level of complexity. At this point, linear detectors, such as zero-forcing (zf) and minimum-mean squared error (mmse) detectors, take place with the cost of a worse bit error rate (ber). Thus, there is a trade-off between the complexity and the BER performance of currently used model-based detectors.

DL applications for the transmitter and receiver sides of a MIMO system.

Preprocessing of transmitting signals, often known as precoding, is another vital topic in efficient massive MU-MIMO systems. The BS uses precoding before transmission to mitigate the fading channel impact and cancel MU interference. However, precoding techniques that currently exist in the standards require high-resolution digital-to-analog converters (dacs) which bring high complexity to the system. Therefore, the design of precoders with low-resolution DACs is of interest among the issues of intelligent massive MIMO transmitter design. Within the context of intelligent MIMO transmitters, another blooming topic is constellation design since the performance of communication systems is highly affected by the shape of the constellation. From their infancy to the cutting-edge technology, information bits are modulated by a limited number of constellations such as quadrature amplitude modulation (qam) and phase-shift keying (psk). Next-generation intelligent MIMO transmitters are expected to modulate information bits by using complex-shaped constellations that are optimized for the transmission environments. It is the mission of the intelligent black box to find the best locations for the constellation points and to optimize the decision boundaries for these points [46]. However, to obtain a performance improvement concerning the conventional constellation designs, the receiver must be aware of the decision boundaries of the used complex-shaped constellations. Therefore, it is mission impossible for the conventional detectors to decode information bits without the initial knowledge of the constellation design. This is where E2E learning appears as one of the remarkably charming concepts for the next-generation wireless communication systems, where the aim is to fully optimize both the transmitter and receiver by representing the whole system as an autoencoder (ae) [47].

DL techniques gleam as a bright solution to those aforementioned drawbacks and further needs of next-generation massive MU-MIMO systems thanks to their ability to learn a suitable mapping from input to output. Model-based blocks of a conventional communication system that depend on optimized mathematical models might be replaced by intelligent data-based blocks that are robust to dynamic changes of the transmission environment. The ultimate goal is to achieve E2E intelligent communication systems, which will, in our opinion, revolutionize next-generation technology. In Fig. <ref>, the applications of DL techniques for MIMO systems are given from a high-level perspective with a categorization under transmitter and receiver parts. The following subsections will shed light on the state-of-the-art research in the literature of DL applications for MIMO systems as well as future directions and open issues in this area. Specifically, in Subsection III.A, we will demonstrate the use of DL in detection for MIMO systems. Subsection III.B investigates DL-based MIMO channel estimation schemes. By combining DL-aided MIMO detectors and channel estimators, we discuss intelligent MIMO receivers and introduce the real-world datasets that might be a benchmark for future studies in Subsection III.C. Subsection III.D describes the intelligent MIMO transmitter designs and AEs. DL-based cutting-edge IM technologies are described in Subsection III.E. Finally, Subsection III.F demonstrates an example implementation of a DL-based IM detector for interested readers.

§.§ Signal Detection for MIMO Systems

The primary motivation behind using DL techniques in MIMO detectors is to overcome the trade-off between computational complexity and BER performance. The MLD operates depending on a strong mathematical model and it is proved that MLD provides the optimum BER performance under ideal conditions [48, 49]. Since DL techniques, especially DNNs, are general function approximators, the model of the MLD can be represented by NNs with significantly low complexity. These approximations are not expected to outperform MLD. Instead, it is expected to reach the performance of MLD without requiring the impractical complexity of MLD. In what follows, we reveal and summarize a brief overview of the literature in this direction.

The authors of [50] and [51] study a DL-based detector called DetNet for a MIMO system. The structure of DetNet is slightly different than a well-known DNN, where it is designed based on the iterative projected gradient descent algorithm. The BER performance of the DetNet is compared to the state-of-the-art detectors, approximate message passing (amp), and semidefinite relaxation (sdrx) under the assumption of perfect CSI at the receiver. As a similar approach to [50] and [51], the study in [52] aims to create a model-driven DL-based detector for a MIMO system based on the existing iterative detection algorithm, orthogonal AMP (oamp). The proposed network, OAMP-Net, wields some additional trainable parameters to improve the BER performance of the OAMP detector under perfect CSI.

Another member of model-driven DL-based networks appears in [53], where the authors combine model-based and data-based detectors to create a partial learning scheme. An NN detects a portion of the amplitude-phase modulation (apm) symbols in a MIMO system as the non-linear data-based part of the detector. The rest of the APM symbols are detected by a conventional ZF detector as the linear model-based part of the detector. This scheme also requires perfect CSI at the receiver.

In contrast to the former solutions, a pure data-driven DNN architecture is proposed to detect symbols transmitted over a multipath fading channel with bandwidth constraints in [54]. Both flat fading and frequency-selective fading channels are implemented to analyze the BER performance of the detector. For a range of pilot overhead percentages in the transmitted signal, the proposed detector is compared to the theoretical BER of the Rayleigh fading channel and the decision feedback equalizer without requiring CSI at the receiver.

As discussed earlier, DL-based detectors are expected to decrease the complexity in MIMO system models with IM techniques. Within this context, data-driven and model-driven DNNs are investigated in [48] and [49], respectively. Instead of a single giant DNN, the work in [48] suggests utilizing data-driven sub-DNNs for the detection process of a MIMO system with generalized spatial modulation (gsm). The proposed architecture performs similarly to the MLD and the single DNN detector with lower complexity under the assumption that the fading channel is fixed and the noise is independent and identically distributed (iid) Gaussian. However, when the noise is correlated or has a deviation in its distribution, the sub-DNN architecture outperforms the MLD and the single DNN detector. CSI at the receiver is not necessary for fixed fading channels, but it is essential for time-varying channels. On the other hand, in [49], the authors propose a model-driven DL-based block detector for GSM inspired by linear block detectors such as Block-ZF and Block-MMSE detectors. The APM symbol of each active transmit antenna is detected by a DNN, assuming that perfect CSI is available at the receiver. The B-DNN model outperforms the linear detectors and performs similarly to the MLD with lower complexity.

The treatises examined so far follow the fundamental principle of DNNs, which is stacking the hidden layers to build a deep network and to increase the learning ability. However, the authors in [55] realize that deepening the network after a certain number of hidden layers does not considerably improve the BER performance. Therefore, they propose a parallel detection network, where several DL detection networks perform in parallel to introduce a diversity effect. In order to obtain diversity, a special loss function is designed. Detection is carried out based on the perfect knowledge of the CSI. Likewise, in [56], a DL-based detector composed of many parallel modified DNNs introduces a diversity gain in a MIMO system model. The modified DNNs employ the residual learning principle and differ in several ways and introduce diversity to the system. Assuming perfect CSI, computer simulations unveil that the proposed detector outperforms the ZF, MMSE, and single DNN-based detectors.

The kind of NN employed in DL-based detectors is another key factor that draws attention. Within this perspective, the work in [57] compares DNN- and CNN-based detectors to address the issues of robust detection with imperfect CSI and efficient DL framework for low-complex detection. This comparison reveals that the DNN-based detector performs significantly better in terms of the BER performance. In addition, it outperforms the ZF, MMSE, and DetNet [50], [51] detectors with both perfect and imperfect CSI at the receiver. However, the MLD detector has a considerably better BER performance than the proposed detector. With a similar approach, DNN-, CNN-, and RNN-based detectors are designed without channel dependency and compared to the MLD in [58]. Computer simulations in this study show that the RNN-based approach performs the best, whereas the DNN-based approach is the poorest. The fact that DNN-based architecture outperforms CNN-based architecture in [57] but degrades in [58] demonstrates that the optimum DL architecture is dependent on the system model and environment.

So far, the suggested detectors have relied on channel assumptions without considering real-world conditions. However, according to [59] and [60], the main issue with the existing DL-based detectors is either unpleasant performance over practical spatially correlated channels or a computational burden because of retraining for each channel realization. Thus, the authors of [59] focus on a robust DL-based MIMO detector that can optimize itself via online training during the transmission over realistic and spatially correlated channel models. The proposed detector, called MMNet, is built upon iterative soft-thresholding algorithms. Experiments conducted by using a dataset of channel realizations from the 3GPP 3D MIMO channel indicate that MMNet outperforms the classical approaches such as AMP and SDRX,  as well as recent DL techniques like DetNet [50], [51], and OAMPNet [52], under the assumption of perfect CSI. The work in [60] introduces an additional NN, a HyperNetwork, that eliminates the need for online retraining on top of a modified version of MMNet to create the HyperMIMO detector. Computer simulation results reveal that the HyperMIMO detector outperforms MMSE and OAMPNet [52] and performs similarly to the MMNet and the MLD with less computing cost under the perfect CSI.

An overview of the literature on DL-based MIMO detectors considering five key factors (Red hues indicate CSI requirement or without real-world dataset and green hues indicate no CSI requirement or with real-world dataset).

table_blue Paper
table_blue CSI Requirement
table_blue DL Structure
table_blue Real-World Dataset
table_blue Benchmarks
table_blue Data- or Model-Driven

table_white Deep MIMO Detection [50]
table_white DetNet
table_white AMP, SDRX
table_white Model

table_white Learning to Detect [51]
table_white DetNet
table_white AMP, SDRX
table_white Model

table_white Model-Driven MIMO Detection [52]
table_white OAMP-Net
table_white OAMP
table_white Model

table_white Partial Learning [53]
table_white DNN
table_white ZF, MLD, Single DNN
table_white Model

table_white Learning for Detection [54]
table_white DNN
table_white Decision Feedback Equalizer, Theoretical Rayleigh
table_white Data

table_white Sub-DNNs for GSM [48]
table_white Flexible
table_white Sub-DNNs
table_white MLD, Single DNN
table_white Data

table_white B-DNN [49]
table_white Block-DNN
table_white B-ZF, B-MMSE, MLD
table_white Model

table_white Parallel DL [55]
table_white PDN
table_white SPD, DetNet
table_white Data

table_white DL-Based Parallel Detector [56]
table_white Parallel Modified DNN
table_white ZF, MMSE, Single DNN
table_white Data

table_white Efficient MIMO Detection [57]
table_white DNN, CNN
table_white ZF, MMSE, MLD, DetNet
table_white Data

table_white Implementation Methodologies [58]
table_white DNN, CNN, RNN
table_white MLD
table_white Data

table_white Adaptive Neural Detection [59]
table_white MMNet
table_white MMSE, MLD, AMP, SDRX, vblast, DetNet, OAMPNet
table_white Data

table_white HyperMIMO [60]
table_white Modified MMNet
table_white MMSE, MLD, OAMPNet, MMNet
table_white Data

table_white Model-Driven Massive MIMO [61]
table_white DNN
table_white Iterative [62], MMSE, DetNet
table_white Model

table_white DL-Based Viterbi [63]
table_white ViterbiNet
table_white Viterbi, Sliding Bidirectional RNN
table_white Data

table_white DL-Based Sphere Decoding [64]
table_white DNN
table_white SPD Variants
table_white Data

table_white DL-Aided Tabu Search [65]
table_white FS-Net, DL-Aided TS
table_white ZF, MMSE, SPD, Ordered SIC, DetNet
table_white Data

table_white RE-MIMO [66]
table_white RE-MIMO
table_white MMSE, MLD, AMP, SDRX, V-BLAST, DetNet, OAMPNet, OAMPNet-2
table_white Data

The development of DL-based detectors by recreating mathematical models using DNNs is popular among researchers. [50], [51], and [52] are examples with model-driven DL architectures. In addition, the work in [61] presents a DNN-based detector for massive MIMO systems. The DNN architecture is model-driven since it depends on an existing iterative detection method given in [62]. There are additional parameters to be learned for MU interference cancellation. The suggested detector outperforms the iterative [62], DetNet [50], [51], and MMSE detectors, assuming perfect CSI at the receiver. However, there are also data-driven architectures utilizing the same concept. The authors of [63] offer a MIMO detector dubbed ViterbiNet that eliminates the CSI reliance by integrating DNN architecture into the Viterbi algorithm. Their simulation results indicate that the ViterbiNet detector has a superior performance compared to the conventional Viterbi decoder. Another classical MIMO detection approach, sphere decoding (spd), meets with DNNs in the study of [64]. The proposed method learns the radius of the decoding hypersphere, outperforming previous SPD versions and coming close to the MLD with much less complexity assuming the perfect CSI. The study in [65] explores the use of DL within tabu search (ts) detection. Two models are proposed: FS-Net and DL-Aided TS, with the latter building on the former. Assuming perfect CSI is available at the receiver, the proposed DL-Aided TS method decreases complexity by roughly 90% compared to existing TS algorithms while keeping almost the same performance.

As opposed to the literature inspected thus far, the authors of [66] consider the MU case and present an NN-based MU-MIMO detector called RE-MIMO. They are concerned with resilience against channel misspecification, the capacity to manage a varying number of users, and invariance to the sequence in which users interact with the system. The proposed model-driven DL detector depends on the neural augmentation strategy, which combines the learning procedure with the inductive biases.

We classify the investigated DL-based detectors by five key specifications in Table <ref>. It should be noted that red and green hues in the CSI column indicate CSI requirement and no CSI requirement, while they represent computer simulations without and with real-world dataset in the Real-World Dataset column, respectively. There are two noteworthy outcomes deduced from Table <ref>. The first is that, to function as intended, most system models require the perfect CSI. The second issue is that most DL-based detectors have never been tested in a real-world setting. In the follow-up, we will scrutinize the literature of DL-based channel estimators in MIMO systems.

§.§ Channel Estimation for MIMO Systems

The fundamental rationale for employing DL methods in massive MIMO channel estimation is to enable reliable channel estimation with considerably fewer pilots than the number of transmit antennas and low-resolution ADCs at the BS. This leads to a robust detection process with a reasonable amount of resources for channel estimation.

A DL-based channel estimator for downlink massive MIMO systems is presented in [67], where the pilot length is less than the number of transmit antennas at the BS. The proposed channel estimator consists of two stages, the first of which involves two DNNs working together to perform pilot design and pilot-aided channel estimation. In the second stage, the channel estimation and symbol detection are implemented iteratively by another DNN to enhance the performance of the estimation, which is called data-aided channel estimation. Computer simulations reveal that the two-stage estimator outperforms the conventional MMSE-based data-aided channel estimator given in [68]. The authors of [69] extend this study to uplink MU-MIMO systems. The recommended channel estimator, unlike [67], skips the data-aided estimation stage. Separate DNNs design the pilot signals for the users at the initial part of the pilot-aided estimation stage. Subsequently, separate DNNs estimate the channels of the users using successive interference cancellation (sic). The suggested estimator outperforms both the traditional MMSE estimator and the DL approach without SIC. In addition, both [67] and [69] investigates the optimal pilot length to obtain higher capacity.

Channel estimation in mmWave massive MIMO systems is complicated since the number of RF chains is limited. The authors of [70] propose a learned denoising-based AMP (ldamp) network to solve this problem. The LDAMP network is an image recovery network that treats the channel matrix as a 2D image, with a single pilot for all channels in the antenna array. The LDAMP network outperforms the conventional AMP variants. The study in [71] focuses on DNN-based direction-of-arrival (doa) estimation in addition to DNN-based channel estimation. Specifically, two DNN-based algorithms operate DOA and channel estimation processes without the need for pilot symbols. Extensive computer simulations verify that the suggested DOA and channel estimation algorithms exceed the traditional techniques in terms of mean squared error (mse) of DOA and BER performance. Motivated by the drawbacks of [70], the authors of [72] develop a better denoising network based on CNNs along with the LDAMP network, resulting in the fully convolutional denoising AMP algorithm. The required pilot length is less than the number of channel components to be estimated. The fully convolutional denoising AMP algorithm has a successive performance over LDAMP [70] and other variants of AMP in terms of both normalized MSE (nmse) and achievable sum rate. Another channel estimator built upon the classical learned AMP (lamp) method is presented in [73], which is prior-aided Gaussian mixture LAMP. In the first stage of the Gaussian mixture LAMP algorithm, a new shrinkage function is developed based on the Gaussian mixture distribution of beamspace channel elements. Then, the Gaussian mixture LAMP-based beamspace channel estimation is performed depending on the derived shrinkage function in the second stage. The proposed algorithm outperforms the OMP, AMP, and LAMP algorithms in terms of NMSE. The methods suggested in [70], [72], and [73] are simulated using the well-known Saleh-Valenzuela channel model. The study [73] also ran further computer simulations with the practical ray-tracing channel dataset provided by [74] and [75].

Along with less pilot-aided channel estimation, DL-based estimators pave the way for reliable prediction even with low-resolution analog-to-digital converters (adcs). In [76], a massive MIMO BS equipped with mixed-resolution ADCs implements channel estimation with DNNs. A direct-input DNN estimates channels using the received signals of all antennas, whereas a selective-input prediction DNN estimates channels of the antennas with low-resolution ADCs via received signals of the antennas with high-resolution ADCs. The proposed algorithm has a superior performance over the classical MMSE and the expectation-maximization Gaussian-mixture generalized AMP methods. The idea of using mixed-resolution ADCs is improved in [77], where the authors propose an optimization technique for mixed-resolution ADCs allocation along with the channel estimation and pilot design. Specifically, the CENet estimates channels, the SELNet assigns one-bit and high-resolution ADCs, and the PDNet constructs the pilot matrix. The suggested framework outperforms the conventional generalized AMP (gamp) and gridless GAMP in terms of NMSE. Besides, the complete framework is compared with the partial scenarios in which all antennas at the BS have one-bit ADCs, and only the CENet exists. Another partial scheme includes all one-bit ADC antennas with both PDNet and CENet. The partial design with non-optimized high-resolution ADCs, PDNet, and CENet performs similarly to the complete framework.

Using DL methods, it is feasible to reduce ADC resolution even further, in which accurate channel estimation is possible despite complete one-bit ADCs. The study in [78] presents a DNN-based channel estimator for one-bit massive MIMO systems. The pilot signal is divided into segments in the first stage, where each one forms the input of a DNN. The estimated channel is the average of all DNNs' outputs. Next, detected data symbols using the initial estimation are interpreted as the remaining pilot signals and utilized to update the initial channel estimation, similar to the procedure in [67]. The proposed algorithm surpasses the traditional LS and Bussgang linear MMSE algorithms in terms of MSE under both i.i.d. and spatially correlated channels. Similarly, the work in [79] proposes a DNN-based channel estimator for one-bit massive MIMO systems. The authors perform investigations to determine the length and architecture of the pilot sequence required to ensure that the quantized received signals are mapped to the channel matrix. According to these analyses, the more BS antennas, the better the channel estimation performance for the same pilot length. Equivalently, the more BS antennas, the less required pilot length for the same channel estimation performance. The suggested network outclasses the GAMP algorithm under the indoor massive MIMO channel scenario offered by [74] and [75]. As opposed to the former one-bit channel estimators, [80] introduces a conditional generative adversarial network (gan) that can learn an adaptive loss function to optimize the training procedure and predict the channels. The proposed conditional GAN algorithm is compared with expectation maximization Gaussian-mixture GAMP algorithm, two-stage algorithm [67], LDAMP, channel and DOA estimation algorithm [71], and one-bit channel estimator [78] using the ray-tracing channel dataset given by [74] and [75]. Computer simulations reveal that the conditional GAN algorithm has a more successive performance than the benchmarks in terms of average NMSE.

Except for [67], all of the channel estimators we have looked at thus far have only considered uplink scenarios. However, downlink channel estimation is also crucial in terms of reliable symbol detection at the user end. It is common to assume that the downlink and uplink channels are reciprocal, and the BS feedbacks the estimated uplink channels to the users. Nevertheless, the detection performance degrades when the channel is not reciprocal. Thus, the authors of [81] introduce a channel calibration network, called CaliNet, based on DNNs to estimate downlink channels accurately from the uplink channels when there is no reciprocality. Computer simulation results indicate that CaliNet performs better than the Argos method, and similar to the Cramér-Rao Bound.

In addition to the prior DL-based channel estimators, the study in [82] investigates reinforcement learning in channel estimation of industrial wireless networks. The proposed algorithm is double deep Q-learning, where the pilot length equals the number of channel components to be estimated. The recommended approach beats the estimated MMSE in an indoor industrial manufacturing setting with 4.7 dB path-loss, 8.6 dB shadowing, and 29 dB obstacles. However, the ideal MMSE surpasses the suggested technique in the same scenario. Similar to how [82] examines a realistic indoor environment, [83] suggests an RNN-based channel estimator for massive MIMO systems in the non-line-of-sight 5G Quadriga 2.0 channel model [84], [85]. The algorithm iteratively searches the channel taps and denoises their amplitudes in the time domain. Considering frame error rate, simulation results for a user speed of 5 km/h show that the RNN-based iterative channel estimator outperforms the traditional MMSE technique.

Considering seven key factors, we categorize the reviewed DL-based channel estimators in Table <ref>. The red and green colors in the Pilot Design column represent if pilot design exists in the study, while they indicate system model with and without low-resolution ADCs in the last column. FCDAMP, EM-GM-GAMP, and GM-LAMP represent fully convolutional denoising AMP, expectation maximization Gaussian-mixture GAMP, and Gaussian-mixture learned AMP, respectively. The most eye-catching observation from Table <ref> is that almost all of the DL-based estimators work for uplink massive MIMO scenarios. It is a favorable consequence that the majority of the approaches fulfill the motivation of using fewer pilots for DL-based channel estimators. It is also encouraging to note that realistic channel scenarios are becoming increasingly common, albeit still insufficiently so, within the framework of DL-based channel estimators. The next subsection combines the DL-based detectors and channel estimators to build an intelligent receiver framework, and provides an insight to the real-world datasets.

An overview of the literature on DL-based MIMO channel estimators considering seven key factors (Red hues indicate no pilot design, no low-resolution ADCs or without real-world dataset and green hues indicate pilot design, low-resolution ADCs or with real-world dataset).

table_blue Paper
table_blue DL Structure
table_blue Real-World Dataset
table_blue Pilot Length
table_blue Benchmarks
table_blue Downlink or Uplink
table_blue Pilot Design
table_blue Low-Resolution ADCs

table_white DL-Based Two-Stage [67]
table_white DNN
table_green Low
table_white Data-Aided MMSE [68]
table_white Downlink

table_white Joint Pilot Design & Channel Estimation for MU [69]
table_white DNN
table_green Low
table_white MMSE
table_white Uplink

table_white DL-Based Beamspace mmWave Massive MIMO (2018) [70]
table_white LDAMP
table_green Low
table_white AMP Variants
table_white Uplink

table_white Super Resolution Channel & DOA Estimation [71]
table_white DNN
table_green No Pilot
table_white Various Conventional Compressed Sensing Methods
table_white Uplink

table_white DL-Based Beamspace mmWave Massive MIMO (2020) [72]
table_white FCDAMP
table_green Low
table_white LDAMP, AMP Variants
table_white Uplink

table_white DL-Based Beamspace mmWave Massive MIMO (2021) [73]
table_white GM-LAMP
table_green Low
table_white AMP, LAMP
table_white Uplink

table_white Mixed-Resolution ADCs [76]
table_white DNN, CNN
table_green Low
table_white MMSE, EM-GM-GAMP
table_white Uplink
table_green Mixed

table_white Joint Channel Estimation & Mixed ADCs Allocation [77]
table_white PDNet, SELNet, CENet
table_white Varying
table_white GAMP, Gridless GAMP
table_white Uplink
table_green Mixed

table_white Segment Average Based Channel Estimation [78]
table_white DNN
table_red High
table_white LS, Bussgang Linear MMSE
table_white Uplink
table_green 1-Bit

table_white DL for 1-Bit ADCs & Fewer Pilots [79]
table_white DNN
table_green Low
table_white GAMP
table_white Uplink
table_green 1-Bit

table_white 1-Bit MU Conditional GAN [80]
table_white conditional GAN
table_green Low
table_white EM-GM-GAMP, [67], LDAMP, [71], [78]
table_white Uplink
table_green 1-Bit

table_white UL-DL Channel Calibration [81]
table_white CaliNet (DNN)
table_green Low
table_white Argos, Cramér-Rao Bound
table_white Both

table_white Double Deep Q Learning [82]
table_white DDQL
table_red High
table_white MMSE
table_white No Info

table_white Iterative RNN [83]
table_white RNN
table_red No Info
table_white MMSE
table_white No Info

§.§ Combined Signal Detection and Channel Estimation & Real-World Datasets

After examining DL-based detectors and channel estimators, we can now integrate them in a single algorithm to get a completely intelligent receiver architecture, which will lead us to our ultimate objective of E2E intelligent communication systems. In [86], a single DNN jointly executes channel estimation and symbol detection for a MIMO-Alamouti system, a unique case of space-time block codes with two transmit antennas. The four-layer DNN takes the real and imaginary parts of the received signal as the input and generates the probabilities of the possible transmit symbol combinations, without explicitly estimating the channel. The recommended network has 1 dB signal-to-noise ratio (snr) loss compared to MLD detection with perfect CSI and 3 dB SNR gain compared to MLD detection with imperfect CSI, at the same BER.

The authors of [87] direct their research towards intelligent receiver for spatial modulation (sm) considering time-variant dynamic channels. The suggested network, called DeepSM, consists of two parallel DNNs, with the upper DNN updating the CSI and the lower one detecting the transmitted symbols at each time slot. The initial CSI is computed using the LS technique utilizing a pilot signal with a length equal to the number of transmit antennas. DeepSM is compared to the conventional model-based and data-driven DNN-based receivers over both time-variant and time-invariant channel models. Both benchmarks implement LS channel estimation and assume that the channel is invariant during the whole simulation. Subsequently, the conventional model-based receiver employs MLD detection, whereas the data-driven DNN-based receiver detects the transmitted symbols by a DNN. Extensive computer simulations reveal that DeepSM outperforms the benchmark schemes in all scenarios. Similar to the iterative architecture of DeepSM, the work in [88] proposes a model-driven DL-based joint channel estimation and symbol detection network, OAMP-Net2, by unfolding the existing iterative algorithm OAMP. OAMP-Net2 performs initial pilot-based channel estimation and detects information symbols afterward. Following that, data-aided channel estimation and symbol detection processes take place iteratively. The recommended approach outperforms the AMP, OAMP, MMSE, MMSE-SIC, DetNet [50], [51], and OAMP-Net [52] in different MIMO setups and modulation orders. MLD and SPD detectors, on the other hand, outclass OAMP-Net2 with their significantly larger complexity. In addition, OAMP-Net2 is specifically compared with OAMP-Net [52] in various MIMO setups and modulation orders considering both i.i.d. and correlated channel scenarios, in which OAMP-Net2 beats the predecessor in all situations. Furthermore, the authors examine OAMP-Net2 under the practical Quadriga 3GPP 3D MIMO channel model [84], [85].

The authors of [89] propose two DL-based receiver structures for uplink MU-MIMO systems. The first technique, FullCon, is a data-driven DNN-based algorithm and detects the information bits by the received signal directly without explicitly executing channel estimation. MdNet, on the other hand, separates the channel estimation and symbol detection phases. DNN-based channel estimation follows the LS estimation in the first stage, while the DL-based detector comes after ZF detection in the second stage. Both receiver algorithms are tested using various MIMO configurations and the number of hidden layers and neurons inside the layers. In addition, they are compared to two conventional receiver structures first of which consists of LS channel estimation and MMSE detection, while the second one includes LS channel estimation and projected gradient iterative detection. According to computer simulation results, both FullCon and MdNet outperform traditional techniques in $2 \times 2$ and $4 \times 4$ MIMO setups. One notable outcome of computer simulations is that FullCon has superior performance than MdNet in $2 \times 2$ MIMO, whereas MdNet takes the lead in $4 \times 4$ MIMO.

As investigated so far, the literature of DL-based receiver architectures has grown to maturity. However, to incorporate them into real-world systems and standards, it is necessary to examine the feasibility of these designs using real-world datasets. It is also critical to have a sufficiently big dataset to reproduce the results of existing algorithms in the literature, create benchmarks, and compare different algorithms based on universal data [74]. As seen from Tables <ref> and <ref>, the popularity of practical datasets is not satisfying, which motivates researchers to focus on generating datasets using real-world scenarios. In [74] and [75], a generic dataset, dubbed DeepMIMO, for mmWave and massive MIMO systems is proposed. The DeepMIMO channels are generated based on accurate ray-tracing data collected from Remcom Wireless InSite software [90]. Therefore, this dataset includes the geometry of the surroundings as well as the transmitter and receiver positions. In addition, the data creation process is parametric, allowing researchers to fine-tune parameters based on the desired scenario. Similarly, [91] and [92] present a parametric and scalable data production framework for vision-aided wireless communications based on the 3D modeling and ray-tracing provided by [90]. The objective is the co-existence of visual and wireless data, with visual data, such as that collected from LiDAR (light detection and ranging) sensors or cameras, assisting wireless communication systems that function in the same device or area. Quadriga, which stands for quasi deterministic radio channel generator, is a tool for simulating practical MIMO radio channels for networks such as indoor environments and satellite communications [84], [85]. As stated in [85], Quadriga facilitates the creation of channel models given by entities such as 3GPP. It can also be considered as an application of the well-known WINNER model with modifications for satellite communications via its geometry-based stochastic approach. It extends the terrestrial snapshot-based simulation system to produce complex-valued baseband time series. The work in [93] provides a procedure for creating realistic channel environments for various 5G scenarios by combining a vehicle traffic simulator and a ray-tracing simulator. By producing propagation channel data, the goal is to aid DL-based solutions to challenges linked to the PHY of mmWave MIMO in 5G. To sum up, the analyzed studies that propose real-world datasets for MIMO communications are summarized in Fig. <ref>. In the next subsection, we will discuss DL-aided algorithms within the context of smart MIMO transmitters and E2E communications.

Major real-world datasets that are considered for DL-aided MIMO techniques.

§.§ Intelligent Transmitter Design and Autoencoders

We analyzed how to optimize individual blocks of a MIMO receiver separately and jointly in the previous subsections. Thus, we show that DL networks can learn and optimize a complete MIMO receiver framework for a range of optimization criteria. However, we have yet to achieve our ultimate goal of communication technology revolution with fully intelligent frameworks. Therefore, we will investigate DL approaches that optimize individual blocks of a MIMO transmitter and eventually learn a whole transmitter framework tuned with the corresponding receiver architecture. The basic idea underlying E2E communications is to replace the separate blocks of a MIMO transmitter, channel, and receiver with a single DL network that can be trained as an AE to address the conventional block structure's sub-optimization issue [25], [94]. As a departure, we will first delve into the details of DL-based constellation design in the following part.

As previously mentioned, constructing constellations with the aid of DL networks is an E2E process since the network must optimize both the positions and decision regions of the constellation points. The objective is to maximize the mutual information between the input and output of the channel, even when the channel model is unknown [46]. The study in [95] provides an AE for designing optimal constellations and receiver architectures for additive white Gaussian noise (awgn) channels with additive radar interference. The optimization metric is mutual information. Under various channel model assumptions for different SNR regimes, several demapping procedures are suggested. In addition, the proposed network enables using powerful coding methods such as turbo codes or low-density parity-check (ldpc) codes. Computer simulations reveal that the AE-based network outperforms the architectures with the traditional constellations. The network in [96] conducts probabilistic shaping to improve the constellations along with the geometric shaping. The probabilistic shaping optimizes the probability of occurrence of the constellation points. Even when the geometry of the constellation is fixed, such as QAM, the probabilistic shaping improves the achievable information rate. This network also maximizes mutual information using AEs, as in [95]. The proposed joint shaping method achieves near-capacity performance and outperforms both non-shaped QAM and just geometric-shaped QAM across AWGN and Rayleigh fading channels. [97] analyzes AE-based constellation design over a MU interference channel to address the dynamic interference problem. Specifically, the adaptive DL-based AE network can learn the degree of interference and enhance BER performance by adjusting each user's constellation accordingly. According to computer simulations, users' learned constellations are well-decomposed, and they concentrate on their clusters in the face of heavy interference. It is, therefore, much easier for the receiver to decode the information in the strong-interference environment. Furthermore, the AE-based network performs very similar to the typical uncoded binary PSK (bpsk) and quadrature PSK (qpsk), with around 2 dB improvement over 8-PSK and 16-QAM.

In contrast to the formers, [98] investigates AE-based Grassmannian constellation design in non-coherent MIMO systems, where neither transmitter nor the receiver requires CSI. Each point on the Grassmann manifold is a unitary matrix, and the distance between each point pair equals the chordal Frobenius norm. The recommended method is compared to Grassmannian constellations designed by traditional techniques and a non-Grassmannian constellation. Simulation results indicate that the proposed network exceeds all state-of-the-art approaches and attains the highest diversity order.

Along with intelligent constellation designs that are more resilient to dynamic propagation environments than the traditional fixed constellations, DL-based approaches have also been applied to precoding in massive MIMO transmitters. The goal is to use DL networks to offer reliable precoding with low-resolution DACs while substantially decreasing complexity. The study in [99] investigates a one-bit precoding design with the optimum constellation design utilizing AEs. In a multicast massive MU-MIMO system where the BS broadcasts the information signal to all users, the transmitter composes a DNN followed by a binary layer to satisfy the one-bit precoding constraint. Because the gradients of the binary layer are always zero, implementing the standard backpropagation approach is complicated. Therefore, the authors employ an approximation which is a variant of the straight-through estimator. Experiments over fixed and varying channels show that the AE-based one-bit precoding and constellation design network outperforms the PSK and QAM transmissions with the conventional one-bit precoder [100]. It is also worth noting that the performance of the AE-based constellation design with the standard one-bit precoder [100] is the best, demonstrating the potential of model-driven DL networks. Likewise, [101] provides AE-based symbol-level precoding (slp) and constellation design for unicast massive MU-MIMO systems in which the BS transmits a unique information signal to each user. Moreover, the authors examine robust SLP design for classic QAM constellations in instances when AE-based constellations are challenging to execute in reality. According to the numerical results, the proposed AE-based SLP and constellation design network outperforms the AE-based SLP with QAM constellation and non-robust SLP with QAM constellation. Another DL-based SLP design is provided in [102], where the authors propose an efficient precoding NN to mitigate MU interference in an MU-MISO system model. The optimization metric is the minimum quality-of-service of all users. Simulation results illustrate that the suggested network has superior performance than the conventional block-level precoding. However, the traditional convex optimized SLP scheme beats the recommended network with considerably greater complexity, indicating that the DL-based SLP network provides robust performance with significantly reduced complexity. The work in [103] is similar to [99] and [101] in that it offers an AE-based precoding network for MIMO systems. The transmitter consists of two DNNs that encode bits to symbols and precode these symbols with power normalization at the end. As seen from computer simulations, the AE-based precoding network surpasses the conventional linear precoding techniques such as ZF and MMSE and the non-linear Tomlinson-Harashima precoding assuming perfect CSI. The authors of [104] present a similar separate DNN structure of [103] in the MIMO transmitter. The first of the two DNNs encodes bits to symbols, while the second does precoding. The receiver of this AE-based approach includes a radio transformer network (rtn). The precoding network at the transmitter and the RTN at the receiver are trained in an E2E fashion to outclass the classical DL-based MIMO system. Simulations with different MIMO configurations reveal that the proposed combined precoding and RTN architecture outperforms the traditional linear precoding methods ZF and MMSE and the DL structures with only precoding or RTN. The authors also thoroughly analyze the precoding network's learned constellations.

The substantial research in [105], [106], and [107] results in a DL-aided precoding framework for downlink massive MU-MIMO systems with a uniform planar array at the BS. The objective of this framework is to implement precoding that maximizes the ergodic rate while limiting overall transmit power under a constraint by using both instantaneous and statistical CSI. The authors, however, convert this ergodic rate maximization issue to an enhanced quality-of-service problem to make it tractable, and the structure of optimum precoding emerges as a result. Thus, the proposed structure employs a DNN to successfully obtain the optimum precoding architecture and to mitigate complexity problem. Consequently, the proposed framework reduces complexity significantly while maintaining almost the same performance as the traditional iterative technique. Furthermore, the authors minimize complexity by splitting the optimization problem into two sections that evaluate instantaneous and statistical CSI, respectively. Experiments with the Quadriga channel model [84], [85] prove that both general and low-complexity frameworks produce near-optimal performance with considerably lower complexity than the classical iterative approach.

The study in [108] proposes another data-driven DNN-based precoding architecture in which the network fully learns the input-output relationship of a nearly optimum precoder to maximize mutual information. Simulation results show that the suggested network produces almost the same performance as the optimum precoder with significantly lower complexity than the traditional iterative precoders, just like in [105], [106], and [107]. The work in [109] takes a similar approach to [108], in which a data-driven DNN learns the behavior of an ideal precoder to maximize the mutual information with considerably reduced complexity. The suggested DNN accepts vectorized water-filling precoding matrix as input and yields the optimum one. Concerning various MIMO setups, simulations indicate that the recommended network achieves the optimum performance while substantially decreasing the execution time.

Apart from the AE-based and DNN-based precoding designs discussed so far, model-driven DL-based networks are also popular within the context of precoding design by combining the conventional algorithms with DL techniques. The work in [110] optimizes the conventional non-linear biconvex 1-bit precoding (c2po) method by automatically tuning the algorithm parameters in a massive MU-MIMO system model. The NN optimized C2PO (nno-c2po) unfolds the iterations of C2PO and utilizes backpropagation to adjust parameters. Simulations conducted using the Quadriga channel model [84], [85]  illustrate that the proposed NNO-C2PO algorithm requires about 50% fewer iterations than C2PO for a similar BER performance. In addition, NNO-C2PO may employ the same parameters learned for a given channel model for various channel models with a moderate performance loss, indicating robustness to dynamic channel circumstances. In like manner, the study in [111] unfolds another conventional iterative algorithm named iterative discrete estimation (ide2) to develop a model-driven DL network for massive MU-MIMO with finite-alphabet precoding. The proposed network, IDE2-Net, precodes the transmit signals using low-resolution DACs by modifying the current IDE2 algorithm with configurable parameters. Simulations reveal that IDE2-Net has substantially better performance than the IDE2 algorithm and slightly outperforms NNO-C2PO [110] for the same number of iterations. Following in the footsteps of the formers, the authors unfold the classical iterative conjugate gradient method for constant envelope precoding in [112]. The suggested network, CEPNet, optimizes the traditional approach by introducing trainable parameters to decrease MU interference and computational cost. Simulations show that CEPNet significantly outperforms the conventional algorithm in terms of both BER and average achievable rate performances with less complexity. CEPNet is also resistant to channel estimation errors and channel model mismatch as in [110]. The study in [113] unfolds the iterative weighted MMSE (wmmse) algorithm into a layer-wise structure to maximize the sum rate. The proposed network embeds trainable parameters to the WMMSE algorithm to eliminate matrix inversions and reduce complexity. Unlike previous studies, the authors develop a CNN-based precoding scheme to compare their model-driven method to a data-driven network. Extensive simulation results in a massive MU-MIMO system model with varying numbers of transmit antennas and users show that the proposed method reaches the sum rate of the WMMSE algorithm and outperforms the CNN-based scheme with much less complexity. By unfolding, another conventional iterative technique, the alternate direction method of multipliers (admm), is mapped to a DL framework in [114]. The authors provide ADMM-Net for large-scale mmWave communications in which the DL-based SLP selects the optimum subset of RF chains to decrease power consumption. Simulation results illustrate that ADMM-Net outperforms conventional ADMM, orthogonal matching pursuit, and coordinated descent algorithms.

Considering a different approach than prior research of model-driven DL networks that unfold existing iterative algorithms, [115] presents a DL-based precoding architecture driven by the constructive interference communication model. The suggested CI-NN model includes a customized loss function implemented through a customized layer following the output. CI-NN attains the same performance as the conventional constructive interference model and outperforms the linear ZF precoder, as indicated in simulations for a different number of users. Besides, CI-NN is a user-adaptive method in which a trained model with a specific number of users operates in scenarios with a variable number of users, which is practical for dynamic environments.

After thoroughly investigating the intelligent massive MU-MIMO transmitter architectures such as AE-based and DNN-based complex-shaped constellations and precoding designs, it is the perfect time to focus on E2E communication systems in which a single giant DL network substitutes for all individual blocks. This approach, illustrated by an example system model in Fig. <ref>, can enable E2E optimization without requiring a mathematical channel model, which is highly practical since the mathematical definition of the channel might be challenging to obtain in some complicated environments. The main issue with the unknown channel model is that the conventional backpropagation technique, on which most of the DL networks rely, necessitates complete knowledge of the gradients in each layer. When the channel model is unknown, it is impossible to obtain gradients which complicates the backpropagation. Researchers in this field have been proposing various solutions to this problem. The study in [116] presents an optimization technique called simultaneous perturbation stochastic optimization to approximate channel gradients so that the E2E model becomes trainable using the standard backpropagation method. The E2E network with the suggested approximation achieves the theoretical BER performance in AWGN and Rayleigh fading channels.

An E2E PHY communication system model, where individual signal processing blocks are replaced with a single adaptive DL network.

The authors of [47] and [117] introduce a novel aspect to the unknown gradient challenge in which the E2E network iterates between the reinforcement learning-based transmitter training and supervised receiver training. The transmitter acts as an agent in an reinforcement learning problem, learning the environment using the loss function from the receiver as the reward. On the other hand, the receiver is trained in a supervised manner to obtain the loss function. Simulation results indicate that the proposed alternating method performs similarly to the E2E supervised DNN network with a known differentiable channel model. On top of this alternating training technique, the work in [118] proposes a reliable feedback system in which the receiver can convey loss value to the transmitter without assuming a flawless feedback link. The suggested feedback system trains both ends of the communication as transmitter and receiver, removing the necessity for a perfect feedback link. When the E2E alternating network [47], [117] equipped with the recommended feedback system is trained in a noisy feedback environment, it performs almost the same as the E2E alternating network with perfect feedback link, as simulations indicate.

The studies in [119] and [120] provide a channel-agnostic E2E network with a conditional GAN modeling the channel to produce gradients. Using the encoded signals from the transmitter and the received pilot signals as the conditioning information, the conditional GAN generates the channel model and paves the way through supervised training. Extensive simulations yield that the suggested network achieves the performance of an E2E-DNN network with a known channel model over AWGN and Rayleigh fading channels. However, the authors of [121] and [122] point out the gradient vanishing and overfitting issues of the GAN-based E2E network. They offer a residual-aided GAN that generates the difference between the transmitted and received signal. Residual-aided GAN introduces extra gradients to the network to overcome the gradient vanishing problem. In addition, the loss function is modified by adding a regularizer to prevent overfitting. Simulations using theoretical AWGN and Rayleigh fading channels and practical ray-tracing channel dataset DeepMIMO [74], [75] illustrate that the proposed residual-aided GAN outperforms the E2E alternating network [47], [117] and conventional GAN-based E2E network [119], [120] in all cases.

Instead of a GAN imitating the channel, the studies of [123] and [124] suggests a stochastic convolutional layer for representing channel in an E2E network without pilot signals. The receiver DNN consists of two DNNs for channel information extraction and data recovery, with the extracted channel information merged with the received signal via a bilinear production operation used for data recovery. Extensive simulations in frequency-selective and flat-fading MIMO channels yield that the recommended approach outperforms the LS and MMSE channel estimations while trailing the GAN-based E2E network. In addition, the E2E method is exceptionally resistant to channel correlations, whereas the conventional techniques deteriorate even with perfect CSI knowledge. Another intriguing experiment is wireless image transmission in which the authors test their E2E network with the recovered image quality, which performs better than conventional compression techniques.

E2E networks are easier to use when the channel model is accessible since channel gradients exist and the network is trainable in an E2E fashion. The pioneering study in this area, which is given in [25], analyzes an E2E network for a given channel model and a loss function. This comprehensive research includes various investigations. First, the AE-based E2E network is compared against an uncoded BPSK scheme at several coding rates, with the proposed network outperforming the latter. To observe expert knowledge in E2E networks, the authors amend an RTN on top of the AE-based network, which improves performance. Extending the AE concept to multiple transmitters and receivers, the authors examine the performance of an AE-based E2E network in an interference channel. Comparison against a time-sharing scheme reveals substantial gains. In addition, a CNN-based DL network yields promising performance within the context of modulation classification and outperforms the expert feature approaches. This exhaustive treatise opens the door for our ultimate goal of PHY revolution. The authors of [125] apply this innovative E2E technique to MIMO system models and investigate the performance of AEs across Rayleigh fading channels for various MIMO configurations. In the first case, an AE optimizes its encoding scheme during training to introduce spatial diversity in a $2 \times 1$ MIMO setup. Compared to the well-known Alamouti coding, the AE creates a superior encoding scheme without CSI at the transmitter, as demonstrated in simulations, where the AE achieves slightly higher diversity in two time slots. This improvement is the consequence of the uneven distribution of power across antennas. In the second case, another AE is trained to obtain spatial multiplexing with lower BER than a traditional MIMO system in a $2 \times 2$ MIMO setting. Similar to the situation of spatial diversity, the AE generates better spatial multiplexing code and yields substantially higher BER performance.

[126] extends the single-input single-output (siso) IC system over AWGN channel investigated in [25] to SISO and MIMO interference channel systems over Rayleigh fading channel. The proposed AE [25] with a modulation order of four in a SISO interference channel system performs slightly better than the standard single-user no interference MIMO system using the QPSK modulation scheme. The demonstration of the received signals reveals well-separated four regions corresponding to the constellation points, which indicates how effectively the AE learns how to transmit with a modulation order of four. With the AE trained in a MIMO interference channel, performance improvement becomes considerable compared to the standard single-user no interference MIMO system. Assuming the channel model is available as in [25], the study in [127] proposes a block AE-based E2E network to deal with the dynamic input lengths, which is notable in terms of efficiency. The suggested network employs RNN to build a memory mechanism. Simulation results show that the RNN-based AE network has superior performance over the CNN-based networks in AWGN and custom fading channels. Furthermore, the authors examine the learned 64-QAM constellation diagram.

The work in [128] provides another thorough research that includes numerous examinations for E2E communication systems. The principal motivation is to switch from symbol-wise AE to bit-wise AE for the first time in the literature to show that optimizing bit metrics improves the overall performance. Bit-wise AE with IEEE 802.11n (Wi-Fi 4) LDPC code outperforms the symbol-wise AE with IEEE 802.11n LDPC code and the conventional communication using PSK or QAM. Also, the learned constellation analysis proves that bit-wise optimization leads to a better separation in the constellation, which is a solid indicator of performance improvement. The authors further enhance the BER performance by replacing the demapper of the model with the iterative demapping and decoding scheme, which results in a superior BER performance compared to the bit-wise AE without iterative demapping and decoding at the receiver. In addition, they optimize the LDPC codes used in iterative demapping and decoding-aided bit-wise AE and obtain enhancement in the BER performance. Although the AWGN channel is assumed in computer simulations, the strength of the proposed technique is its flexibility to be applied to any channel model without modification, as demonstrated by simulations utilizing software-defined radio (sdr). The study in [129] investigates a solution to the problem of finding $(n,k)$ block codes to maximize the minimum Hamming distance between the codewords by using AEs. Along with a custom loss function, the authors suggest two receiver structures depending on DNN and bidirectional RNN. The proposed model performs almost the same as the optimal block codes in the AWGN channel, and the minimum Hamming distance learned by the AE achieves its optimal value specified in theory. In addition, AE-based block code outperforms the theoretical optimum under channel model mismatches in which the performance difference increases as the correlation level grows.

According to the reviewed major studies in the literature, we note that there is noticeable progress toward entirely intelligent and E2E optimized wireless communication systems by replacing all separate blocks in conventional communication systems with a single massive DL network. Without a doubt, there will be more developments soon to incorporate these innovative approaches into real life. In the next subsection, we will explore the emerging applications of DL in transforming classical IM to make it a compelling technology for 6G.

§.§ DL-Empowered IM 2.0: Improving Efficiency of Conventional IM

We have covered the main developments in DL applications for massive MIMO systems thus far, beginning with symbol detection at the receiver and progressing to E2E intelligent communication systems. Another promising aspect within the 6G vision is MIMO-IM, which, as previously stated, enables high energy and spectral efficiency together while also decreasing the number of required RF chains by leveraging the building blocks of a communication system as additional information sources. However, the trade-off between detector complexity and BER performance is an issue for IM systems. In addition, employing several advanced algorithms such as transmit antenna selection (tas), power allocation, and MCS selection can further improve the BER performance of IM systems. Because of the immense complexity of traditional methods to implement these algorithms, DL approaches seem appealing for the efficiency improvement of IM systems. Fig. <ref> categorizes the primary contributions of DL to IM systems.

The study in [130] proposes a novel data-driven framework for TAS and power allocation issues in SM-MIMO systems by considering two different ML techniques. As the initial contribution, the framework achieves feasible solutions with decreased complexity comparing conventional optimization approaches by utilizing supervised-learning classifiers such as the $K$-nearest neighbors (knn) and support vector machine (svm). The authors examine a DNN-based framework for optimization tasks in the second part. Furthermore, they explore the impact of feature vector design on BER performance for both methods. Specifically, they suggest three distinct feature vector generation (fvg) methods: conventional FVG with the modulus of the channel elements, separate FVG with the modulus of the real and imaginary parts of the channel elements, and joint FVG with the correlation of the channel matrix's column pairs. Extensive simulations over various MIMO configurations and modulation schemes show that SVM-based TAS-SM outperforms KNN-based TAS-SM, while DNN-based TAS-SM exceeds both. Joint FVG has superior performance than the other two FVG approaches with near-optimal performance. Similarly, the DNN-based power allocation in SM with joint FVG achieves the best BER performance compared to other power allocation scheme and FVG combinations. On top of that, [131] minimizes the computing overhead by removing the repeated elements of the antenna selection process. The proposed DNN outperforms the proposed method in [130] with fewer hidden neurons in which both schemes employ the joint FVG technique. However, Euclidean distance-optimized antenna selection (edas) has substantial performance than these DNN-based TAS-SM techniques.

Four main applications of DL in MIMO-IM.

In [132], the authors suggest a DNN-based and a gradient boosting decision tree based TAS-SM technique. Considering the importance of features obtained from the channel matrix, they employ channel matrix correlation, same with the joint FVG in [130], as the feature vector. Simulation results indicate that both DNN- and gradient boosting decision tree based TAS-SM methods perform near-optimal performance while outperforming conventional SM and random TAS-SM schemes. In addition, gradient boosting decision tree attains lower complexity than DNN with slightly worse BER performance. Similar to [132], the work in [133] provides a DNN-based and a decision tree based TAS-GSM scheme under practical channel scenarios. The authors create a $16 \times 4$ test-bed using SDRs for tests and estimation of the impairments of the channel for theoretical optimization benchmark. Experiments reveal that the suggested techniques outperform EDAS while DNN-based TAS-GSM exceeds decision tree based TAS-GSM under the real-life scenario. The authors of [134] offer another TAS-SM scheme depending on DNNs combining the modulus of the channel matrix and the channel matrix correlation as the feature vector, which corresponds to the combination of conventional FVG and joint FVG in [130]. DNN-based TAS-SM with the recommended FVG type performs similarly to EDAS while outperforming random forest decision-based TAS-SM and various conventional TAS-SM schemes and achieves lower complexity.

The TAS problem is addressed in [135] for full-duplex SM systems in which self-interference is a crucial issue using DNN and SVM methods. The authors first derive the upper and lower bounds of the full-duplex SM's channel capacity and subsequently utilize these derivations in EDAS and capacity-optimized antenna selection to generate a training label set. In contrast to previous TAS methods majority of which use joint FVG of [130], they propose eigenvector FVG based on principal component analysis to extract vector features of the channel matrix and enhance the BER performance. Extensive simulations illustrate that the proposed eigenvector FVG leads to a better performance than joint FVG of [130] in various transmit antenna number and modulation scheme settings. The proposed DNN and SVM methods function almost the same and are comparable to EDAS. They outperform capacity-optimized antenna selection, KNN of [130], and other traditional antenna selection algorithms.

Apart from TAS and power allocation, MCS selection is another essential concept for enhancing the efficiency of IM systems. The study in [136] describes a DNN-based codebook selection method for SM-MIMO systems. The receiver predicts the symbol error rate (ser) corresponding to each codebook using the instantaneous channel state and sends the optimal codebook back to the transmitter. Each codebook assigns different constellation combinations to transmit antennas, where the total number of bits per channel use remains constant. Dynamically switching between these codebooks according to DNN-based SER predictions yields significantly better results than using any codebook throughout the transmission. The work in [137] offers a DNN-based approach to calculating mutual information of an SM-MIMO system on the fly, allowing for dynamic MCS adaptation. The authors explore different input feature vectors, where they contain some features from channel matrix and SNR. The proposed DNN-based method obtains mutual information almost perfectly with significantly lower complexity than traditional Taylor and Jensen approximations, paving the door for MCS adaptation. Furthermore, the authors expand their analyses in [138] to calculate the channel capacity of a GSM-MIMO system model, for the same purpose with [137] as allowing dynamic adaptation. The suggested DNN-based model computes the capacity with almost no error, as provided by simulations.

[139] presents a DNN-based code rate selection technique in SM-MIMO systems. The proposed DNN extracts the norms of channel matrix columns and the angles between each pair of channel matrix columns and combines them with SNR as input features, as described in [137]. The DNN output is the maximum coding rate that is suitable under a BER constraint. Simulation results show that DNN-based dynamic code rate selection delivers near-maximum throughput and substantially outperforms the fixed-rate scenarios under a BER constraint. The authors extend their model in [140] to predict the optimal MCS rather than only code rate depending on the same input features. The DNN-based MCS selection approach attains near-maximum performance, as does the sole coding rate optimization scenario in [139], indicating enhanced efficiency in SM-MIMO systems.

In contrast to the former studies, [141] provides a DNN-based modulation order selection scheme without considering the coding rate of the system. The recommended DNN model predicts the optimal modulation order by maximizing the minimum Euclidean distance, similar to the methods implemented in TAS-IM algorithms. Simulation results indicate that DNN-based modulation order selection aided SM-MIMO significantly outperforms the classical SM with substantially lower complexity than conventional selection algorithms.

Studies discovered thus far unlock the potential of IM systems for future 6G and beyond communication technologies, owing to DL approaches that further enhance system efficiency by employing sophisticated algorithms such as TAS, power allocation, and MCS selection with remarkably low complexity. We may now conclude the examination of the literature on DL-aided MIMO systems and move on to the implementation of the generic MIMO-IM model presented in [49], which might give a sense of MIMO-DL programming to interested readers.

§.§ MIMO-DL in Action

GSM is one of the most promising techniques for the PHY of future wireless communication systems by combining the benefits of SM and MIMO. Reducing the number of required RF chains by activating only a subset of the transmit antennas, GSM has lower complexity than conventional MIMO. It also improves SM efficiency by activating more transmit antennas and conveying more information bits. However, when the number of transmit antenna combinations and information bits increases, the complexity of MLD detection escalates to intolerable levels, as discussed earlier. On the other hand, linear detectors, such as ZF and MMSE detectors, produce significantly worse BER performance than MLD, introducing a trade-off into GSM systems. To address this BER-complexity trade-off, the authors of [49] suggest a DNN-based detector for GSM systems. They consider a GSM system model with $N_{t}$ transmit and $N_{r}$ receive antennas $(N_{r} < N_{t})$, where $N_{p}$ $(2 \leq N_{p} \ll N_{t})$ out of $N_{t}$ transmit antennas are selected by index bits to be activated in any time slot. There are $N = 2^{\lfloor \text{log}_{2} \binom{N_{t}}{N_{p}} \rfloor}$ legitimate transmit antenna combinations (tac), where each active antenna transmits a symbol from normalized $M$-QAM constellation $\mathcal{S}$. Thus, the total number of bits transmitted in each time slot equals $B = \text{log}_{2}N + N_{p}\text{log}_{2}M$.

Assuming a quasi-static flat fading MIMO channel matrix, $\mathbf{H} \in \mathbb{C}^{N_{r} \times N_{t}}$, the entries of which follow complex Gaussian distribution $\mathcal{C} \mathcal{N}(0, 1)$, the received signal vector $\mathbf{y} \in \mathbb{C}^{N_{r} \times 1}$ can be expressed as follows:

\begin{equation} \label{received_signal_vector}
\mathbf{y} = \mathbf{H}\mathbf{x} + \mathbf{n} =
\mathbf{H}_{I} \mathbf{s} + \mathbf{n},
\end{equation}

where $\mathbf{x}$ is the vector of transmit symbols, $\mathbf{n} \in \mathbb{C}^{N_{r} \times 1}$ is the additive white Gaussian noise samples vector with complex Gaussian distribution $\mathcal{C} \mathcal{N}(\mathbf{0}, \sigma^{2} \mathbf{\mathbf{I}})$, $\mathbf{s}$ is the symbol vector of the active transmit antennas, and $\mathbf{H}_{I}$ is the channel matrix corresponding to the active transmit antennas.

System model of the B-DNN detector that integrates a DNN into classical block detection for GSM systems [49].

The optimal MLD detector for this GSM transmission scheme performs an exhaustive search over all possible TACs and information symbols, as shown below

\begin{equation} \label{ML_Detection}
(\hat{I}, \hat{\mathbf{s}}) = \argmin_{I \in \mathbb{I}, \mathbf{s} \in \mathbb{S}} ||\mathbf{y} - \mathbf{H}_{I} \mathbf{s}||^{2}_{F},
\end{equation}

where $\mathbb{I} = \{I_{1},I_{2}, \dots, I_{N}\}$ is the set of legitimate TACs and $\mathbb{S}$ is the set of possible QAM symbol vectors of active transmit antennas. The linear detectors deal with solving an inverse operation including the channel matrix, $\mathbf{H}$, which can be given as follows:

\begin{equation} \label{ZF_MMSE_Detectors}
\begin{aligned}
&\hat{\mathbf{x}}_{ZF} = (\mathbf{H}^{H}\mathbf{H})^{-1} \mathbf{H}^{H} \mathbf{y},
\\
&\hat{\mathbf{x}}_{MMSE} = (\mathbf{H}^{H}\mathbf{H} + \sigma^{2}\mathbf{I})^{-1} \mathbf{H}^{H} \mathbf{y}.
\end{aligned}
\end{equation}

Here, $\hat{\mathbf{x}}_{ZF}$ and $\hat{\mathbf{x}}_{MMSE}$ represent detected symbol vectors by ZF and MMSE detectors, respectively. To perform a successful inverse operation, $N_{r}$ should at least equal to $N_{t}$ which ensures that $\mathbf{H}^{H}\mathbf{H}$ has a full rank. However, this requirement reduces the practical feasibility of GSM in massive MIMO systems with hundreds of transmit antennas at the BS since it is almost impossible for a user equipment to install that many antennas. Therefore, block linear detectors slightly change the equations in (<ref>) and employ $\mathbf{H}_{I}$ instead of $\mathbf{H}$, which results in $N$ solutions of $\hat{\mathbf{s}}_{I}$ for each legitimate TAC. In the final stage, block linear detectors compare the Euclidean distance between the received signal vector $\mathbf{y}$ and each $\mathbf{H}_{I} \hat{\mathbf{s}}_{I}$ product to estimate selected TAC and QAM symbols, as given below

\begin{equation} \label{block_linear_detection_final_stage}
\begin{aligned}
(\hat{I}) &= \argmin_{I \in \mathbb{I}} ||\mathbf{y} - \mathbf{H}_{I} \hat{\mathbf{s}}_{I}||^{2}_{F},
\\
(\hat{\mathbf{s}}) &= \hat{\mathbf{s}}_{\hat{I}}.
\end{aligned}
\end{equation}

This block-structured detection scheme achieves better BER performance than the classical linear detectors since it incorporates the MLD approach to the linear detectors. However, complexity increases with the growing number of transmit antennas, as expected, and the block detection still performs worse than the optimal MLD due to the suboptimal linear part. Thus, the authors of [49] replace the linear component of the block detection with a DNN to enhance the accuracy of $\hat{\mathbf{s}}_{I}$ estimation. Fig. <ref> demonstrates the GSM transmitter and the considered block-DNN detector.

Network and training parameters [49].

table_blue Parameters
table_blue Value
table_blue Parameters
table_blue Value

table_white Input Nodes
table_white $2(N_{r} + N_{r} N_{p})$
table_white Learning Rate
table_white $0.005$

table_white Hidden Layer
table_white $3$
table_white Number of Training Set