An Efficient Model Driven Deep Learning Based Approximate Message Passing Detector for MIMO Systems

Ⅰ. Introduction

In the modern wireless communication systems like fifth generation and beyond, the large capacity and high speed communication are vital for massive connectivity^[1]. Multiple-input multiple-output (MIMO) systems have key role in fifth generation and beyond communication systems^[1-3]. Every year substantial growth in the mobile devices results in both traffic and computational burden at base stations. Several challenges are required to be addressed in implementing large-scale signal processing to meet the growing demand at base stations^[4]. The massive MIMO systems with large number of antennas can achieve substantial improvement in spectral efficiency.

The substantial amount of computational burden for signal detection in a base station has become a key issue in the implementation of the next generation systems. The maximum likelihood (ML) detection method is optimal, but computational complexity increases significantly as antenna size increases. Although linear methods such as minimum mean square error (MMSE) and zero-forcing (ZF) detector has comparatively low computational complexity, they still require a heavy computational burden for the matrix inversion computation.

The approximate inverse detectors such as conjugate gradient method, Gauss-Seidel method, and Neumann series approximation, can reduce the significant amount of complexity^[5-7]. However, their performance is system ratio dependent, i.e. the ratio between the number of BS antennas to user terminals. The tree search algorithms like K-best method can achieve near ML performance^[8,9], but they usually suffer from computational complexity which increases with antenna size and modulation order^[10].

Message passing algorithms can be the potential signal detection technique due to its lower complexity and better performance compared to existing detection methods like MMSE, ZF, and tree search methods. The MIMO detection schemes using various message passing algorithms such as orthogonal approximate message passing (OAMP), and belief propagation (BP) have shown promising results^[11-14]. Particularly, the OAMP method has shown good performance but its computational complexity is comparatively high due to matrix inversion operation per iteration^[12].

Another approximate message passing (AMP) based detection method known as LArge MIMO AMP (LAMA) algorithm was proposed^[13]. The LAMA is a suitable method for the system with an antenna ratio, [TeX:] $$B \gt;1$$, where [TeX:] $$B=N/M$$, and [TeX:] $$M$$ and [TeX:] $$N$$ are the number of transmit and receiving antennas, respectively. Furthermore, LAMA algorithm suffers from performance degradation in lower order antenna size.

The recent success of machine learning makes it attractive to consider learning the MIMO detection directly from data^[15-17]. In [15], the authors proposed a deep neural network (DNN) and a recurrent neural network (RNN) based methods for single-path MIMO communication channels. Furthermore, convolutional neural network (CNN) based deep learning structure was proposed for the multi-path MIMO channel. However, these methods can be considered for MIMO system with [TeX:] $$M=N=2$$. The joint use of the ML detection and a CNN was proposed, where the ML detection is employed to produce an initial detection result and CNN improves the detection by exploiting the local correlation to suppress the interference^[17]. However, the computational complexity incurred from both ML detection and CNN makes it impractical for consideration. Several issues such as large number of parameters, huge training data set, generalization, and network architecture still require considerable amount of attention before considering them for practical implementation.

Therefore, model-driven DL methods, which unfold the iterative algorithms has attracted attention due to their excellent complexity and performance trade-off, as we can see from the example of applying RNN to iterative decoding process^[18]. Message passing methods can be extended to model driven deep learning (DL) architecture due to iterative nature and model driven design flow. Several model-driven DL based MIMO detection methods have been proposed where unfolding methods were used to enhance the performance^[18-23]. The MMNet and OAMPNet could improve the detection performance, especially for uncorrelated channels^[21,22]. The OAMPNet method adds two parameters to the original OAMP method per iteration. The performance of the OAMPNet method degrades on real-world channel models. The inverse operation in each layer makes the OAMPNet computationally complex and its complexity increases as antenna size increases, which hinders its suitability for large MIMO systems^[21].

Although a model-driven DL approach generally learns comparatively fast with a few parameters, some methods still require prohibitively long training time. For example, a DL method proposed in [23] used the calculation procedure of gradient-based optimization, and required up to [TeX:] $$10^6$$ training parameters, which made it computationally complex. In summary, full potential and flexibility of the neural networks has not been realized for MIMO detection, leaving room for improvement.

In this paper, we propose a novel model driven DL based massive MIMO detector. We employ a strategy that unfolds the iterations of AMP algorithm into DL layers^[24]. We unroll the iterations of the LAMA algorithm into sequentially connected layers of a deep neural network. The proposed method employs a trainable parameter for existing LAMA method and enhances the performance. The main advantage of the proposed method is that it adopts only one trainable parameter. Therefore, the proposed method requires less training time and it has lower computational complexity than the existing model driven DL based MIMO detection methods.

The remaining of the paper is as follows. Section Ⅱ describes the MIMO system model and details about the existing LAMA detection method. Section Ⅲ presents the proposed model driven DL based AMP method. Simulation results of proposed method are demonstrated in Section Ⅳ. Simulation results and complexity analysis of the proposed method is given in Section Ⅴ. Finally, the paper is concluded in Section Ⅵ.

Ⅱ. Related Works

2.1 System Model

Consider a [TeX:] $$M \times N$$ MIMO system with a transmitted signal vector, [TeX:] $$\mathbf{s}=\left[s_1, s_2, \ldots, s_M\right]^T$$, where each symbol, [TeX:] $$s_m$$ is independently chosen from a complex constellation set, [TeX:] $$X=\left\{c_i: i=1, \ldots,|X|\right.$$, with alphabet size of [TeX:] $$|X|=Q$$. The received signal vector is denoted as [TeX:] $$\mathbf{y} \in \mathbb{C}^N, \mathbf{y}=\left[\begin{array}{lll}y_1, & y_2, \ldots, & y_N \end{array}\right]^T$$, can be represented with an [TeX:] $$N \times M$$ complex channel matrix, [TeX:] $$H$$ as follows:

where [TeX:] $$n$$ is an [TeX:] $$N \tmes 1$$ complex Gaussian noise vector with variance of [TeX:] $$N_0$$.

To perform a signal detection at the receiver, the ML detection method is optimal which solves the closest lattice point problem by calculating the Euclidean distance (ED) between the received signal, y and all possible lattice points, Hs, and makes the decision of which lattice point minimizes the ED to y, i.e,

The above ML detection scheme achieves the optimal performance when all the transmitted symbol vectors are equally likely. However, its complexity increases exponentially with the number of transmit antennas and modulation order.

2.2 Conventional LAMA algorithm

The AMP is one of the iterative algorithms initially proposed for compressed sensing applications^[25]. Further advancement of AMP has been developed for different tasks in communication, and image processing^[26]. The AMP algorithm has been employed for MIMO detection problem known as LAMA detection method^[13,27].

Consider prior distribution [TeX:] $$p(\mathbf{s})=\prod_{m=1}^M p\left(s_m\right)$$ for each transmit symbol [TeX:] $$s_m$$ as follows:

Here, [TeX:] $$p_c$$ is the prior probability of each constellation point [TeX:] $$c \in X$$ with [TeX:] $$\sum_{c \in X} p_c=1$$ and [TeX:] $$\delta$$ (·) is the Dirac delta function; for uniform priors we have [TeX:] $$p_c = 1 / |X|$$.

Fig. 1 shows the algorithm for the conventional LAMA method. First, the preprocessing is performed and algorithm is initialized, where [TeX:] $$\theta_\rho$$ and [TeX:] $$\theta_\tau$$ are damping constants. In Fig. 1, the superscript of each variable denotes the iteration index, e.g., for the lth iteration of algorithm, [TeX:] $$\tau^1$$ is the signal variance. The mean value of [TeX:] $$\mathbf{z}^l$$ (i.e., new estimate of [TeX:] $$\hat{\mathbf{S}}^{l+1}$$) is found by using the function [TeX:] $$\mathrm{F}\left(\mathbf{z}^I, \rho^{\prime} \mathbf{g}\right)$$ and the variance of [TeX:] $$\mathbf{z}^l$$ is found by using [TeX:] $$\mathrm{G}\left(\mathrm{z}^{\prime}, \rho^{\prime} \mathrm{g}\right)$$. After preprocessing and initialization, the signal estimation process start through the iterations.

Inside the iterations, first, the mean value is computed as follows:

where [TeX:] $$w_c\left(\hat{s}_m, \tau^2\right)$$ can be found as:

Second, the message variance is computed as follows:

Fig. 1.

Algorithm for LAMA detection method ^[13,27].

Furthermore, the [TeX:] $$\bar{\tau}^l$$, and [TeX:] $$\rho^1$$ are the damping parameters estimated for signal mean and variance, respectively. Then, the Onsager term, [TeX:] $$\mathbf{v}^1$$, is estimated followed by the signal estimate [TeX:] $$\mathbf{z}^{1+1}$$.

Ⅲ. Proposed Model Driven Machine Learning Based MIMO Detection

3.1 Operational Principle

We first transform (1) into an equivalent real number operations in order to apply DL based model architecture, as follows:

where the received signal vector is denoted as [TeX:] $$\overline{\mathbf{y}} \in \mathbb{R}^{2 N}$$ and [TeX:] $$\overline{\mathbf{S}}$$ is the transmitted symbol vector. The [TeX:] $$\overline{\mathbf{H}} \in \mathbb{R}^{2 N \times 2 M}$$ is channel matrix with extended [TeX:] $$2N \times 2M$$ dimensions and [TeX:] $$\overline{\mathbf{n}} \in \mathbb{R}^{2 N}$$ is Gaussian noise with variance [TeX:] $$N_0$$.

Fig. 2.

An architectural structure of the proposed model driven DL based AMP method for L cascaded layers and internal structure of a single layer.

Fig. 2 shows the architectural structure of the proposed model by unfolding the iterations of LAMA algorithm into DL layers. The network is composed of [TeX:] $$L$$ cascaded layers and each layer has the same structure but with different parameter values. Furthermore, Fig. 2 shows the structure of layer l of the proposed method. Specifically, the detailed operations of the proposed method are explained by using the structurein Fig. 2 along with the algorithm in Fig. 3. First, we perform the preprocessing as given in line 2 of the algorithm and forward [TeX:] $$\overline{\mathbf{y}}^{M F}, \tilde{\mathbf{G}}$$, and g to each network layer of the proposed method. Then each layer uses the preprocessed data and estimates [TeX:] $$\hat{\mathbf{S}}^{l+1}, \mathbf{z}^{l+1}, \rho^{l+1}$$, and [TeX:] $$\tau^{l+1}$$ which are then forwarded to the next layer where they are utilized for new estimates by considering them as the estimates of the previous iteration.

Fig. 3 shows the algorithm for the proposed model drive DL based AMP detection method. After the pre-processing and initialization, the training block calls for AMP layer procedure which is given in Fig. 4. Each layer estimates the transmitted symbols and other parameters. After [TeX:] $$L$$ layers, the gradient is computed using loss function, followed by back propagation to update the learnable parameter. We set a learnable parameter [TeX:] $$\gamma^l$$ for the Onsager term by multiplying as follows:

Fig. 3.

Algorithm for proposed model driven DL based AMP detection method.

Fig. 4.

Procedure for proposed method with each layer in DL containing trainable parameter

where [TeX:] $$\gamma^l$$ is a scalar value at each layer. Therefore, the proposed network have to train [TeX:] $$L$$ different values for learnable parameter.

After the completion of training process, the algorithm returns the [TeX:] $$L$$ trainable parameters, [TeX:] $$$$\gamma^l, 1 \leq i \leq L, and MIMO detection is performed by using these. After repeating the preprocessing and initialization steps, the function LAYERS is called for MIMO detection. The function returns the estimate [TeX:] $$\mathbf{z}^L$$ which is the output vector containing estimation for the transmitted signal.

3.2 Selection of network parameters

The training data is randomly generated in pairs [TeX:] $$\mathbf{d}^j \triangleq\left(\overline{\mathbf{s}}^j, \overline{\mathbf{y}}^j\right)$$. For each sample pair, channel matrix [TeX:] $$\overline{\mathbf{H}}$$ is randomly generated by using Rayleigh fading MIMO channel model. The data [TeX:] $$\overline{\mathbf{S}}^j$$ is generated from QPSK modulation scheme for [TeX:] $$M$$ transmit antennas. The total number of layers are set to 10, i.e, [TeX:] $$L=10$$, for the proposed method. We train the network with 10,000 epochs. The network is trained using the Adam optimizer. The learning rate is set to be 0.0001. we choose the [TeX:] $$l_2$$ loss function as the cost function which can be defined as:

where [TeX:] $$D$$ denotes the number of training examples. Based on the loss, the learnable parameters are updated in the back propagation step of the Adamoptimizer. The measurement is repeated for each signal-to-noise ratio (SNR) separately. Therefore, optimal variables [TeX:] $$\gamma^L$$ may be different for different SNR values.

Ⅳ. Simulation Results

In this section, we compare the performance of the proposed model-driven DL detector for MIMO system. We assume a Rayleigh MIMO channel with perfect channel state information (CSI). The performance is compared with ML, conventional LAMA, K-best and MMSE methods. For the K-best, K is set to 5 and total number of iterations employed for LAMA were 10.

Fig. 5 shows the bit error rate (BER) performance comparison of the proposed method with the exiting methods for a 8 × 8 MIMO system. The proposed method produces better performance compared to the conventional LAMA and MMSE methods. Fig. 6 represents the BER performance comparison of the proposed method with the exiting methods for a 16 × 16 MIMO system. The proposed method produces better performance compared to the conventional LAMA, K-best and MMSE methods. The proposed method shows error floor in higher SNR ranges, and worse performance than the K-best algorithm for the 8 × 8 MIMO systems. On the other hand, it can provide much better performance than the existing algorithms including the LAMA, K-best, and MMSE methods for the 16 × 16 MIMO system. Therefore, the proposed method has significant performance improvement approaching to ML performance in higher order antenna size.

Fig. 5.

BER performance comparison of proposed method with various existing methods for the 8 × 8 MIMO system.

Although the proposed method produced appreciable performance gain compared to the conventional LAMA, there still exist noticeable performance gap between the proposed method and the ML method. It is also possible to define more learnable parameters in each layer of LAMA algorithm to further enhance the performance approaching to the optimal performance, at the expense of marginal computational complexity.

Fig. 6.

BER performance comparison of proposed method with various existing methods for the 16 × 16 MIMO system.

Table I compares the complexity of the of proposed and conventional methods. The ML method has the highest complexity and it increases with antenna size, which makes it impractical. The complexity of the K-best scheme increases rapidly as antenna size increases in the order of [TeX:] $$O(KMN^2)$$. Due to the matrix inversion in the filtering process, MMSE scheme complexity increases significantly with transmit antenna size. The complexities of the proposed and conventional AMP algorithms are evaluated for [TeX:] $$L$$ iterations. The main contribution of complexity is due to estimation of mean and variance of signal. The additional complexity of the proposed method is due to multiplication of trainable parameter. However, the trainable parameter involves negligible computational complexity increase.

Fig. 7 shows the complexity comparison of the proposed with the conventional methods. For the 8 × 8 MIMO system, the complexity of MMSE is lowest but its performance is the worst. Although the performance of the K-best is better than the proposed method but the complexity of the K-best algorithm is much higher than the proposed scheme, and it increases drastically as antenna size increases. For 16 × 16 MIMO systems, the proposed method has better performance than the conventional LAMA, MMSE, and K-best methods. Furthermore, the proposed method has lower complexity compared to both MMSE and K-best methods. As antenna size increases the proposed method achieves near ML performance with significantly lower complexity than the MMSE and K-best methods.

Fig. 7.

Computational complexity comparison of proposed and conventional methods, [TeX:] $$L=10$$

Ⅴ. Conclusion

In this paper, we proposed a novel machine learning based AMP algorithm for performance enhancement. The proposed method shows significant performance enhancement as the antenna size is increased. Furthermore, the proposed method has negligence computational complexity increase due to the adoption of a compact trainable parameter. Therefore, the result of this study demonstrated the possibility of further performance enhancement by adopting additional trainable parameters. Our future study will investigate the feasibility of employing new trainable parameters such as dampening factors, thereby improving the performance. It is also possible to define a learnable matrix with the dimensions [TeX:] $$\mathbb{R}^{M \times L}$$ with each layer containing learnable parameter of dimension [TeX:] $$M$$.