Linear Minimum Mean-Square-Error Transceiver Design for Amplify-and-Forward Multiple Antenna Relaying Systems

Linear Minimum Mean-Square-Error Transceiver Design for Amplify-and-Forward Multiple Antenna Relaying Systems

Linear Minimum Mean-Square-ErrorTransceiver Design for Amplify-and-Forward

Multiple Antenna Relaying Systemssubmitted by

With the source, relay and destination all equipped with multiple antennas, anatural question is how to allocate the limited power resource to make the commu-nication as efficient as possible This problem is addressed by linear transceiverdesign in this thesis Transceiver designs for point-to-point MIMO or multi-userMIMO systems have been widely addressed previously However, for AF MIMO

In this thesis, we start with a fundamental three nodes source-relay-destinationMIMO system The forwarding matrix at relay and equalizer at destination arejointly designed, under the realistic scenario that channel estimates in both hopcontains Gaussian error Two robust design algorithms are proposed to minimizethe mean-square-error (MSE) of the output signal at the destination The first oneis an iterative algorithm with its convergence proved analytically The other is anapproximated closed-form solution with much lower complexity than the iterativealgorithm.

Next, we consider the AF MIMO orthogonal frequency division multiplexing(OFDM) system over frequency selective fading channels Again, the forwardingmatrix at relay and equalizer at destination are jointly designed by minimizing thetotal MSE of the output signal at the destination, under channel estimation errors.However, since OFDM is a multicarrier modulation, transceiver design in such sys-tem involves power allocation in both spatial and frequency domains, and thus ismore complicated than the first system In the proposed solution, the second-ordermoments of channel estimation errors in the two hops are first deduced in the fre-quency domain Then, the optimal designs for both correlated and uncorrelatedchannel estimation errors are investigated The relationship between the proposedsolutions with existing algorithms is also disclosed.

responds to the case where one base station communicates with multiple terminalsvia one relay station In this system, the source precoder, relay forwarding matrixand destination equalizer are jointly designed by minimum MSE criterion Bothuplink and downlink cases are considered It is found that the uplink and downlinktransceiver designs share some common features and can be solved by a generaliterative algorithm On the other hand, another proposed algorithm for fully loadedor overloaded uplink system is shown to include several existing results as specialcases.

(Total words: 477)

Chengwen Xing

Transceiver Design for Amplify-and-ForwardMultiple Antenna Relaying Systems

Chengwen Xing

B.Eng., Xidian University, Xi’an, P R China

A thesis submitted in partial fulfillment ofthe requirements for the degree of

Copyright c° 2010 Chengwen Xing

I declare that this thesis represents my own work, except where due ment is made, and that it has not been previously included in a thesis, dissertation orreport submitted to this University or to any other institution for a degree diplomaor other qualifications.

Chengwen Xing

First, I am most grateful to my supervisors, Dr Yik-Chung Wu and Dr RickyYu-Kwong Kwok I am very lucky to have two very nice supervisors in my Ph.D.study I really appreciate Dr Kwok for giving me the opportunity to study in theUniversity of Hong Kong I am deeply indebted to Dr Wu for his patient guidanceand encouragement which acquaints me with signal processing and optimizationtheory The door of his office is always open for me, and he selflessly shares withme his invaluable experience in research He taught me how to read a paper, how tofind a direction, how to write a paper and even how to be a teacher I am indebtedto Dr Wu for all his help, support and kind consideration This thesis would not bepossible without the help from him.

I would like to express my sincere gratitude to Dr Shaodan Ma, for her helpfulreviews, insightful comments and selfless help on my research I would also like tothank Prof Tung-Sang Ng for his encouragement and help in my study.

Moreover, I would like to thank my friends I have met in the University of HongKong over the years: Dr Tyrone Tai-On Kwok, Dr Fanglei Sun, Dr Gan Zheng,Dr Menglong Jiang, Dr Xiaoshan Liu, Dr Yiqing Zhou, Dr Hongzheng Wang,Mei Leng, Lanlan He, Dr Carson Ka-shun Hung, Dr Jianwu Chen, Dr Chiu-WaNg, Xiao Li, Jun Zheng, Kun Cai, Xun Cai, Steve, Terry, Jing Xie, Jian Du, RuiMin and Bin Luo for their kindly help.

I would like to thank the University of Hong Kong for providing financial port via postgraduate studentship and CRCG conference grants I must also ac-knowledge the Department of Electrical and Electronic Engineering for all its sup-port to postgraduate students, including the departmental conference grant.

sup-Finally, I am seriously indebted to my parents and my wife for their love Mywife, Ms Xingyuan Hao has been waiting for me for 5 years in Beijing during mypostgraduate study Thanks for her love.

1.1.2 Multiple-Input Multiple-Output Systems 2

1.1.3 AF MIMO Relay Systems 4

1.2 Research Motivation and Problems to be Tackled 4

1.3 Organization and Contributions of the Thesis 8

1.4 Commonly Used Notations 9

2 Robust Transceiver Design for AF MIMO Relay Systems 11

2.4.3 Summary and convergence analysis 20

2.5 The Proposed Closed-Form Solution 21

2.6 Extension to Weighted MSE Criterion 27

2.7 Simulation Results and Discussions 29

2.7.1 Simulation Setup 29

2.7.2 Convergence Performance of Iterative Algorithm 30

Page2.7.3 Effect of Estimation Error σ2

3.3 Channel Estimation Error Modeling 44

3.4 Transceiver Design Problem Formulation 47

3.5 Proposed Closed-Form Solution 50

3.5.1 Uncorrelated Channel Estimation Error 56

3.5.2 Correlated Channel Estimation Error 58

3.6 Simulation Results and Discussions 60

4.2 Downlink Transceiver Design 75

4.2.1 System model and problem formulation 75

4.2.2 Proposed iterative algorithm 78

4.2.3 Summary and Initialization 81

4.3 Uplink Transceiver Design 82

4.3.1 System model and analogy with downlink design 82

4.3.2 Uplink transceiver design for fully loaded or overloadedsystems 85

4.3.3 Special cases 90

4.4 Simulation Results and Discussions 91

List of Figures

1.1 Cooperative communication system 2

1.2 MIMO system 3

1.3 Single-user AF MIMO relay system 4

1.4 Multi-user AF MIMO relay system 5

2.1 Amplify-and-forward MIMO relay diagram 12

2.2 Convergence performance of the iterative algorithm with α = 0, β =0.2 and σ2e= 0.002 . 30

2.3 The MSEs for the closed-form solution and the iterative algorithm fordifferent β and σ2e, when α = 0 and σ2e= 0.002 . 31

2.4 Convergence behaviors of the iterative algorithm with different izations, when α = 0.6, β = 0.5 and σ2e= 0.001 . 32

initial-2.5 The MSEs for the closed-form solution, the iterative algorithm andthe algorithm based on estimated channels only for different σ2e, whenα = 0.6 and β = 0.45 . 33

2.6 The MSEs for the two proposed solutions and the algorithm based onestimated channels only for different α, when β = 0.45 and σ2e= 0.005. 342.7 The MSEs for the two proposed solutions and the algorithm based onestimated channels only for different β, when α = 0.45 and σ2e =0.005. 35

2.8 The BERs for the proposed closed-form solution, iterative algorithmand the algorithm based on estimated channels only for different σ2e,when α = 0.45 and β = 0.45 . 36

Figure Page3.1 Amplify-and-forward MIMO OFDM relaying 423.2 MSE of received signal at the destination for different σ2

ewhen α = 0. 623.3 MSE of received signal at the destination for different σ2

ewhen α =0.4 . 633.4 MSE of received data at the destination for HSA, SPA and proposed

robust algorithm when α = 0.4 . 633.5 MSE of received data at the destination for different α . 643.6 BER of received data at the destination for different σ2

ewhen α = 0.5. 654.1 Amplify-and-forward MIMO relay downlink and uplink cellular systems 754.2 The convergence behavior of the proposed Algorithm 1 when NB = 4,

NR= 4 and NM,k = 2 with 2 users 924.3 Total MSEs of detected data of the proposed Algorithm 1 and subopti-

mal algorithms, when NB= 4, NR= 4, NM,k= 2 and Pr/σ2

v=20dB 934.4 Total MSEs of detected data of the proposed Algorithm 1 with and

without precoder design 944.5 The convergence behavior of Algorithm 2 for uplink when NB = 4,

NR= 4 and NM,k = 2 954.6 Total MSEs of detected data of Algorithm 2 and suboptimal algo-

rithms, when NB= 4, NR= 4, NM,k= 2 and Ps/σ2

n=20dB 964.7 Total MSEs of the detected data of the Algorithm 1, Algorithm 2 with

relaxation and the algorithm proposed in [30] 97

Multiple-antenna communication system is an important research topic in thepast decades It increases the data rate or diversity in reception, without occupy-ing additional frequency or time resource On the other hand, amplify-and-forward(AF) relaying attracts a lot of attention lately, as it is suitable in cases where thesource cannot directly communicate with the destination, but is possible via a relayin the middle The AF relay simply amplifies the received signal without decoding,thus its operation is favorable in implementation The combination of multiple-input multiple-output (MIMO) communication and AF relaying technique is cur-rently under consideration for several future wireless communication standards.

With the source, relay and destination all equipped with multiple antennas, anatural question is how to allocate the limited power resource to make the commu-nication as efficient as possible This problem is addressed by linear transceiverdesign in this thesis Transceiver designs for point-to-point MIMO or multi-userMIMO systems have been widely addressed previously However, for AF MIMO

relaying system, due to the relaying operation, transceiver design becomes morechallenging.

In this thesis, we start with a fundamental three nodes source-relay-destinationMIMO system The forwarding matrix at relay and equalizer at destination arejointly designed, under the realistic scenario that channel estimates in both hopcontains Gaussian error Two robust design algorithms are proposed to minimizethe mean-square-error (MSE) of the output signal at the destination The first oneis an iterative algorithm with its convergence proved analytically The other is anapproximated closed-form solution with much lower complexity than the iterativealgorithm.

Next, we consider the AF MIMO orthogonal frequency division multiplexing(OFDM) system over frequency selective fading channels Again, the forwardingmatrix at relay and equalizer at destination are jointly designed by minimizing thetotal MSE of the output signal at the destination, under channel estimation errors.However, since OFDM is a multicarrier modulation, transceiver design in such sys-tem involves power allocation in both spatial and frequency domains, and thus ismore complicated than the first system In the proposed solution, the second-ordermoments of channel estimation errors in the two hops are first deduced in the fre-quency domain Then, the optimal designs for both correlated and uncorrelatedchannel estimation errors are investigated The relationship between the proposedsolutions with existing algorithms is also disclosed.

Finally, we consider the AF MIMO relaying system with multiple users It responds to the case where one base station communicates with multiple terminalsvia one relay station In this system, the source precoder, relay forwarding matrixand destination equalizer are jointly designed by minimum MSE criterion Bothuplink and downlink cases are considered It is found that the uplink and downlinktransceiver designs share some common features and can be solved by a generaliterative algorithm On the other hand, another proposed algorithm for fully loadedor overloaded uplink system is shown to include several existing results as specialcases.

cor-Chapter 1Introduction

1.1 Background

1.1.1 Cooperative Communication

Recently, cooperative communication has gained significant research interest,due to its great potential to improve reliability, coverage and capacity of wirelesslinks [1, 2] Exploiting relay, cooperative communication can take advantage of thespacial diversity provided by relay to improve the communication between sourceand destination As Fig 1.1 shows, when the direct link between source and destina-tion is not good enough, source can rely on relay to communicate with destination.This is a simple application of cooperative communication In cellular networksdesign, deployment of relay station to enlarge coverage of base station is one of themost important parts in future communication protocols, e.g., LTE, IMT-Advancedand Winner project [3, 4].

Generally speaking, there are three kinds of cooperative protocols: forward (AF), decode-and-forward (DF) and compress-and-forward (CF) For AFstrategy, the relay only amplifies and forwards the received signal from the sourceto the destination, without knowing what it has received While for DF strategy,the relay will decode the received data and then retransmit it to the destination.

Figure 1.1 Cooperative communication system.

In compress-and-forward strategy, the data before retransmission at relay will becompressed first to reduce the redundance.

Among these three protocols, AF is a relatively simple one It does not needto know the detailed transmission information at source, such as the modulationscheme, channel coding strategy, or source coding standard All operations at relayare only taken on analogue signal level, so AF strategy is also named as analoguerelay scheme Because of its simplicity and low implementation complexity, AFstrategy has attracted a lot of researchers’ attention [5–14] Meanwhile, AF coop-erative communication has small time delay and high security, thus also welcomedby wireless industries, and taken as a candidate of the relay strategies in futurecooperative communication protocols [3, 4].

1.1.2 Multiple-Input Multiple-Output Systems

It is well-known that in fully scattered environment, multi-antenna systems vide substantial spatial diversity and multiplexing gains [15–17] With multipleantennas at both transmitter and receiver, multiple data streams can be simultane-ously transmitted without any increase in frequency or time resources as shownin Fig 1.2 Multiple-input multiple-output (MIMO) system is a great success inwireless communication theory research in the past decades [15–22] It is also oneof the most important technologies for the third generation mobile communicationsystems and beyond.

Figure 1.2 MIMO system.

To exploit the benefits of MIMO systems, transceiver design for MIMO systemsis of great meaning and has been extensively studied [15–17, 19, 21, 22] Propertransceiver design can greatly improve the performance of a MIMO system andcan even reduce its sensitivity to channel estimation errors From different designpurposes or to tackle different problems, the criteria of transceiver design can bevarious In general, there are two criteria for transceiver design: capacity maxi-mization and data mean-square-error (MSE) minimization The first one mainlyfocuses on maximizing the throughput between source and destination Usually, itaims at the problem how much information can be transmitted over wireless chan-nel under a transmit power constraint On the other hand, the design based on thesecond criterion tries to solve the problem how accurate a transmit signal can berecovered from the received signal Their relationship has been revealed in a land-mark paper [23]: capacity maximization transceiver design can be interpreted as aminimum weighted MSE transceiver design.

From signal processing perspective, in order to minimize the data estimation ror from the received signal, MSE is a very important metric for transceiver design[23–26] Furthermore, based on implementation consideration, linear minimummean-square-error (LMMSE) transceiver is more preferable compared to its non-linear counterparts which may have prohibitive complexity Therefore, LMMSE

Figure 1.3 Single-user AF MIMO relay system.

transceiver design for conventional MIMO systems has been extensively studied invarious scenarios in the past decade [23–26].

1.1.3 AF MIMO Relay Systems

The benefits of multiple-antenna systems can be directly introduced into AFcooperative communications via deployment of multiple antennas at the source, re-lay and destination, and such AF MIMO relay systems receive a lot of attentionlately However, due to the AF operation at relay, the transceiver design is verychallenging Furthermore, depending on the specific modulation and number ofusers, the transceiver design is of different natures and must be tackled in differentways In this thesis, we address the problem of transceiver design for single-userAF MIMO relay systems, where there is only one source, one relay and one destina-tion [13, 14, 27–32] and multiuser AF MIMO relay systems, where multiple mobileterminals communicate with base station via one relay station [33, 34].

1.2 Research Motivation and Problems to be Tackled1.2.0.1 Single-user AF MIMO relay systems

In a classical single-user AF MIMO relay system as shown in Fig 1.3, thereis one source, one relay and one destination This case corresponds to a practicalscenario when one mobile terminal wants to communicate with base station via the

Relay station

Base stationMobile terminals

Figure 1.4 Multi-user AF MIMO relay system.

help of a relay station It is the basis of the more complicated AF MIMO relaysystems.

Capacity bounds under different channel state information (CSI) assumptionsfor single-user AF MIMO relay systems have been derived in [27], [28] Transceiverdesign maximizing the capacity of single-user AF MIMO relay systems based onperfect CSI assumption have been studied in [13, 14, 29].

In terms of transceiver design minimizing MSE, for dual-hop AF MIMO relaysystems with single relay, the optimal closed-form solution for joint optimal for-warding and equalizer matrices assuming perfect CSI has been proposed in [30] Ithas also been shown in [30] that the joint design has a better performance than thevarious separate design schemes Furthermore, the joint source precoder, relay for-warding matrix and destination equalizer design for dual hop MIMO-OFDM relaysystems under perfect CSI is proposed in [31] However, there is no closed-formsolution and an iterative algorithm is proposed.

Notice that the existing algorithms on LMMSE transceiver design for user AF MIMO relay systems require the CSI to be perfectly known Unfortu-nately, in practice, CSI is generally obtained through estimation and perfect estima-tion is very difficult to achieve Due to limited length of training sequences and/or

single-time-varying nature of wireless channels, channel estimation errors inevitably ist, causing substantial system performance degradation Robust transceiver design,which could mitigate such performance degradation by taking the channel estima-tion errors into account, is therefore of great importance and highly desirable forpractical applications.

ex-When channel uncertainties are considered, both min-max and stochastic cluding both probability-based and Bayesian) approaches can be employed Ifquality-of-service (QoS) requirement is considered (e.g., outage probability level[35, 36]), min-max or probability-based approach is preferred On the other hand,if the goal is to minimize an average objective function over channel uncertainties,e.g., the total MSE of multiple data streams, Bayesian approach is more suitable.

(in-In this thesis, Bayesian robust LMMSE transceiver design for single user AFMIMO relay systems (including both single carrier and multi-carrier systems) isinvestigated Several existing algorithms can be considered as special cases of ourproposed solutions.

1.2.0.2 Multi-user AF MIMO relay systems

One of the most important application scenarios of cooperative communicationsis cellular network Due to shadowing or deep fading of wireless channels, the basestation may not be able to sufficiently cover all mobile terminals in a cell, especiallythose on the edge Deployment of relay stations is an effective and economic wayto improve the communication quality in cellular networks, as shown in Fig 1.4.

With multiple antennas at mobile terminals, relay station and base station, anatural question is how to allocate limited power resource in the spatial domain.In general, power allocation is equivalent to beamforming matrices design at base

station, relay and mobile terminals, and the objective can also be maximizing pacity [37] or minimizing the MSE of the recovered data [38] The MSE criterion isa widely chosen one since it aims at the recovered data to be as accurate as possible,and is extensively used in transceiver design in classical point-to-point [16, 17, 23]or multiuser MIMO systems [39–46] The MSE minimization is also related to ca-pacity maximization in the multiuser scenario [44] if a suitable weighting is appliedto different data streams.

ca-In a cellular network, the base station and relay station are usually allowed tobe equipped with multiple antennas For each mobile terminal, if it is equippedwith single antenna, such relay cellular networks has been investigated from vari-ous point-of-views For example, transceiver design for capacity maximization hasbeen considered in [33], and quality-of-service based transceiver design has beeninvestigated in [34] However, in the next generation multi-media wireless com-munications, it is likely that the size of a mobile terminal allows multiple antennasto be deployed Unfortunately, extension from the previous works on single an-tenna mobile terminals to multi-antenna terminals is by no mean straightforward.The transceiver design becomes more challenging as it needs to allocate the poweramong different users and even among different data streams in the same user Themutual interference from different users should also be properly managed.

In this thesis, LMMSE transceiver design for AF MIMO relaying cellular work is investigated, in which base station, relay station and mobile terminals areall equipped with multiple antennas Precoder at source, forwarding matrix at re-lay and equalizer at destination will be jointly designed Both uplink and downlinkscenarios are considered.

net-1.3 Organization and Contributions of the Thesis

This thesis is organized as follows:

Chapter 2—Robust Transceiver Design for AF MIMO Relay Systems: In thischapter, the problem of robust linear relay precoder and destination equalizerdesign for a dual-hop AF MIMO relay system, with Gaussian random channeluncertainties in both hops is investigated By taking the channel uncertaintiesinto account, for AF MIMO relay systems two robust design algorithms areproposed to minimize the mean-square-error (MSE) of the output signal at thedestination One is an iterative algorithm with its convergence proved analyt-ically The other is an approximated closed-form solution with much lowercomplexity than the iterative algorithm Although the closed-form solutioninvolves a minor relaxation for the general case, when the column covariancematrix or the row coveriance of the channel estimation error at the secondhop is proportional to the identity matrix, no relaxation is needed and theproposed closed-form solution is the optimal solution.

Chapter 3—Robust Transceiver Design for AF MIMO-OFDM Relay tems: In this chapter, joint design of linear relay precoder and destinationequalizer for dual-hop non-regenerative AF MIMO-OFDM systems underchannel estimation errors is investigated Second-order moments of chan-nel estimation errors in the two hops are first deduced Then based on theBayesian framework, joint design of linear robust precoder at the relay andequalizer at the destination is proposed to minimize the total mean-square-error (MSE) of the output signal at the destination The optimal designs for

Sys-both correlated and uncorrelated channel estimation errors are considered.The relationship with existing algorithms is also disclosed.

Chapter 4—LMMSE Transceiver Design for AF MIMO Relay Cellular works: In this chapter, linear transceiver design for AF relaying cellular net-works is considered, in which base station, relay station and mobile terminalsare all equipped with multiple antennas The design is based on minimumMSE criterion, and both uplink and downlink scenarios are considered Itis found that the downlink and uplink transceiver design problems are in thesame form, and iterative algorithms with the same structure can be used tosolve the design problems For the specific cases of fully loaded or overloadeduplink systems, a novel algorithm is derived and its relationships with severalexisting transceiver design algorithms for conventional MIMO or multiusersystems are revealed Simulation results are presented to demonstrate theperformance advantage of the proposed design algorithms.

Net-Chapter 5—Conclusions and Future Research: We conclude our work in thischapter, and describe some challenges concerning future research directions.

1.4 Commonly Used Notations

The following notations are used throughout this thesis Boldface lowercaseletters denote vectors, while boldface uppercase letters denote matrices The nota-tion ZHdenotes the Hermitian of the matrix Z, and Tr(Z) is the trace of the matrixZ The symbol IMdenotes an M × M identity matrix, while 0M,N denotes an

M × N all zero matrix The notation Z1/2 is the Hermitian square root of the itive semidefinite matrix Z, such that Z1/2Z1/2 = Z and Z1/2 is also a Hermitian

pos-matrix The operation diag{[A B]} denotes a block diagonal matrix with A and

B as block diagonal On the other hand, d{Z} denotes a vector consisting of thediagonal elements of Z The symbol E{.} represents the expectation operation.

The operation vec(Z) stacks the columns of the matrix Z into a single vector The

symbol ⊗ denotes the Kronecker product.

Relay DestinationSource

Figure 2.1 Amplify-and-forward MIMO relay diagram.

Furthermore, the proposed closed-form solution has a comparable performance tothe proposed iterative algorithm, but with a lower complexity.

This section is organized as follows System model is presented in Section 2.2.The optimization problem for minimizing the total MSE is formulated in Sec-tion 2.3 In Section 2.4, an iterative algorithm is proposed to solve the optimizationproblem, while an approximated closed-form solution is given in Section 2.5 InSection 2.6, the iterative and closed-form solutions are further generalized to theweighted MSE criterion Finally, simulation results are given in Section 2.7 andconclusions are drawn in Section 2.8.

2.2 System Model

In this chapter, a dual-hop amplify-and-forward (AF) cooperative

communica-tion system is considered In the considered system, there is one source with NSantennas, one relay with MRreceive antennas and NR transmit antennas, and one

destination with MD antennas, as shown in Fig 2.1 At the first hop, the sourcetransmits data to the relay The received signal, x, at the relay is

where s is the NS× 1 data vector transmitted by the source with the covariance

matrix Rs= E{ssH} The matrix Hsr is the MIMO channel matrix between the

source and the relay Symbol n1 is MR× 1 Gaussian noise vector with covariance

matrix Rn1 At the relay, the received signal x is multiplied by a precoder matrixF, under a power constraint Tr(FRxFH) ≤ Pr where Rx = E{xxH} and Pr

is the maximum transmit power Then the resulting signal is transmitted to thedestination The received signal at the destination, y, can be written as

y = HrdFHsrs + HrdFn1+ n2, (2.2)where Hrdis the MIMO channel matrix between the relay and the destination, andn2 is the additive Gaussian noise vector at the second hop with covariance matrixRn2 In order to guarantee the transmitted data s can be recovered at the destination,

it is assumed that MR, NR, and MDare greater than or equal to NS [10].

It is assumed that both the relay and destination have the estimated channel stateinformation (CSI) When channel estimation errors are considered, we have

unit variance The MR× MR matrix Σsrand NS× NS matrix ΨT

sr are the rowand column covariance matrices of ∆Hsr, respectively [16] It is easy to see thatvec(∆HT

sr) ∼ CN (0MRNS×1, Σsr⊗ ΨTsr), where CN (m, C) denotes a complex

Gaussian random vector with mean m and covariance C Furthermore, the matrix∆Hsr is said to have a matrix-variate complex Gaussian distribution, which can be

where the MD× MDmatrix Σrdand NR× NRmatrix ΨT

rdare the row and columncovariance matrices of ∆Hrd, respectively It is assumed that Hsr and Hrd areestimated independently, so the channel estimation errors, ∆Hsr and ∆Hrd, areindependent.

Remark 1: In general, the expressions of Ψsr, Σsr, Ψrd and Σrd depend onspecific channel estimation algorithms If the channel estimation algorithm pro-posed in [49] is used, we have Ψsr = RT,sr, Σsr= σ2

e,srRR,sr, Ψrd = RT,rdand Σrd= σ2

e,rdRR,rd The matrices RT,sr and RR,sr are the transmit and ceive antennas correlation matrices at the source and the relay, respectively, and

e,sr(IMR+ σ2

e,srR−1R,sr)−1, Ψrd= RT,rdandΣrd= σ2

R,rd)−1 Notice that for the channel estimation algorithmsin [49, 50], Ψsr, Σsr, Ψrd and Σrd are functions of the second-order statistics ofCSI, which can be considered to change very slowly and to be known a priori [5].

However, in the following, the proposed algorithms are developed without ing any specific form of Ψsr, Σsr, Ψrdand Σrd.

assum-2.3 Problem Formulation

At the destination, a linear equalizer G is adopted to detect the transmitted datas (see Fig 2.1) The problem is how to design the linear precoder matrix F at therelay and the linear equalizer G at the destination to minimize the mean-square-error (MSE) of the received data at the destination:

MSE(F, G) = E{Tr¡(Gy − s)(Gy − s)H¢

where the expectation is taken with respect to s, ∆Hsr, ∆Hrd, n1 and n2.

Since s, n1and n2are independent, the MSE expression (2.7) can be written as

For the inner expectation, due to the fact that the distribution of ∆Hsr is variate complex Gaussian with zero mean, the following equation holds [47]

rd)GH). (2.11)Similarly, the second term of MSE in (2.8) can be simplified as

Notice that the matrix Rxis the autocorrelation matrix of the receive signal x at therelay.

Subject to the transmit power constraint at the relay, the joint design of izer at the destination and precoder at the relay can be expressed as the followingoptimization problem

2.4 The Proposed Iterative Algorithm

In this section, we derive an iterative algorithm [40, 51] to solve for F and G.In the following, it is shown that given F, there is a closed-form solution for G,and vice versa Therefore, the proposed algorithm computes F and G iteratively,starting with an initial value.

i+1 The KKT conditions can be shown to be

Fi+1= ( ¯HHrdGHi+1Gi+1H¯rd+ Tr(Gi+1ΣrdGHi+1)Ψrd+ γi+1I)−1H¯HrdGHi+1RsH¯HsrR−1x ,

iter-Obviously from (2.19a), in order to compute the optimal Fi+1, the Lagrangian

multiplier γi+1should be calculated first However, there is no closed-form solution

of γi+1simultaneously satisfying (2.19b) and (2.19c) [40] Below we propose a lowcomplexity method to solve (2.19b) and (2.19c) First, notice that in order to have

(2.19b) satisfied, either γi+1 = 0 or Tr(Fi+1RxFH

i+1) = Prmust hold If γi+1 = 0

also makes (2.19c) satisfied, γi+1 = 0 is a solution to (2.19b) and (2.19c) Sincegiven Gi+1, the optimization problem (2.16) is a convex quadratic programmingproblem of Fi+1, which has only one solution for Fi+1, γi+1= 0 is the only solution

to (2.19b) and (2.19c) in this case On other hand, if γi+1= 0 does not make (2.19c)satisfied, we have to solve Tr(Fi+1RxFH

i+1) = Pr It is proved in Appendix 2.9 thatwhen Gi+1is fixed, the function

i+1) ≤ Pr, (2.19c) is satisfied automatically

in this case In summary, the proposed procedure for calculating γi+1 is given asfollows:

γi+1 =

Solve q(γi+1) = Prusing bisection algorithm Otherwise

. (2.22)

2.4.3 Summary and convergence analysis

The proposed iterative algorithm proceeds between (2.18) and (2.19a), whichcan be summarized as Algorithm 2.1 This iterative algorithm can be shown toAlgorithm 2.1 The iterative algorithm for joint design of F and G

Initialize the algorithm with F0 where the N × N principle sub-matrix of F0

Tr(Rx)IN, and N = min{NR, MR}, while the remaining entries of F0

being zeros.repeat

Update Giusing (2.18);

Compute γiusing (2.22);Update Fi using (2.19a);

Although the iterative algorithm results in precoder and equalizer design thatgives satisfactory performances, it requires high complexity because of iterations.Furthermore, in practice, it is not known in advance how many iterations are needed

for the iterative algorithm to converge In the next section, we propose a form solution, which approximately solves (2.16) in the general case, but exactlywhen Ψrd∝ INR In the simulation, we find that the closed-form solution has acomparable performance to that of the iterative algorithm.

closed-Remark 2: As mentioned previously, when Gi+1 is given, the optimizationproblem (2.16) is a convex quadratic programming problem for Fi+1, which canbe reformulated as a semi-definite programming (SDP) problem and solved byinterior-point polynomial algorithms [54] But these algorithms have a much highercomplexity compared to the proposed algorithm based on KKT conditions (2.19a)-(2.19c).

2.5 The Proposed Closed-Form Solution

Since the constraint in the problem (2.16) does not involve the equalizer G, theoptimal G can be directly derived as a function of F, by differentiating the objective

function MSE(F, G) with respect to G∗ and setting the result to zero This resultsin

G = Rs( ¯HrdF ¯Hsr)H( ¯HrdFRxFHH¯Hrd+ K)−1, (2.23)where K = Tr(FRxFHΨrd)Σrd+ Rn2 was previously defined in (2.15) Substi-tuting (2.23) into the MSE expression (2.13), the MSE can be written as