1

Deterministic Models
A Tutorial Review Jing Huang Department of Electrical Engineering University of Notre Dame, Notre Dame, Indiana 46556

I. I NTRODUCTION One objective of information theory is to characterize the capacity regions of different channels. Unfortunately, the capacity regions of certain multi-user scenarios have been open for decades and are one of the most important problems in multiuser information theory. Even for the comparatively simple twouser Gaussian interference channel, the literature has made slow progress towards fully characterization of the capacity region. On the other hand, the literature has made signicant progress in characterizing the capacity regions if the outputs are deterministic functions of the inputs. The capacity regions of a class of deterministic discrete memoryless interference channels are established as early as 1982 in [1]. Recently, more attention has been paid to deterministic models, since the literature has found certain important connections between the capacity regions of some deterministic channels and the corresponding deterministic models. For example, [2] shows that the deterministic channel uniformly approximates the Gaussian channel, and the capacity regions only differs by a universal constant. The deterministic channel model can provide deep insight into the structure and property of some more realistic, but also more complicated, channel models. This is because that the deterministic model is often easier to analyze due to its simplicity. Furthermore, the capacity region of the deterministic model can often be characterized exactly. The deterministic model has been widely used recently in the literature to help in characterizing the capacity region of different channels. The deterministic interference channel is rst proposed by Tse et

al. in [3] to aid the analysis of Gaussian relay networks, and later used in [2] to show that the high
SNR behavior of the Gaussian interference channel can be fully captured by the deterministic interference channel. The deterministic model can also be used to characterize the capacity regions of many-to-one and one-to-many interference networks [4] and the fading broadcast channel [5]. The rest of this tutorial is organized as follows. Section II introduces several common channel models and the corresponding deterministic channel models. Section III summarizes the capacity regions of several

channel models. II. D ETERMINISTIC M ODEL This section presents several deterministic channel models for some common channel models that are frequently used in the literature of information theory. The section begins by introducing the most popular point to point AWGN channel and the idea of how to build the corresponding deterministic model. This idea can let us gain an intuitive understanding of the connections between a Gaussian channel and its corresponding deterministic model. The section then describes several other commonly used channel models with the corresponding deterministic models. In the following, x, y and z are input, output and additive while Gaussian noise respectively. A. Deterministic Point-to-point AWGN channel Consider the well-known point to point AWGN channel y= SNRx + z,
1 N N i=1

where z CN (0, 1) and the input satises an average power constraint

E[x2 ] 1. The following of i

this subsection introduces how to get the deterministic point to point AWGN channel. Let us rewrite the output signal y in terms of the binary expansions of x and z. For simplicity assume x and z are positive, we have y = 2 2 log SN R
i=1
1

x(i)2i +
i=

z(i)2i ,

Now assume there is a peak power constraint to the background noise to simplify the noise effect, i.e., E[|z|2 ] 1, we have y=2 By simple manipulation, we have
n1
1 2

log SN R i=1

x(i)2

+
i=1

z(i)2i ,

y 2n1 where n1 = 1 log SNR+ . 2

x(i)2i +
i=1 i=1

(x(i + n1 ) + z(i))2i ,

(1)

Therefore, if we simply ignore the 1 bit of the carry-over from the second summation term, we can model the point to point AWGN channel as a pipe that truncates the input signal x and only pass the bits that are above the noise level. This idea is illustrated in g. 1. Therefore, think that the transmitted signal x is a sequence of bits at different signal levels, with the highest signal level being the most signicant bit and the lowest level being the least signicant bit. Let

Fig. 1. The deterministic point to point AWGN channel. Each bit of the input occupies a signal level. Bits of lower signicance are lost due to noise.

the signal x be interpreted as occupying a succession of levels [2]: x = 0.b1 b2 b3 b4 b5 ..., where the input in written in binary. The most signicant bit coincides with the highest level, the least signicant bit with the lowest level, and a level corresponds to a unit of power. Noise is modeled in the deterministic channel by truncation. Bits of smaller order than the noise are lost. As we notice in equation (1), if we ignore the 1 bit of the carry-over in the section summation term as described before, there will be no background noise any more and the channel model become a deterministic model shown in g. 1. The deterministic point to point AWGN channel can be written as y = 2n2 x, where n2 = log SNR. B. Deterministic MAC Consider the two-user multiple-access channel y = h1 x1 + h2 x2 + z, where z CN(0, 1), and each input satises an average power constraint The corresponding deterministic channel model can be written as y = 2n1 x1 2n2 x2 ,
1 N N k=1

dene the signal-to-noise ratios as SNR1 = |h2 |P1 , SNR2 = |h2 |P2 , and we assume that SNR1 SNR2 . 1 2

E[x2 ] Pi , i = 1, 2. We i,k

where addition is performed on each bit (modulo two) and is the integer-part function. in an alternative form y = Sqn1 x1 Sqn2 x2 , where summation and multiplication are in F2 and S 0 0 1 0 S = 0 1 . . . 0 0 C. Deterministic Interference Channel Consider the Gaussian interference channel given by y1 = h11 x1 + h12 x2 + z1 y2 = h21 x1 + h22 x2 + z2 where zi CN(0, 1) and the channel inputs satisfy an average power constraint
1 N N k=1

Now let the input and output be x1 , x2 , y Fq , where q = max(n1 , n2 ). The channel can be written 2

is a q q shift matrix dened as ... 0 ... 0 0 . . .. . . . ...1 0

i = 1, 2. We dene the signal-to-noise ratios as SNR1 = |h2 |P1 , SNR2 = |h2 |P2 and interference-to11 22 noise ratios as INR1 = |h2 |P1 , INR2 = |h2 |P2 . 21 12 The output at receiver i of the deterministic interference channel is yi(t) = Sqni1 x1 (t) Sqni2 x2 (t), where summation and multiplication are in F2 and S is dened as above.

E[x2 ] Pi , i,k

The deterministic interference channel is relatively simple, yet retains two essential features of the Gaussian interference channel: the loss of information due to noise, and the superposition of transmitted signals at each receiver. The relevance of the deterministic model is greatest in the high-SNR regime, where communication is interferencerather than noiselimited; however, we shall see that even for nite signal-to-noise ratios the deterministic channel model provides signicant insight towards the more complicated Gaussian model [4].

D. General Deterministic Network Consider a directed network G = (V, E), where the received signal yj at node j V is given by yj = gj (xiiNj ), where we dene the input neighbors Nj of j as the set of nodes whose transmissions affect j, and can be formally dened as Nj = i : (i, j) E. E. Linear Finite-Field Deterministic network A special deterministic model motivated by its close connection to the Gaussian model is the linear nite-eld model, where the received signal yj Fq is a vector dened over a nite eld Fp given by p yj =
iV

Gi,j xi ,

where the transmitting signals xk Fq , and the channel matrices Gi,j Fqq . All the operations are done p p over the nite eld Fp , and the network G. III. C APACITY R EGION This section illustrates the connections between characterizing the capacity regions of the deterministic model and their Gaussian counterpart. The section mainly focuses on the recent works by Tse et al. on the two-user Gaussian interference channel and Gaussian relay networks. These works build insights from the deterministic models to help in analyzing the corresponding Gaussian models. A. Two-user Multiple-access Channel This subsection directly illustrates the connections between the capacity regions of the two-user MAC and the deterministic MAC. The capacity region of the MAC is R1 log(1 + SNR1 ) log(SNR1 ) R2 log(1 + SNR2 ) log(SNR2 ) R1 + R2 log(1 + SNR1 + SNR2 ) log(SNR1 ) The capacity region of the deterministic MAC is R1 n1 R2 n2 R1 + R2 max(n1 , n2 ).

Comparing with the capacity region of the Gaussian MAC, we can make the correspondence n1 = log SNR1 , n2 = log SNR2 . The simple two-user multiple-access channel case illustrates the idea that one can attempt to reduce the Gaussian problem to a deterministic one by proving a constant gap between the capacity regions of the two models.

B. Two-user Gaussian Interference Channel The following theorems for the two-user Gaussian interference channel are proved in [2]. Theorem 1. The capacity of the two-user Gaussian interference channel with signal and interference to noise ratios SNR1 ,SNR2 , INR1 , INR2 is within 42 bits per user of the capacity of a deterministic interference channel with gains 2n11 = 2log SN R1 , 2n12 = 2log IN R2 , 2n21 = 2log IN R1 , and 2n22 = 2log SN R2 . Theorem 2. The capacity of the two-user Gaussian interference channel with signal and interference to noise ratios SNR1 ,SNR2 , INR1 , INR2 is within 18.6 bits per user of the capacity of a deterministic interference channel with gains 2n11 = 2 2 log SN R1 , 2n12 = 2 2 log IN R2 , 2n21 = 2 2 log IN R1 , and 2n22 = 2 2 log SN R2 . Theorem 2 is the same as Theorem 1 but for the real-valued Gaussian interference channel, while Theorem 1 is for complex-valued case. The main ingredients used in the proofs of Theorem 1 and Cdet Cgaussian + constant. For example, the proof of Theorem 2 can be done in this direction, i.e., Cgaussian Cdet + (13.6, 13.6) and Cdet Cgaussian + (5, 5). Note that the 18.6 bits in Theorem 2 comes from the addition of 5 bits and 13.6 bits. Each direction is completed in a sequence of steps. Each step compares the capacity region of a new channel to that of the previous step. The rst and the last channels with be either Gaussian or deterministic channels. Consider the proof of Cdet Cgaussian + (5, 5) for Theorem 2 as an example. The rst step shows that the capacity region does not decrease if the modulo 2 addition of the deterministic channel is replaced by real addition. Step 2 shows that the capacity region of the deterministic channel is the same if the gain 2nij is replaced by a real-valued hij with nij = log |hij |. Step 3 adds Gaussian noise. Detailed proof will not be listed here for space constraint. C. Gaussian Relay Networks This subsection focuses on the recent comprehensive work by Tse et al. on Gaussian relay networks [3]. This work starts with deterministic models to build insights and the use them as foundation to study the information ow in the Gaussian models. With the insights from the well-analyzed deterministic models, the work proposes a new scheme for general Gaussian relay networks called quantize and forward, which is motivated from the deterministic relay network, and a proof that it can achievable to within a constant gap to the cut-set bound [6] with no SNR limitations. As a denition, a cut-set bound upperbounds the reliable transmission rate of a relay network. The capacity of a relay network is dened as the supremum of all achievable rates of reliable communication from the source to the destination. The following theorems are the main results of this work. Theorem 2 are the same. Each proof requires two directions, namely, Cgaussian Cdet + constant and
1 1 1 1

Theorem 3. A rate of min rank(G,c ).

max
iV

p(Xi ) D

min H(Yc |Xc ) can be achieved on a deterministic network.

Theorem 4. The capacity C of a linear nite-eld deterministic relay network is given by C =

D

Extension to the multicast case of Theorem 3 and Theorem 4 is also provided in [3]. Theorem 5. Given a Gaussian relay network, we can achieve all rates R up to C . Therefore the capacity of this network satises C C C, where C is the cut-set upper bound on the capacity of the network, and is a constant and is upper bounded by
|V| i=1 |V| i=1 max(Mi , Ni )),

max(Mi , Ni )(15+log

where Mi and Ni are respectively the number

of transmit and receive antennas at node i. R EFERENCES

[1] A. E. Gamal and M. Costa, The capacity region of a class of deterministic interference channels, IEEE Trans. Inform. Theory, vol. 2, pp. 343C346, March 1982. [2] G. Bresler and D.N.C. Tse, The Two-User Gaussian Interference Channel: A Deterministic View, European Transactions on Telecommunications, June 2008. [3] S. Avestimehr, S. Diggavi, and D.N.C. Tse, Wireless Network Information Flow: A Deterministic Approach, submitted to IEEE Trans. Inform. Theory, June 2009 (http://arxiv.org/abs/0906.5394). [4] G. Bresler, A. Parekh, and D. Tse, The approximate capacity of the many-to-one and one-to-many Gaussian interference channels, In Allerton Conference on Communication, Control, and Computing, Monticello, IL, September 2007. [5] D. Tse and R. Yates, Fading broadcast channels with state information at the receivers, Allerton Conference on Computing and Control, Sep 2008. [6] T. M. Cover and J. A. Thomas, Elements of Information Theory, New York, Wiley, 1991.