第4章-信道容量ppt课件.ppt

《第4章-信道容量ppt课件.ppt》由会员分享，可在线阅读，更多相关《第4章-信道容量ppt课件.ppt（62页珍藏版）》请在淘文阁 - 分享文档赚钱的网站上搜索。

1、Chapter 4.Channel and Channel Capacity Information Theory and Coding严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Review property of mutual info.function:Property.1Relationship between average mutual info.and channel input probability distribution Property 1:I(X;Y)is an upper convex functi

2、on of the channel input probability distribution p(x).I(X;Y)p(x)严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Review property of mutual info.function:Property.2 Relationship between Info.content and channel transition probability distribution Property 2:I(X;Y)is a concave function of chann

3、el transition probability distributes p(y/X).I(X;Y)p(y/x)严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。What is channel?The channel is a carrier transmitting messages a passage through which signal passes.The information is abstract,but the channel is concrete.For instance:If two people con

4、verse,the air is the channel;If the two call each other,the telephone line is the channel;If we watch the television,listen to the radio,the space between the receiver and the transmitter is the channel.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.1.The model and classification of the c

5、hannel In this part,we will mainly introduce two parts:Channel models Channel classifications 严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.1.1 Channel Models we can treat channel as a converter which transfer events.the channel model can be indicated as the follow Fig:严格执行突发事件上报制度、校外活动报

6、批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Binary Symmetric Channel(BSC)is the simplest channel modelA BSC is shown as below:BSC 严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。DMCWe assume that the channel and the modulation is memoryless.The inputs and outputs can then be related by a set of co

7、nditional probabilities严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。DMCThis channel is known as a Discrete Memoryless Channel(DMC)and is depicted as严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.1.2 Channel classifications Channel can be classified into several types.严格执行突发事件上报

8、制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。(对流层)(电离层)严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.2 Channel doubt degree and average mutual information 4.2.1 Channel doubt degree 4.2.2 Average mutual information 4.2.3 Properties

9、 of mutual information function4.2.4 Relationship between entropy,channel doubt degree and average mutual information严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.2.1 Channel doubt degree Assume r.v.X indicates the input set of channel,and r.v.Y indicates the output set of channel,the ch

10、annel doubt degree is:The meaning of“channel doubt degree H(X|Y)”is that when the receiving terminal gets message Y,the average uncertainty still leaves about source X.In fact,the uncertainty comes from the noise in channel.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。This means that if t

11、he average uncertainty of source X is H(X),well get more or less information which eliminates the uncertainty of the source X when get the output message Y.So we have the following concept of average mutual information.Since we have:严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.2.2 Avera

12、ge mutual information The average mutual information is the entropy of source X minus the channel doubt degree.The above meaning is that when the receiver gets a message Y,the average information he can get about X from every symbol he received.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件

13、。4.2.3 Properties of mutual information function Property 1:Relationship between mutual information and channel input probability distribution.I(X;Y)is an upper convex function of the channel input probability distribution P(X).This can be shown in Fig.4.5 and Fig.4.6.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到

14、及时发现、制止、汇报并处理各类违纪行为或突发事件。Fig.4.5.I(X;Y)is convex function of P(X)Fig.4.6.Message passing through the channel严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。E.g.4.1 Considering a dual element channel,the probability distribution isand the matrix of channel is Where is the probability of trans

15、mission error.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Then the mutual information is,严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。And we can get the following results,So,The average mutual information diagram is shown in the following Fig.4.7.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做

16、到及时发现、制止、汇报并处理各类违纪行为或突发事件。Fig.4.7.Mutual information of the dual symmetric channel From the diagram,we can see that when the input symbols satisfy“equal probability distribution”,the average mutual information I(X;Y)reaches the maximum value,and only at this time the receiver gets the largest inform

17、ation from every symbol he received.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Property 2 Relationship between information and channel transition probability distribution.I(X;Y)is a concave function of channel transition probability distribution of p(Y|X).Fig.4.8.I(X;Y)is a concave func

18、tion of P(X|Y)严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。E.g.4.2(This is the follow-up of E.g.4.1)Considering dual channel,now we know the average mutual information is,when the source distribution isaverage mutual information I(X;Y)is the concave function of p,just see it from the foll

19、owing diagram,the严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Mutual info.of fixed binary source From the diagram,we can see,once the binary source fixed,when the channel property p changes,well get the different mutual information I(X;Y),when p=1/2,I(X;Y)=0,that means the receiver get th

20、e lest information from this channel,and all the information is lost in the way of transmission,this channel has the most loudly noise.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Property 3 If the channel input is discrete and without memory,we have the following inequalityProperty 4 If

21、the channel is discrete and without memory,we have(Remember the results)严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.2.4 Relationship between entropy,channel doubt degree and average mutual information 严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。E.g.4.3 There is a source Its

22、 messages pass through a channel with noise.The symbols received by the other end of the channel are .The channels transfer matrix is ,please calculate(1)The self-information included in the symbolandof event.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。(2)The information about the receiv

23、er gets when it observes the message(3)The entropy of source X and received Y(4)The channel doubt degree H(X|Y)and the noise entropy H(Y|X).(5)The average mutual information got by receiver when it receives Y.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Solution:(1):(2):严格执行突发事件上报制度、校外活动报

24、批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。(3):严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。(5):(4):严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.3 Discrete channel without memoryThree groups of variables to describe the channel:(1)Channel input probability space,(2)Channel output

25、probability space,(3)Channel transfer probability,So,the channel can be represented by严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。This can be indicated by the following illustration,and the channel transfer matrix is 严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。When K=1,it deg

26、enerates to the single message channel;and when n=m=2,it degenerates to the binary single message channel.If it satisfies symmetry,it constitutes the most commonly used BSC.Fig.4.11.Binary message symmetrical channel严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。严格执行突发事件上报制度、校外活动报批制度等相关规章制度

27、。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.4 Channel capacity 4.4.1 The concept of channel capacity4.4.2 Discrete channel without memory and its channel capacity4.4.3 Continuous channel and its channel capacity严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.4.1 The concept of channel capacity The capac

28、ity of channel can be defined as the maximum value of average mutual information,The unit of channel capacity C is bit/symbol or nat/symbol严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。From the property mentioned before,we know I(X;Y)is an upper convex function of probability distribution

29、p(x)of input variable X.For a specific channel,there always exists a source which maximizes the information of every message transmitting through the channel.That means the maximum of I(X;Y)exists.And the probability distribution p(x)is called the optimum input distribution.严格执行突发事件上报制度、校外活动报批制度等相关规

30、章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.4.2 Discrete channel without memory and its channel capacity Classification of the discrete message sequence channel严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。For discrete channel without memory,it satisfies the following relationship.According to the“pr

31、operty 4”of the mutual information I(X;Y)of the message sequence,for the discrete channel without memory,we have 严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Note:only when the source is of without memory,the equal relationship in this formula may be satisfied So we can get the following

32、deduction which gets the formula of channel capacity C严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Theorem of discrete channel without memory Assuming that the transmission probability matrix of the discrete channel without memory is Q,the sufficient conditions under which the input lette

33、r probability distribution p*can make the mutual information I(p;Q)to achieve maximum value are Where is the average mutual information when source letter is sent;and C is the channel capacity of this channel.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Understanding this theoremFirstly,u

34、nder this kind of distribution,each letter whose probability is above zero provides mutual information C,and each letter whose probability is zero provides mutually information lower than or equal to C.Secondly,only under this kind of distribution,it may cause I(p;Q)to obtain the maximum value C.Thi

35、rdly,I(X;Y)is the average of .That is to say,it satisfies this equation 严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。(1)If we want to enhance I(X;Y),enhancing p(ak)may be a good idea.(2)However,once p(ak)is enhanced,I(x=ak;Y)may be reduced.(3)To adjust p(ak)repeatedly,make I(x=ak;Y)all eq

36、ual to C (4)This time I(X,Y)=CThe theorem only provides a sufficient condition of to make distribution and the value of C;but it may help to get the value of C of several kinds of channels in simple situation.It does not give the concrete严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。E.g.4.

37、4 Assume the transmission matrix of binary discrete symmetrical channel is(1)If please calculate(2)Please calculate the capacity of channel,and the probability distribution when reaching the capacity of channel.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。(1)严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做

38、到及时发现、制止、汇报并处理各类违纪行为或突发事件。(2)Application Example 3.6Where m represents the number of output symbol set;Hmi is the entropy of the row vector of channel matrix.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.4.3 Continuous channel and its channel capacity Characteristic of continuous channel

39、Analog channelBasic knowledge to addable channel Shannon formulaUsage of Shannon formula严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Characteristic of continuous channelCharacteristic 1:The time is discrete,the value scope is continuous.Characteristic 2:At each moment,it is the single ran

40、dom variable whose value is continuous.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Analog channel严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Basic knowledge to addable channel X:channel input N:channel noiseY:channel output If two of X,Y,N are Gauss distributions,then the oth

41、er is also the Gauss distribution.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。The differential entropy of the r.v.satisfying Gaussian distribution only concerns with its variance and has nothing to do with the average value.Fig.4.22.Addable channel严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇

42、报并处理各类违纪行为或突发事件。TheoremTheorem:When a generally stationary random process source with limited frequency(F)and time(T)passes through a white Gaussian channel which has limited power(PN),the channel capacity is:Shannon formulaThis is the famous Shannon Shannon formula formula for continuous channel.Wh

43、en T=1,the capacity is:严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Proof of Shannon formula:Assume,where X and N are independent discrete r.v.s;and SinceWe haveThe biggest entropy theorem of limited average power严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Proof of Shannon for

44、mula:Due to the limited frequency F for and according to the Nyquist sample theorem,the continuous signal X(t,w)can be equivalent to 2F discrete signals per second.That is:Considering time duration T:严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Fig.4.23.Shannon formula严格执行突发事件上报制度、校外活动报批制

45、度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Usage of Shannon formulaIn analog communications,the frequency modulation surpasses the amplitude modulation;the wider the frequency band,the stronger is the anti-disturbance.In digital communications,the pseudo-noise(PN)code straightforwardly expands the signal.T

46、he wider the band width,the more the frequency increases,and the stronger is the anti-disturbance.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Another form of Shannon formulaWhere N0 is the noise power intensity in unit bandwidth.is the bit energy.Andis the normalization of SNR.When Concl

47、usionConclusion:when the SNR is very low,the channel capacity can be approximately determined by its signal noise ratio.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。E.g.4.6In photo transmission,every frame has about 2.25 106 pixels.In order to reproduce good image,we need about 16 points

48、brightness levels.Assuming equal probability distribution of the brightness levels,please calculate the transmission channel bandwidth requirement of 30 images per second(signal-to-noise ratio is 30dB).严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Solution:The channel capacity of additive white Gaussian noise(AWGN)channel in every unit time is,The required information transmission rate is:Example 3.8(p26)Application example严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。End of Chapter 4