第4章-信道容量ppt课件.ppt
Chapter 4.Channel and Channel Capacity Information Theory and Coding严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Review property of mutual info.function:Property.1Relationship between average mutual info.and channel input probability distribution Property 1:I(X;Y)is an upper convex function of the channel input probability distribution p(x).I(X;Y)p(x)严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Review property of mutual info.function:Property.2 Relationship between Info.content and channel transition probability distribution Property 2:I(X;Y)is a concave function of channel transition probability distributes p(y/X).I(X;Y)p(y/x)严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。What is channel?The channel is a carrier transmitting messages a passage through which signal passes.The information is abstract,but the channel is concrete.For instance:If two people converse,the air is the channel;If the two call each other,the telephone line is the channel;If we watch the television,listen to the radio,the space between the receiver and the transmitter is the channel.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.1.The model and classification of the channel In this part,we will mainly introduce two parts:Channel models Channel classifications 严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.1.1 Channel Models we can treat channel as a converter which transfer events.the channel model can be indicated as the follow Fig:严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Binary Symmetric Channel(BSC)is the simplest channel modelA BSC is shown as below:BSC 严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。DMCWe assume that the channel and the modulation is memoryless.The inputs and outputs can then be related by a set of conditional probabilities严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。DMCThis channel is known as a Discrete Memoryless Channel(DMC)and is depicted as严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.1.2 Channel classifications Channel can be classified into several types.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。(对流层)(电离层)严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.2 Channel doubt degree and average mutual information 4.2.1 Channel doubt degree 4.2.2 Average mutual information 4.2.3 Properties of mutual information function4.2.4 Relationship between entropy,channel doubt degree and average mutual information严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.2.1 Channel doubt degree Assume r.v.X indicates the input set of channel,and r.v.Y indicates the output set of channel,the channel doubt degree is:The meaning of“channel doubt degree H(X|Y)”is that when the receiving terminal gets message Y,the average uncertainty still leaves about source X.In fact,the uncertainty comes from the noise in channel.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。This means that if the average uncertainty of source X is H(X),well get more or less information which eliminates the uncertainty of the source X when get the output message Y.So we have the following concept of average mutual information.Since we have:严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.2.2 Average mutual information The average mutual information is the entropy of source X minus the channel doubt degree.The above meaning is that when the receiver gets a message Y,the average information he can get about X from every symbol he received.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.2.3 Properties of mutual information function Property 1:Relationship between mutual information and channel input probability distribution.I(X;Y)is an upper convex function of the channel input probability distribution P(X).This can be shown in Fig.4.5 and Fig.4.6.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Fig.4.5.I(X;Y)is convex function of P(X)Fig.4.6.Message passing through the channel严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。E.g.4.1 Considering a dual element channel,the probability distribution isand the matrix of channel is Where is the probability of transmission error.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Then the mutual information is,严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。And we can get the following results,So,The average mutual information diagram is shown in the following Fig.4.7.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Fig.4.7.Mutual information of the dual symmetric channel From the diagram,we can see that when the input symbols satisfy“equal probability distribution”,the average mutual information I(X;Y)reaches the maximum value,and only at this time the receiver gets the largest information from every symbol he received.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Property 2 Relationship between information and channel transition probability distribution.I(X;Y)is a concave function of channel transition probability distribution of p(Y|X).Fig.4.8.I(X;Y)is a concave function of P(X|Y)严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。E.g.4.2(This is the follow-up of E.g.4.1)Considering dual channel,now we know the average mutual information is,when the source distribution isaverage mutual information I(X;Y)is the concave function of p,just see it from the following diagram,the严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Mutual info.of fixed binary source From the diagram,we can see,once the binary source fixed,when the channel property p changes,well get the different mutual information I(X;Y),when p=1/2,I(X;Y)=0,that means the receiver get the lest information from this channel,and all the information is lost in the way of transmission,this channel has the most loudly noise.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Property 3 If the channel input is discrete and without memory,we have the following inequalityProperty 4 If the channel is discrete and without memory,we have(Remember the results)严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.2.4 Relationship between entropy,channel doubt degree and average mutual information 严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。E.g.4.3 There is a source Its messages pass through a channel with noise.The symbols received by the other end of the channel are .The channels transfer matrix is ,please calculate(1)The self-information included in the symbolandof event.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。(2)The information about the receiver gets when it observes the message(3)The entropy of source X and received Y(4)The channel doubt degree H(X|Y)and the noise entropy H(Y|X).(5)The average mutual information got by receiver when it receives Y.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Solution:(1):(2):严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。(3):严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。(5):(4):严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.3 Discrete channel without memoryThree groups of variables to describe the channel:(1)Channel input probability space,(2)Channel output probability space,(3)Channel transfer probability,So,the channel can be represented by严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。This can be indicated by the following illustration,and the channel transfer matrix is 严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。When K=1,it degenerates to the single message channel;and when n=m=2,it degenerates to the binary single message channel.If it satisfies symmetry,it constitutes the most commonly used BSC.Fig.4.11.Binary message symmetrical channel严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.4 Channel capacity 4.4.1 The concept of channel capacity4.4.2 Discrete channel without memory and its channel capacity4.4.3 Continuous channel and its channel capacity严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.4.1 The concept of channel capacity The capacity of channel can be defined as the maximum value of average mutual information,The unit of channel capacity C is bit/symbol or nat/symbol严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。From the property mentioned before,we know I(X;Y)is an upper convex function of probability distribution p(x)of input variable X.For a specific channel,there always exists a source which maximizes the information of every message transmitting through the channel.That means the maximum of I(X;Y)exists.And the probability distribution p(x)is called the optimum input distribution.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.4.2 Discrete channel without memory and its channel capacity Classification of the discrete message sequence channel严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。For discrete channel without memory,it satisfies the following relationship.According to the“property 4”of the mutual information I(X;Y)of the message sequence,for the discrete channel without memory,we have 严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Note:only when the source is of without memory,the equal relationship in this formula may be satisfied So we can get the following deduction which gets the formula of channel capacity C严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Theorem of discrete channel without memory Assuming that the transmission probability matrix of the discrete channel without memory is Q,the sufficient conditions under which the input letter probability distribution p*can make the mutual information I(p;Q)to achieve maximum value are Where is the average mutual information when source letter is sent;and C is the channel capacity of this channel.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Understanding this theoremFirstly,under this kind of distribution,each letter whose probability is above zero provides mutual information C,and each letter whose probability is zero provides mutually information lower than or equal to C.Secondly,only under this kind of distribution,it may cause I(p;Q)to obtain the maximum value C.Thirdly,I(X;Y)is the average of .That is to say,it satisfies this equation 严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。(1)If we want to enhance I(X;Y),enhancing p(ak)may be a good idea.(2)However,once p(ak)is enhanced,I(x=ak;Y)may be reduced.(3)To adjust p(ak)repeatedly,make I(x=ak;Y)all equal to C (4)This time I(X,Y)=CThe theorem only provides a sufficient condition of to make distribution and the value of C;but it may help to get the value of C of several kinds of channels in simple situation.It does not give the concrete严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。E.g.4.4 Assume the transmission matrix of binary discrete symmetrical channel is(1)If please calculate(2)Please calculate the capacity of channel,and the probability distribution when reaching the capacity of channel.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。(1)严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。(2)Application Example 3.6Where m represents the number of output symbol set;Hmi is the entropy of the row vector of channel matrix.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。4.4.3 Continuous channel and its channel capacity Characteristic of continuous channelAnalog channelBasic knowledge to addable channel Shannon formulaUsage of Shannon formula严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Characteristic of continuous channelCharacteristic 1:The time is discrete,the value scope is continuous.Characteristic 2:At each moment,it is the single random variable whose value is continuous.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Analog channel严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Basic knowledge to addable channel X:channel input N:channel noiseY:channel output If two of X,Y,N are Gauss distributions,then the other is also the Gauss distribution.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。The differential entropy of the r.v.satisfying Gaussian distribution only concerns with its variance and has nothing to do with the average value.Fig.4.22.Addable channel严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。TheoremTheorem:When a generally stationary random process source with limited frequency(F)and time(T)passes through a white Gaussian channel which has limited power(PN),the channel capacity is:Shannon formulaThis is the famous Shannon Shannon formula formula for continuous channel.When T=1,the capacity is:严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Proof of Shannon formula:Assume,where X and N are independent discrete r.v.s;and SinceWe haveThe biggest entropy theorem of limited average power严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Proof of Shannon formula:Due to the limited frequency F for and according to the Nyquist sample theorem,the continuous signal X(t,w)can be equivalent to 2F discrete signals per second.That is:Considering time duration T:严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Fig.4.23.Shannon formula严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Usage of Shannon formulaIn analog communications,the frequency modulation surpasses the amplitude modulation;the wider the frequency band,the stronger is the anti-disturbance.In digital communications,the pseudo-noise(PN)code straightforwardly expands the signal.The wider the band width,the more the frequency increases,and the stronger is the anti-disturbance.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Another form of Shannon formulaWhere N0 is the noise power intensity in unit bandwidth.is the bit energy.Andis the normalization of SNR.When ConclusionConclusion:when the SNR is very low,the channel capacity can be approximately determined by its signal noise ratio.严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。E.g.4.6In photo transmission,every frame has about 2.25 106 pixels.In order to reproduce good image,we need about 16 points brightness levels.Assuming equal probability distribution of the brightness levels,please calculate the transmission channel bandwidth requirement of 30 images per second(signal-to-noise ratio is 30dB).严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。Solution:The channel capacity of additive white Gaussian noise(AWGN)channel in every unit time is,The required information transmission rate is:Example 3.8(p26)Application example严格执行突发事件上报制度、校外活动报批制度等相关规章制度。做到及时发现、制止、汇报并处理各类违纪行为或突发事件。End of Chapter 4
