第4章n　多元回归估计与假设检验.ppt

《第4章n　多元回归估计与假设检验.ppt》由会员分享，可在线阅读，更多相关《第4章n　多元回归估计与假设检验.ppt（88页珍藏版）》请在淘文阁 - 分享文档赚钱的网站上搜索。

1、第4章n多元回归估计与假设检验 Still waters run deep.流静水深流静水深,人静心深人静心深 Where there is life,there is hope。有生命必有希望。有生命必有希望2Parallels with Simple RegressionYi=b0+b1Xi1+b2Xi2+.bkXik+uib0 is still the interceptb1 to bk all called slope parameters,also called partial regression coefficients and any coefficient bj denote

2、 the change of Y with the changes of Xj as all the other independent variables fixed.u is still the error term(or disturbance)Still minimizing the sum of squared residuals,so have k+1 first order conditions3Obtaining OLS Estimates4Obtaining OLS Estimates,cont.The above estimated equation is called t

3、he OLS regression line or the sample regression function(SRF)the above equation is the estimated equation,is not the really equation.The really equation is population regression line which we dont know.We only estimate it.So,using a different sample,we can get another different estimated equation li

4、ne.The population regression line is5Interpreting Multiple Regression6An Example(Wooldridge,p76)The determination of wage(dollars per hour),wage:Years of education,educYears of labor market experience,experYears with the current employer,tenureThe relationship btw.wage and educ,exper,tenure:wage=b0+

5、b1educ+b2exper+b3tenure+uThe estimated equation as below:wage=-2.873+0.599educ+0.022exper+0.169tenure7A“Partialling Out”Interpretation8A“Partialling Out”Interpretation9“Partialling Out”continuedPrevious equation implies that regressing Y on X1 and X2 gives same effect of X1 as regressing Y on residu

6、als from a regression of X1 on X2This means only the part of Xi1 that is uncorrelated with Xi2 are being related to Yi,so were estimating the effect of X1 on Y after X2 has been“partialled out”10The wage determinationsThe estimated equation as below:wage=-2.873+0.599educ+0.022exper+0.169tenureNow,we

7、 first regress educ on exper and tenure to patial out the exper and tenures effects.Then we regress wage on the residuals of educ on exper and tenure.Whether we get the same result.?educ=13.575-0.0738exper+0.048tenure denote residuals residwage=5.896+0.599residWe can see that the coefficient of resi

8、d is the same of the coefficien of the variable educ in the first estimated equation.So is in the second equation.11Goodness-of-Fit:R212Goodness-of-Fit13Goodness-of-Fit(continued)How do we think about how well our sample regression line fits our sample data?Can compute the fraction of the total sum

9、of squares(SST)that is explained by the model,call this the R-squared of regression R2=ESS/TSS=1 RSS/TSS14Goodness-of-Fit(continued)15More about R-squaredwR2 can never decrease when another independent variable is added to a regression,and usually will increasewBecause R2 will usually increase with

10、the number of independent variables,it is not a good way to compare models16An ExamplewUsing wage determination model to show that when we add another new independent variable will increase the value of R2.17Adjusted R-SquaredwR2 is simply an estimate of how much variation in y is explained by X1,X2

11、,Xk.That is,wRecall that the R2 will always increase as more variables are added to the modelwThe adjusted R2 takes into account the number of variables in a model,and may decrease18Adjusted R-Squared(cont)wMost packages will give you both R2 and adj-R2w You can compare the fit of 2 models(with the

12、same Y)by comparing the adj-R2awge=-3.391+0.644educ+0.070exper adj-R2=0.2222awge=-2.222+0.569educ+0.190tenure adj-R2=0.2992w You cannot use the adj-R2 to compare models with different ys(e.g.y vs.ln(Y)awge=-3.391+0.644educ+0.070exper adj-R2=0.2222alog(wge)=0.404+0.087educ+0.026exper adj-R2=0.3059aBe

13、cause the variance of the dependent variables is different,the comparation btw them make no sense.19Assumptions for Unbiasedness20Assumptions for UnbiasednessPopulation model is linear in parameters:Y=b0+b1X1+b2X2+bkXk+uWe can use a sample of size n,(Xi1,Xi2,Xik,Yi):i=1,2,n,from the population model

14、,so that the sample model is Yi=b0+b1Xi1+b2Xi2+bkXik+ui Cov(uXi)=0,E(uXi)=0,i=1,2,n.E(u|X1,X2,Xk)=0,implying that all of the explanatory variables are exogenous.E(u|X)=0,where X=(X1,X2,Xk),which will reduce to E(u)=0 if independent variables X are not random variables.None of the Xs is constant,and

15、there are no exact linear relationships among them.The new additional assumption.21About multicollinearityIt does allow the independent variables to be correlated;they just cannot be perfectly linear correlated.Student performance:colGPA=b0+b1 hsGPA+b2ACT+b3 skipped+uConsumption function:consum=b0+b

16、1inc+b2inc2+uBut,the following is invalid:log(consum)=b0+b1log(inc)+b2log(inc2)+uIn this case,we can not estimate the regression coefficients b1,b2.22Unbiasedness of OLS estimationUnder the three assumptions above,we can get23Too Many or Too Few Variables24Too Many or Too Few VariablesWhat happens i

17、f we include variables in our specification that dont belong?There is no effect on our parameter estimate,and OLS remains unbiasedWhat if we exclude a variable from our specification that does belong?OLS will usually be biased 25Omitted Variable Bias26Omitted Variable Bias(cont)27Omitted Variable Bi

18、as(cont)28Omitted Variable Bias(cont)There are two cases where the estimated parameter is unbiased:If b2=0,so that X2 does not appear in the true modelIf tilde of d1=0,the tilde b1 is unbiased for b1 29Summary of Direction of BiasCorr(X1,X2)0Corr(X1,X2)0Positive biasNegative biasb2 0 and H1:bj 0One-

19、Sided Alternatives(cont)0ca(1-a)Fail to rejectreject58An Example:Hourly Wage EquationwWage determination:(wooldridge,p123)wlog(wge)=0.284+0.092educ+0.0041exper+0.022tenurew (0.104)(0.007)(0.0017)(0.003)w n=526 R2=0.316wWhether the return to exper,controlling for educ and tenure,is zero in the popula

20、tion,against the alternative that it is positive.wH0:bexper=0 vs.H1:bexper 0wThe t statistic is t=0.0041/0.00172.41wThe degree of freedom:df=n-k-1=526-3-1=522wThe critical value of 5%is 1.645wAnd the t statistic is larger than the critical value,ie.,2.411.645wThat is,we will reject the null hypothes

21、is and bexper is really positive.01.645(1-a)Fail to reject5%reject59Another example:Student Performance and School SizewWhether the school size has effect on student performance?amath10,math test scores,reveal the student performanceatotcomp,average annual teacher compensationastaff,the number of st

22、aff per one thousand studentsaenroll,student enrollment,reveal the school size.wThe Model Equationamath10=b0+b1totcomp+b2staff+b3enroll+uaH0:b3=0,H1:b3-1.645,so we cant reject the null hypothesis.-1.645reject-09160One-sided vs Two-sidedwBecause the t distribution is symmetric,testing H1:bj 0 is stra

23、ightforward.The critical value is just the negative of beforewWe can reject the null if the t statistic c,then we fail to reject the nullwFor a two-sided test,we set the critical value based on a/2 and reject H0:bj=0 if the absolute value of the t statistic c61yi =b0 +b1Xi1 +bkXik+uiH0:bj=0 H1:bj 0c

24、0a/2(1-a)-ca/2Two-Sided Alternativesrejectrejectfail to reject62Summary for H0:b bj=0wUnless otherwise stated,the alternative is assumed to be two-sidedwIf we reject the null,we typically say“Xj is statistically significant at the 100a%level”wIf we fail to reject the null,we typically say“Xj is stat

25、istically insignificant at the 100a%level”63An Example:Determinants of College GPA(wooldridge,p128)wVariables:acolGPA,college GPAaskipped,the average number of lectures missed per weekaACT,achievement test scoreahsGPA,high school GPAwThe estimated modelaolGPA=1.39+0.412 hsGPA+0.015 ACT 0.083 skipped

26、a (0.33)(0.094)(0.011)(0.026)a n=141,R2=0.234wH0:bskipped=0,H1:bskipped 0wfd:n-k-1=137,the critical value t137=1.96wThe t statistic is|-0.083/0.026|=3.19 t137=1.96,so we will reject the null hypothesis and the bskipped is signanificantly beyond zero.-1.96reject-3.191.96reject64Testing other hypothes

27、eswA more general form of the t statistic recognizes that we may want to test something like H0:bj=aj wIn this case,the appropriate t statistic is65An Example:Campus Crime and Enrollment(wooldridge,p129)lVariableslcrime,the annual number of crimes on college campuseslenroll,student enrollment,reveal

28、 the size of college.lThe regression modelllog(crime)=b0+b1log(enroll)+ulWhether b1=1,that is H0:b1=1,H1:b1 1llog(crime)=-6.63+1.27 log(enroll)l (1.03)(0.11)n=97 R2=0.585ldf:n-k-1=95,the critical value at 5%is t95=1.645lThe t-statistic is(1.27-1)/0.112.45t95=1.645lSo we reject the null hypothesis an

29、d the evidence prove that b1 1.66Confidence IntervalswAnother way to use classical statistical testing is to construct a confidence interval using the same critical value as was used for a two-sided testw A 100(1-a)%confidence interval is defined as67Computing p-values for t testswAn alternative to

30、the classical approach is to ask,“what is the smallest significance level at which the null would be rejected?”wSo,compute the t statistic,and then look up what percentile it is in the appropriate t distribution this is the p-valuewp-value is the probability we would observe the t statistic we did,i

31、f the null were true68Stata and p-values,t tests,etc.wMost computer packages will compute the p-value for you,assuming a two-sided testwIf you really want a one-sided alternative,just divide the two-sided p-value by 2wStata provides the t statistic,p-value,and 95%confidence interval for H0:bj=0 for

32、you,in columns labeled“t”,“P|t|”and“95%Conf.Interval”,respectively69Testing a Linear CombinationSuppose instead of testing whether b1 is equal to a constant,you want to test if it is equal to another parameter,that is H0:b1=b2,or b1-b2=0Use same basic procedure for forming a t statistic70Testing Lin

33、ear Combination(cont)71Testing a Linear Combo(cont)So,to use formula,need s12,which standard output does not haveMany packages will have an option to get it,or will just perform the test for youIn Stata,after reg Y X1 X2 Xk you would type test X1=X2 to get a p-value for the testMore generally,you ca

34、n always restate the problem to get the test you want72Example:Suppose you are interested in the effect of campaign expenditures on outcomesModel is voteA=b0+b1log(expendA)+b2log(expendB)+b3prtystrA+uH0:b1=-b2,or H0:q1=b1+b2=0b1=q1 b2,so substitute in and rearrange voteA=b0+q1log(expendA)+b2log(expe

35、ndB)log(expendA)+b3prtystrA+u73Example(cont):This is the same model as originally,but now you get a standard error for b1 b2=q1 directly from the basic regressionAny linear combination of parameters could be tested in a similar mannerOther examples of hypotheses about a single linear combination of

36、parameters:b1=1+b2;b1=5b2;b1=-1/2b2;etc 74Multiple Linear RestrictionsEverything weve done so far has involved testing a single linear restriction,(e.g.b1=0 or b1=b2)However,we may want to jointly test multiple hypotheses about our parametersA typical example is testing“exclusion restrictions”we wan

37、t to know if a group of parameters are all equal to zero75Testing Exclusion RestrictionsNow the null hypothesis might be something like H0:bk-q+1=0,.,bk=0The alternative is just H1:H0 is not trueCant just check each t statistic separately,because we want to know if the q parameters are jointly signi

38、ficant at a given level it is possible for none to be individually significant at that level76Exclusion Restrictions(cont)To do the test we need to estimate the“restricted model”without Xk-q+1,Xk included,as well as the“unrestricted model”with all Xs includedIntuitively,we want to know if the change

39、 in RSS is big enough to warrant inclusion of Xk-q+1,Xk 77The F statisticThe F statistic is always positive,since the RSS from the restricted model cant be less than the RSS from the unrestricted.Essentially the F statistic is measuring the relative increase in RSS when moving from the unrestricted

40、to restricted model q=number of restrictions,or dfr dfur n k 1=dfur78The F statistic(cont)To decide if the increase in RSS when we move to a restricted model is“big enough”to reject the exclusions,we need to know about the sampling distribution of our F statNot surprisingly,F Fq,n-k-1,where q is ref

41、erred to as the numerator degrees of freedom and n k 1 as the denominator degrees of freedom 790c(1-a)f(F)FThe F statistic(cont)rejectfail to rejectReject H0 at a significance level if F ca80Example:the determinations of league baseball players salaries(wooldridge,p143)The regression modellog(salary

42、)=b0+b1year+b2gamesyr+b3bavg+b4hrunsyr+b5rbisyr+usalary,the 1993 total salaryyears,years in the leaguegamesyr,average games played per yearbavg,career batting averagehrunsyr,home runs per yearrbisyr,runs battled in per yearThe null hypothesis is H0:b3=0,b4=0,b5=0,which is called multiple hypotheses

43、test or joint hypotheses test.The alternative hypothesis is H1:H0 is not true.The unrestricted model:log(salary)=11.19+0.0689year+0.0126gamesyr+0.00098bavg+0.0144hrunsyr+0.0108rbisyr (0.29)(0.0689)(0.0026)(0.00110)(0.0161)(0.0072)n=353,SSR=183.186,R2=0.6278The restricted modellog(salary)=11.22+0.071

44、3year+0.0202gamesyr (0.11)(0.0125)(0.0013)n=353,SSR=198.311,R2=0.597181Example:the determinations of league baseball players salaries,cont.The restricted number and the degree of the freedom of restricted model is q=3;The degree of freedom of unrestricted model is 353-5-1=347;Then the F statistic is

45、82The R2 form of the F statisticBecause the RSSs may be large and unwieldy,an alternative form of the formula is usefulWe use the fact that RSS=TSS(1 R2)for any regression,so can substitute in for RSSu and RSSur83Example:ParentExample:Parent s Education in a Birth s Education in a Birth Weight Equat

46、ion(wooldridge,p150)Weight Equation(wooldridge,p150)Variablesbwght,birth weight in pounds;cigs,average number of cigarettes the mother smoked per day during pregnancy;parity,the birth order of this child;faminc,annual family income;motheduc,years of schooling for the mother;fatheduc,years of schooli

47、ng for the father.Model:bwght=b0+b1cigs+b2parity+b3faminc+b4motheduc+b5fatheduc+uWhether the parents education has any effect on birth weight?This is stated as H0:b4=0,b5=0,so q=2.bwght=114.524-0.596cigs+1.788parity+0.056faminc-0.370motheduc+0.472fatheduc (3.728)(0.110)(0.659)(0.037)(0.320)(0.283)n=

48、1191 R2=0.0387bwght=115.470-0.598cigs+1.832parity+0.067faminc (1.656)(0.109)(0.658)(0.032)n=1191 R2=0.0364F Statistic is F=(0.0387-0.0364)/2/(1-0.0387)/(1191-5-1)=1.42 F2,1185=19.5We fail to reject H0.In other words,motheduc and fatheduc are jointly insignificant in the birth weight equation84Overal

49、l SignificanceA special case of exclusion restrictions is to test H0:b1=b2=bk=0Since the R2 from a model with only an intercept will be zero,the F statistic is simply85General Linear Restrictions The basic form of the F statistic will work for any set of linear restrictions First estimate the unrest

50、ricted model and then estimate the restricted model In each case,make note of the RSS Imposing the restrictions can be tricky will likely have to redefine variables again86Example:Use same voting model as beforeModel is voteA=b0+b1log(expendA)+b2log(expendB)+b3prtystrA+unow null is H0:b1=1,b3=0Subst