优秀大学统计学授课讲义statistics.ppt

《优秀大学统计学授课讲义statistics.ppt》由会员分享，可在线阅读，更多相关《优秀大学统计学授课讲义statistics.ppt（89页珍藏版）》请在淘文阁 - 分享文档赚钱的网站上搜索。

1、QuantitativeDataAnalysis:StatisticsSherlockHolmes.while man is an insoluble puzzle,in the aggregate he becomes a mathematical certainty.You can,for example,never foretell what any one man will do,but you can say with precision what an average number will be up to.Individuals vary,but percentages rem

2、ain constant.So says the statisticianOverviewnGeneralStatisticsnTheNormalDistributionnZ-TestsnConfidenceIntervalsnT-TestsGeneralStatisticsTHEGOLDENRULEStatisticsNEVERreplacethejudgmentoftheexpert.ApproachtoStatisticalResearch1.FormulateaHypothesis2.Statepredictionsofthehypothesis3.Performexperiments

3、orobservations4.Interpretexperimentsorobservations5.Evaluateresultswithrespecttohypothesis6.Refinehypothesisandstartagain(Basicallythesameasallotherresearch)HypothesisTestingH0:Null Hypothesis,statusquoHA:Alternative Hypothesis,researchquestionSo,either:The data does not support H0orWe fail to rejec

4、t H0TypesofDatanContinuousheight,age,timenDiscrete#ofdaysworkedthisweek,#leavesonatreenOrdinalGood,O.K.,BadnNominalYes/No,Teacher/Chemist/HaberdasherPicturingTheDataPieChartsnNominal/OrdinalnOnlysuitablefordatathataddsupto1nHardtocomparevaluesinthechartBarChartsnNominal/OrdinalnEasiertocomparevalues

5、thanpiechartnSuitableforawiderrangeofdataDotPlotsnNominal/OrdinalnRepresentsallthedatanDifficulttoreadBoxPlotsnNominal/Ordinaln1IQR,3IQRnOutliersScatterPlotsnExcellentforexaminingassociationbetweentwovariablesHistogramsnContinuousDatanDivideDataintorangesTime-SeriesPlotsnTimerelatedDatane.g.StockPri

6、cesQuestion1Inatelephonesurveyof68households,whenaskeddotheyhavepets,thefollowingweretheresponses:n16:NoPetsn28:Dogsn32:CatsDrawtheappropriategraphictoillustratetheresults!Question1-SolutionTotalnumbersurveyed=68Numberwithnopets=16=Totalwithpets=(68-16)=52Buttotal28dogs+32cats=60=Sosomepeoplehavebot

7、hcatsanddogsQuestion1-SolutionHowmany?Itmustbe(60-52)=8peopleNopets=16Dogs=20Cats=24Both=8-Total=68Question1-SolutionGraphic:PieChartorBarChartThe Literary DigestPolln1936USPresidentialElectionnAlfLandon(R)vs.FranklinD.Roosevelt(D)The Literary DigestPollnLiterary Digesthadbeenconductingsuccessfulpre

8、sidentialelectionpollssince1916nTheyhadcorrectlypredictedtheoutcomesofthe1916,1920,1924,1928,and1932electionsbyconductingpolls.nThesepollswerealucrativeventureforthemagazine:readerslikedthem;newspapersplayedthemup;andeach“ballot”includedasubscriptionblank.The Literary DigestPollnTheysentout10million

9、ballotstotwogroupsofpeople:prospectivesubscribers,“whowerechieflyupper-andmiddle-incomepeople”alistdesignedtocorrectforbiasfromthefirstlist,consistingofnamesselectedfromtelephonebooksandmotorvehicleregistriesThe Literary DigestPollnResponserate:approximately25%,or2,376,523responsesnResult:Landoninal

10、andslide(predicted57%ofthevote,Rooseveltpredicted40%)nElectionresult:Rooseveltreceivedapproximately60%ofthevoteThe Literary DigestPollnPOSSIBLE CAUSES OF ERRORnSelection Bias:Bytakingnamesandaddressesfromtelephonedirectories,surveysystematicallyexcludedpoorvoters.Republicansweremarkedlyoverrepresent

11、edin1936,Democratsdidnothaveasmanyphones,notaslikelytodrivecars,anddidnotreadtheLiterary Digest“SamplingFrame”istheactualpopulationofindividualsfromwhichasampleisdrawn:SelectionbiasresultswhensamplingframeisnotrepresentativeofthepopulationofinterestThe Literary DigestPollnPOSSIBLE CAUSES OF ERRORnNo

12、n-response Bias:Becauseonly20%of10millionpeoplereturnedsurveys,non-respondentsmayhavedifferentpreferencesfromrespondentsIndeed,respondentsfavoredLandonGreaterresponseratesreducetheoddsofbiasedsamplesTerminologynPopulation:isasetofentitiesconcerningwhichstatisticalinferencesaretobedrawn.nSample:anumb

13、erofindependentobservationsfromthesameprobabilitydistributionnParameter:thedistributionofarandomvariableasbelongingtoafamilyofprobabilitydistributions,distinguishedfromeachotherbythevaluesofafinitenumberofparametersnBias:afactorthatcausesastatisticalsampleofapopulationtohavesomeexamplesofthepopulati

14、onlessrepresentedthanothers.Outliers(andtheirtreatment)nAnoutlierisanobservationthatdoesnotfitthepatternintherestofthedatanCheckthedatanCheckwiththemeasurernIfreasontobelieveitisNOTreal,changeitifpossible,otherwiseleaveitout(butnote).nIfreasontobelieveitisreal,leaveitoutandnote.TheMeannTheMean(Arith

15、metic)nThemeanisdefinedasthesumofalltheelements,dividedbythenumberofelements.nThestatisticalmeanofasetofobservationsistheaverageofthemeasurementsinasetofdataTheVariancenButtherecanbealotofvarianceinindividualelements,e.g.teachersalariesAverage=22,000Lowest=12,000Difference=12,000-22,000=-10,000TheVa

16、riancenSumof(Sample-Average)=0,thusweneedtodefinevariance.nThevarianceofasetofdataisacumulativemeasureofthesquaresofthedifferenceofallthedatavaluesfromthemeandividedbysamplesizeminusone.StandardDeviationnThestandarddeviationofasetofdataisthepositive square rootofthevariance.-1-1Question2nFindthemean

17、andvarianceofthefollowingsamplevalues:36,41,43,44,46Question2nMean:(36+41+43+44+46)/5=42nVarianceDifferenceSquaren3642=-636n4142=-11n4342=11n4442=24n4642=416n-n58n58/(5-1)=58/4=14.5TheNormalDistributionDensityCurves:PropertiesTheNormalDistributionnThegraphhasasinglepeakatthecenter,thispeakoccursatth

18、emeannThegraphissymmetricalaboutthemeannThegraphnevertouchesthehorizontalaxisnTheareaunderthegraphisequalto1CharacterizationnAnormaldistributionisbell-shapedandsymmetric.nThedistributionisdeterminedbythemeanmu,m,m,andthestandarddeviationsigma,s s.nThemeanmucontrolsthecenterandsigmacontrolsthespread.

19、TheNormalDistributionnIfavariableisnormallydistributed,then:withinonestandarddeviationofthemeantherewillbeapproximately68%ofthedatawithintwostandarddeviationsofthemeantherewillbeapproximately95%ofthedatawithinthreestandarddeviationsofthemeantherewillbeapproximately99.7%ofthedataTheNormalDistribution

20、Why?nOnereasonthenormaldistributionisimportantisthatmanypsychologicalandorgansationalvariablesaredistributedapproximatelynormally.Measuresofreadingability,introversion,jobsatisfaction,andmemoryareamongthemanypsychologicalvariablesapproximatelynormallydistributed.Althoughthedistributionsareonlyapprox

21、imatelynormal,theyareusuallyquiteclose.Why?nAsecondreasonthenormaldistributionissoimportantisthatitiseasyformathematicalstatisticianstoworkwith.Thismeansthatmanykindsofstatisticaltestscanbederivedfornormaldistributions.Almostallstatisticaltestsdiscussedinthistextassumenormaldistributions.Fortunately

22、,thesetestsworkverywellevenifthedistributionisonlyapproximatelynormallydistributed.Sometestsworkwellevenwithverywidedeviationsfromnormality.OneTail/TwoTailnImagineweundertookanexperimentwherewemeasuredstaffproductivitybeforeandafterweintroducedacomputersystemtohelprecordsolutionstocommonissuesofwork

23、Averageproductivitybefore=6.4Averageproductivityafter=9.2OneTail/TwoTailBefore=6.4After=9.2010OneTail/TwoTailBefore=6.4After=9.2100OneTail/TwoTailBefore=6.4After=9.2100OneTail/TwoTailBefore=6.4After=9.2100OneTail/TwoTailBefore=6.4After=9.2100OneTail/TwoTailBefore=6.4After=9.2100OneTail/TwoTailBefore

24、=6.4After=9.2100OneTail/TwoTailBefore=6.4After=9.2100OneTail/TwoTailnOne-TailedH0:m1=m2HA:m1m2nTwo-TailedH0:m1=m2HA:m1m2STANDARDNORMALDISTRIBUTIONnNormalDistributionisdefinedasnN(mean,(Stddev)2)nStandardNormalDistributionisdefinedasnN(0,(1)2)STANDARDNORMALDISTRIBUTIONnUsingthefollowingformula:nwillc

25、onvertanormaltableintoastandardnormaltable.ExercisenIftheaverageIQinagivenpopulationis100,andthestandarddeviationis15,whatpercentageofthepopulationhasanIQof145orhigher?AnswernP(X=145)nP(Z=(145-100)/15)nP(Z=3)nFromtables:99.87%arelessthan3n=0.13%ofpopulationTrends in Statistical Tests used in Researc

26、h PapersHistoricallyCurrentlyResultsin:Accept/RejectResultsin:p-ValueResultsin:Approx.MeanConfidenceIntervalsnAconfidenceintervalisusedtoexpresstheuncertaintyinaquantitybeingestimated.Thereisuncertaintybecauseinferencesarebasedonarandomsampleoffinitesizefromapopulationorprocessofinterest.Tojudgethes

27、tatisticalprocedurewecanaskwhatwouldhappenifweweretorepeatthesamestudy,overandover,gettingdifferentdata(andthusdifferentconfidenceintervals)eachtime.ConfidenceIntervalsnIfweknowthetruepopulationmeanandsamplenindividuals,weknowthatifthedataisnormallydistributed,Averagemeanofthesensampleshasa95%chance

28、offallingintotheintervalConfidenceIntervalsnwherethestandarderrorfora95%CImaybecalculatedasfollows;Example1Example1nDoesFF-PD-GhavemoreofthepopularvotethanFG-L?Inarandomsampleof721respondents:n382FF-PD-Gn339FG-LnCanweconcludethatFF-PD-Ghasmorethan50%ofthepopularvote?Example1-SolutionSampleproportion

29、=p=382/721=0.53Samplesize=n=721StandardError=(SqRt(p(1-p)/n)=0.0295%ConfidenceIntervaln0.53+/-1.96(0.02)n0.53+/-0.04n0.49,0.57nThus,wecannotconcludethatFF-PD-Ghadmoreofthepopularvote,sincethisintervalspans50%.So,wesay:thedataareconsistentwiththehypothesisthatthereisnodifferenceExample2Example2nDidOb

30、amahavemoreofthepopularvotethanMcCain?Inarandomsampleof1000respondentsn532Obaman468McCainnCanweconcludethatObamahadmorethan50%ofthepopularvote?Example295%CISampleproportion=p=532/1000=0.532Samplesize=n=1000StandardError=(SqRt(p(1-p)/n)=0.01695%ConfidenceIntervaln0.532+/-1.96(0.016)n0.532+/-0.03136n0

31、.5006,0.56336nThus,wecanconcludethatObamahadmoreofthepopularvote,sincethisintervaldoesnotspan50%.So,wesay:thedataareconsistentwiththehypothesisthatthereisadifferenceina95%CIExample299%CISampleproportion=p=532/1000=0.532Samplesize=n=1000StandardError=(SqRt(p(1-p)/n)=0.01699%ConfidenceIntervaln0.532+/

32、-2.58(0.016)n0.532+/-0.041n0.491,0.573nThus,wecannotconcludethatObamahadmoreofthepopularvote,sincethisintervaldoesspan50%.So,wesay:thedataareconsistentwiththehypothesisthatthereisnodifferenceina99%CIExample299.99%CISampleproportion=p=532/1000=0.532Samplesize=n=1000StandardError=(SqRt(p(1-p)/n)=0.016

33、99.99%ConfidenceIntervaln0.532+/-3.87(0.016)n0.532+/-0.06n0.472,0.592nThus,wecannotconcludethatObamahadmoreofthepopularvote,sincethisintervaldoesspan50%.So,wesay:thedataareconsistentwiththehypothesisthatthereisnodifferenceina99.99%CIT-TestsOneTail/TwoTailT-testZ-testT-Testsnpowerfulparametrictestfor

34、calculatingthesignificanceofasmallsamplemeannnecessaryforsmallsamplesbecausetheirdistributionsarenotnormalnonefirsthastocalculatethedegreesoffreedomT-TestsnThet-testisoftencalledtheStudentst-test.ItwascreatedbyachiefbrewernamedWilliamS.GossettwhoworkedfortheGuinnessBrewery.Hediscoveredthisstatistica

35、spartofhisworkinthebrewerytocomparethedifferentbrewingprocessesforchangingrawmaterialsintobeer.nGuinnessdidnotallowitsemployeestopublishresultsbutthemanagementdecidedtoallowGossetttopublishitunderapseudonym-Student.HencewehavetheStudentst-test.T-TestTHEGOLDENRULEUsethet-Testwhenyoursamplesizeislesst

36、han30T-TestsnIftheunderlyingpopulationisnormalnIftheunderlyingpopulationisnotskewedandreasonabletonormaln(n15)nIftheunderlyingpopulationisskewedandsomeoutliersn(n24)T-TestsnFormofConfidenceIntervalwitht-ValueMean+/-tValue*SE-asbeforeasbeforeTwoSampleT-Test:UnpairedSamplenConsideraquestionnaireoncomp

37、uterusetofinalyearundergraduatesinyear2007andthesamequestionnairegivetoundergraduatesin2008.Asthereisnodirectone-to-onecorrespondencebetweenindividualstudents(infact,theremaybedifferentnumberofstudentsindifferentclasses),youhavetosumupalltheresponsesofagivenyear,obtainanaveragefromthat,downthesamefo

38、rthefollowingyear,andcompareaverages.TwoSampleT-Test:PairedSamplenIfyouaredoingaquestionnairethatistestingtheBEFORE/AFTEReffectofparameteronthesamepopulation,thenwecanindividuallycalculatedifferencesbetweeneachsampleandthenaveragethedifferences.Thepairedtestisamuchstrong(morepowerful)statisticaltest.ChoosingtherighttestChoosingastatisticaltesthttp:/Choosingastatisticaltesthttp:/