SPSS混合线性模型.ppt

2OutlineClassification of EffectsRandom EffectsTwo-Way Random LayoutSolutions and estimatesGeneral linear modelFixed Effects ModelsThe one-way layoutMixed Model theoryProper error termsTwo-way layoutFull-factorial modelContrasts with interaction termsGraphing Interactions3Outline-Contd Repeated Measures ANOVA Advantages of Mixed Models over GLM.4Definition of Mixed Models by their component effectsMixed Models contain both fixed and random effectsFixed Effects: factors for which the only levels under consideration are contained in the coding of those effectsRandom Effects: Factors for which the levels contained in the coding of those factors are a random sample of the total number of levels in the population for that factor.5Examples of Fixed and Random EffectsFixed effect: Sex where both male and female genders are included in the factor, sex. Agegroup: Minor and Adult are both included in the factor of agegroupRandom effect: Subject: the sample is a random sample of the target population6Classification of effectsThere are main effects: Linear Explanatory Factors There are interaction effects: Joint effects over and above the component main effects.78Classification of Effects-contdHierarchical designs have nested effects. Nested effects are those with subjects within groups.An example would be patients nested within doctors and doctors nested within hospitalsThis could be expressed bypatients(doctors)doctors(hospitals)910Between and Within-Subject effectsSuch effects may sometimes be fixed or random. Their classification depends on the experimental designBetween-subjects effects are those who are in one group or another but not in both. Experimental group is a fixed effect because the manager is considering only those groups in his experiment. One group is the experimental group and the other is the control group. Therefore, this grouping factor is a between- subject effect. Within-subject effects are experienced by subjects repeatedly over time. Trial is a random effect when there are several trials in the repeated measures design; all subjects experience all of the trials. Trial is therefore a within-subject effect.Operator may be a fixed or random effect, depending upon whether one is generalizing beyond the sampleIf operator is a random effect, then the machine*operator interaction is a random effect.There are contrasts: These contrast the values of one level with those of other levels of the same effect.11Between Subject effects Gender: One is either male or female, but not both. Group: One is either in the control, experimental, or the comparison group but not more than one.12Within-Subjects Effects These are repeated effects. Observation 1, 2, and 3 might be the pre, post, and follow-up observations on each person. Each person experiences all of these levels or categories. These are found in repeated measures analysis of variance.13Repeated Observations are Within-Subjects effects Trial 1 Trial 2 Trial 3 GroupGroup is a between subjects effect, whereas Trial is a within subjects effect.14The General Linear ModelThe main effects general linear model can be parameterized as()()()exp( ,)ijijijijiijjijYbwhereYobservation for ithgrand mean an unknown fixed parmeffect of ith value ofabeffect of jth value of b berimental errorN2015A factorial modelIf an interaction term were included, the formula would beijiiijijyeThe interaction or crossed effect is the joint effect, over and above the individual main effects. Therefore, the main effects must be in the model for the interaction to be properly specified.()()i jijijyy16Higher-Order InteractionsIf 3-way interactions are in the model, then the main effects and all lower order interactions must be in the model for the 3-way interaction to be properly specified. For example, a 3-way interaction model would be:ijkijkijikjkijkijkyabcabacbcabce17The General Linear Model In matrix terminology, the general linear model may be expressed asYXwhereYtheobserved datavectorXthedesignmatrixthevectorof unknown fixed effect parametersthevectorof errors18AssumptionsOf the general linear model( )var( )var( )( )EIYIE YX22019General Linear Model Assumptions-contd1. Residual Normality.2. Homogeneity of error variance3. Functional form of Model: Linearity of Model4. No Multicollinearity5. Independence of observations6. No autocorrelation of errors 7. No influential outliersWe have to test for these to be sure that the model is valid. We will discuss the robustness of the model in face of violations of these assumptions.We will discuss recourses when these assumptions are violated.20Explanation of these assumptionsFunctional form of Model: Linearity of Model: These models only analyze the linear relationship.Independence of observationsRepresentativeness of sampleResidual Normality: So the alpha regions of the significance tests are properly defined.Homogeneity of error variance: So the confidence limits may be easily found.No Multicollinearity: Prevents efficient estimation of the parameters.No autocorrelation of errors: Autocorrelation inflates the R2 ,F and t tests. No influential outliers: They bias the parameter estimation.21Diagnostic tests for these assumptionsFunctional form of Model: Linearity of Model: Pair plotIndependence of observations: Runs testRepresentativeness of sample: Inquire about sample designResidual Normality: SK or SW testHomogeneity of error variance Graph of Zresid * ZpredNo Multicollinearity: Corr of XNo autocorrelation of errors: ACFNo influential outliers: Leverage and Cooks D.22Testing for outliersFrequencies analysis of stdres cksd.Look for standardized residuals greater than 3.5 or less than 3.5 And look for Cooks D.23Studentized Residuals( )( )()isiiisiiieeshwhereestudentized residualsstandard deviationwhereithobsisdeletedhleverage statistic21Belsley et al (1980) recommend the use of studentizedResiduals to determine whether there is an outlier.24Influence of OutliersLeverage is measured by the diagonal components of the hat matrix.The hat matrix comes from the formula for the regression of Y.()(),YXXX XX Ywhere XX XXthe hatmatrix HThereforeYHY1125Leverage and the Hat matrixThe hat matrix transforms Y into the predicted scores.The diagonals of the hat matrix indicate which values will be outliers or not. The diagonals are therefore measures of leverage.Leverage is bounded by two limits: 1/n and 1. The closer the leverage is to unity, the more leverage the value has.The trace of the hat matrix = the number of variables in the model.When the leverage 2p/n then there is high leverage according to Belsley et al. (1980) cited in Long, J.F. Modern Methods of Data Analysis (p.262). For smaller samples, Vellman and Welsch (1981) suggested that 3p/n is the criterion.26Cooks DAnother measure of influence.This is a popular one. The formula for it is:()iiiiiheCook s Dphsh22111Cook and Weisberg(1982) suggested that values of D that exceeded 50% of the F distribution (df = p, n-p)are large.27Cooks D in SPSSFinding the influential outliersSelect those observations for which cksd (4*p)/n Belsley suggests 4/(n-p-1) as a cutoffIf cksd (4*p)/(n-p-1);28What to do with outliers1. Check coding to spot typos2. Correct typos3. If observational outlier is correct, examine the dffits option to see the influence on the fitting statistics. 4. This will show the standardized influence of the observation on the fit. If the influence of the outlier is bad, then consider removal or replacement of it with imputation. 29Decomposition of the Sums of SquaresMean deviations are computed when means are subtracted from individual scores.This is done for the total, the group mean, and the error terms.Mean deviations are squared and these are called sums of squaresVariances are computed by dividing the Sums of Squares by their degrees of freedom.The total Variance = Model Variance + error variance30Formula for Decomposition of Sums of SquaresSS total = SS error + SSmodel.()(.)()()(.)()()(.)i jijjji jijjji jijjjyyyyyytotaleffecterror withinmodel effectwe square the termsyyyyyyand sum them over the data setyyyyyySStotalSSerrorGroupSSwhere SSSumsof222222Squares31Variance DecompositionDividing each of the sums of squares by their respective degrees of freedom yields the variances.Total variance= error variance + model variance.in fixed effects modelsmodelvarianceFerrorvarianceSStotalSSerrorSSmodelnnkk1132Proportion of Variance ExplainedR2 = proportion of variance explained.SStotal = SSmodel + SSerrrorDivide all sides by SStotalSSmodel/SStotal =1 - SSError/SStotalR2=1 - SSError/SStotal33The Omnibus F testThe omnibus F test is a test that all of the means of the levels of the main effects and as well as any interactions specified are not significantly different from one another.Suppose the model is a one way anova on breakingpressure of bonds of different metals.Suppose there are three metals: nickel, iron, andCopper.H0: Mean(Nickel)= mean (Iron) = mean(Copper)Ha: Mean(Nickel) ne Mean(Iron) or Mean(Nickel) ne Mean(Copper) or Mean(Iron) ne Mean(Copper)34Testing different Levels of a Factor against one another Contrast are tests of the mean of one level of a factor against other levels.:aHH012312231335Contrasts-contd A contrast statement computes ( )( )L L V LLZZFrank L1 The estimated V- is the generalized inverse of the coefficient matrix of the mixed model. The L vector is the kb vector.The numerator df is the rank(L) and the denominatordf is taken from the fixed effects table unless otherwisespecified.36Construction of the F tests in different modelsThe F test is a ratio of two variances (Mean Squares).It is constructed by dividing the MS of the effect to betested by a MS of the denominator term. The divisionshould leave only the effect to be tested left over as a remainder.A Fixed Effects model F test for a = MSa/MSerror.A Random Effects model F test for a = MSa/MSabA Mixed Effects model F test for b = MSa/MSabA Mixed Effects model F test for ab = MSab/MSerror37Data format The data format for a GLM is that of wide data.38Data Format for Mixed Models is Long39Conversion of Wide to Long Data Format Click on Data in the header bar Then click on Restructure in the pop-down menu40A restructure wizard appearsSelect restructure selected variables into cases and click on Next41A Variables to Cases: Number of Variable Groups dialog box appears. We select one and click on next.42We select the repeated variables and move them to the target variable box43After moving the repeated variables into the target variable box, we move the fixed variables into the Fixed variable box, and select a variable for case idin this case, subject.Then we click on Next44A create index variables dialog box appears. We leave the number of index variables to be created at one and click on next at the bottom of the box45When the following box appears we just type in time and select Next.46When the options dialog box appears, we select the option for dropping variables not selected.We then click on Finish.47We thus obtain our data in long format48The Mixed Model The Mixed Model uses long data format. It includes fixed and random effects.It can be used to model merely fixed or random effects, by zeroing out the other parameter vector.The F tests for the fixed, random, and mixed models differ.Because the Mixed Model has the parameter vector for both of these and can estimate the error covariance matrix for each, it can provide the correct standard errors for either the fixed or random effects.49The Mixed ModelyXZwherefixed effects parameter estimatesXfixed effectsZRandom effects parameter estimatesrandom effectserrorsVariance of yVZGZRG and R require covariancestructurefitting50Mixed Model Theory-contdLittle et al.(p.139) note that u and e are uncorrelated random variables with 0 means and covariances, G and R, respectively.,()()Because thecovariance matrixVZGZRthe solution forX VXX VyuGZ VyX11V- is a generalized inverse. Because V is usually singular and noninvertible AVA = V- is an augmented matrix that is invertible. It can later be transformed back to V.The G and R matrices must be positive definite.In the Mixed procedure, the covariance type of the random (generalized) effects defines the structure of G and a repeated covariance type defines structure of R.51Mixed Model Assumptions0uE 00uGVarianceR A linear relationship between dependent and independent variables52Random Effects Covariance Structure This defines the structure of the G matrix, the random effects, in the mixed model. Possible structures permitted by current version of SPSS: Scaled Identity Compound Symmetry AR(1) Huynh-Feldt53Structures of Repeated effects (R matrix)-contdVariance Components2122232400 000 0000000Compound Symmetry222221111222221211222221131222221114( )AR 2322321111154Structures of Repeated Effects (R matrix)HuynhFeldt22222131212222223212222223132322222255Structures of Repeated effects (R matrix) contdunstructured 21121213132212122323231313232356R matrix, defines the correlation among repeated random effects.R 211121112111211121112111One can specify the nature of the correlation among therepeated random effects.57GLM Mixed ModelThe General Linear Model is a special case of theMixed Model with Z = 0 (which means thatZu disappears from the model) and 2RI58Mixed Analysis of a Fixed Effects modelSPSS tests these fixed effects just as it does with the GLMProcedure with type III sums of squares.We analyze the breaking pressure of bonds made from three metals. We assume that we do not generalize beyond our sample and that our effects are all fixed.Tests of Fixed Effects is performed with the help of the L matrix by constructing the following F test: ()( )L L X VXLLFrank L1Numerator df = rank(L)Denominator df = RESID (n-rank(X) df = Satherth 59Estimation: Newton ScoringiisHgwhereggradientmatrixofst derivativesHHessian matrixofnd derivativessincrementof step parameter111260Estimation: Minimization of the objective functions11111111( , ):log|log (1 log(2 / )2221( , ):log|log|22log 1 log|2 /()|22()()(nnML G RVr V rnnREML G RVX VXnpnpr V rnpwhere ryX X VXX Vyprank of Xso that the probabilities ofX VXX Vy andGZ V1().yXare maximizedUsing Newton Scoring, the following functions are minimized61Significance of Parameters11111:0Lis a linear combinationHotLCLwhereX R XX R ZCZ R XZR ZG 62Test one covariance structure against the other with the IC The rule of thumb is smaller is better -2LL AIC Akaike AICC Hurvich and Tsay BIC Bayesian Info Criterion Bozdogans CAIC63Measures of Lack of fit: The information Criteria-2LL is called the deviance. It is a measure of sum of squared errors.AIC = -2LL + 2p (p=# parms)BIC = Schwartz Bayesian Info criterion = 2LL + plog(n)AICC= Hurvich and Tsays small sample correction on AIC: -2LL + 2p(n/(n-p-1)CAIC = -2LL + p(log(n) + 1)64Procedures for Fitting the Mixed Model One can use the LR test or the lesser of the information criteria. The smaller the information criterion, the better the model happens to be. We try to go from a larger