斯坦福大学机器学习课程讲义——多变量的线性回归模型表达学习教案.pptx

《斯坦福大学机器学习课程讲义——多变量的线性回归模型表达学习教案.pptx》由会员分享，可在线阅读，更多相关《斯坦福大学机器学习课程讲义——多变量的线性回归模型表达学习教案.pptx（39页珍藏版）》请在淘文阁 - 分享文档赚钱的网站上搜索。

1、会计学1斯坦福大学机器学习课程讲义斯坦福大学机器学习课程讲义 多变量多变量(binling)的线性回归模型表达的线性回归模型表达第一页，共39页。Size(feet2)Number of bedroomsNumber of floorsAge of home(years)Price($1000)2104514546014163240232153432303158522136178Multiple features(variables).Notation:=number of features=input(features)of training example.=value of featur

2、e in training example.第3页/共40页第三页，共39页。Hypothesis:Previously:第4页/共40页第四页，共39页。For convenience of notation,define .Multivariate linear regression.第5页/共40页第五页，共39页。第6页/共40页第六页，共39页。Linear Regression with multiple variablesGradient descent for multiple variablesMachine Learning第7页/共40页第七页，共39页。Hypothes

3、is:Cost function:Parameters:(simultaneously update for every )RepeatGradient descent:第8页/共40页第八页，共39页。(simultaneously update )Gradient DescentRepeatPreviously(n=1):New algorithm :Repeat(simultaneously update for )第9页/共40页第九页，共39页。第10页/共40页第十页，共39页。Linear Regression with multiple variablesGradient de

4、scent in practice I:Feature ScalingMachine Learning第11页/共40页第十一页，共39页。E.g.=size(0-2000 feet2)=number of bedrooms(1-5)Feature ScalingIdea:Make sure features are on a similar scale.size(feet2)number of bedrooms第12页/共40页第十二页，共39页。Feature ScalingGet every feature into approximately a range.第13页/共40页第十三页

5、，共39页。Replace with to make features have approximately zero mean(Do not apply to ).Mean normalizationE.g.第14页/共40页第十四页，共39页。第15页/共40页第十五页，共39页。Linear Regression with multiple variablesGradient descent in practice II:Learning rateMachine Learning第16页/共40页第十六页，共39页。Gradient descent-“Debugging”:How to

6、make sure gradient descent is working correctly.-How to choose learning rate .第17页/共40页第十七页，共39页。Example automatic convergence test:Declare convergence if decreases by less than in one iteration.0100200300400No.of iterationsMaking sure gradient descent is working correctly.第18页/共40页第十八页，共39页。Making

7、sure gradient descent is working correctly.Gradient descent not working.Use smaller .No.of iterationsNo.of iterationsNo.of iterations-For sufficiently small ,should decrease on every iteration.-But if is too small,gradient descent can be slow to converge.第19页/共40页第十九页，共39页。Summary:-If is too small:s

8、low convergence.-If is too large:may not decrease on every iteration;may not converge.To choose ,try第20页/共40页第二十页，共39页。第21页/共40页第二十一页，共39页。Linear Regression with multiple variablesFeatures and polynomial regressionMachine Learning第22页/共40页第二十二页，共39页。Housing prices prediction第23页/共40页第二十三页，共39页。Polyn

9、omial regressionPrice(y)Size(x)第24页/共40页第二十四页，共39页。Choice of featuresPrice(y)Size(x)第25页/共40页第二十五页，共39页。第26页/共40页第二十六页，共39页。Linear Regression with multiple variablesNormal equationMachine Learning第27页/共40页第二十七页，共39页。Gradient DescentNormal equation:Method to solve for analytically.第28页/共40页第二十八页，共39页

10、。Intuition:If 1DSolve for(for every )第29页/共40页第二十九页，共39页。Size(feet2)Number of bedroomsNumber of floorsAge of home(years)Price($1000)12104514546011416324023211534323031518522136178Size(feet2)Number of bedroomsNumber of floorsAge of home(years)Price($1000)2104514546014163240232153432303158522136178Exa

11、mples:第30页/共40页第三十页，共39页。examples ;features.E.g.If第32页/共40页第三十二页，共39页。is inverse of matrix .Octave:pinv(X*X)*X*y第33页/共40页第三十三页，共39页。training examples,features.Gradient DescentNormal EquationNo need to choose .Dont need to iterate.Need to choose .Needs many iterations.Works well even when is large.Ne

12、ed to computeSlow if is very large.第34页/共40页第三十四页，共39页。第35页/共40页第三十五页，共39页。Linear Regression with multiple variablesNormal equation and non-invertibility(optional)Machine Learning第36页/共40页第三十六页，共39页。Normal equation-What if is non-invertible?(singular/degenerate)-Octave:pinv(X*X)*X*y第37页/共40页第三十七页，共39页。What if is non-invertible?Redundant features(linearly dependent).E.g.size in feet2 size in m2Too many features(e.g.).-Delete some features,or use regularization.第38页/共40页第三十八页，共39页。第39页/共40页第三十九页，共39页。