模式识别课件prch5part3_ding(精品).ppt

《模式识别课件prch5part3_ding(精品).ppt》由会员分享，可在线阅读，更多相关《模式识别课件prch5part3_ding(精品).ppt（13页珍藏版）》请在淘文阁 - 分享文档赚钱的网站上搜索。

1、Pattern ClassificationAll materials in these slides were taken All materials in these slides were taken from from Pattern Classification(2nd ed)by R.O.Pattern Classification(2nd ed)by R.O.DudaDuda,P.E.Hart and D.G.Stork,John,P.E.Hart and D.G.Stork,John Wiley&Sons,2000Wiley&Sons,2000 with the permiss

2、ion of the authors and with the permission of the authors and the publisherthe publisher5.8 Minimum Squared Error Proceduresn n Criterion function involves all of the samples,not just misclassified onesnPreviously we were interested in making all of the inner products positivenNow try to make where

3、are somearbitrarily specified positive constantsnThus replace the problem of solving a set oflinear inequalities with more stringent but better understood problem of finding a solution to a set of linear equations nMinimum Squared Error and PseudoinverseFor all the samples we want a weight For all t

4、he samples we want a weight vector a so that for some arbitrarily vector a so that for some arbitrarily specified positive numbers.The matrix notation:specified positive numbers.The matrix notation:Error vector:Error vector:2 2n nSum-of-squared-error criterion function:Sum-of-squared-error criterion

5、 function:n nThe gradientThe gradientSet it to zero,we getSet it to zero,we getIf is nonsingular,If is nonsingular,The d by n matrix is call the The d by n matrix is call the pseudoinversepseudoinverse of Y.of Y.n nRemarks:For an arbitrarily fixed b,MSE solution Remarks:For an arbitrarily fixed b,MS

6、E solution may not be a separating vector.may not be a separating vector.3 34 4n nThe Widrow-Hoff or LMS Procedure Need not converge to a separating hyperplane even if there exist one5 55.9 The Ho-Kashyap Proceduren nTake the criterion function as a function of two variables a and b:n nIf the traini

7、ng samples are linearly separable,then there should exist an such that:If we knew such beforehand.We would get the separating vector using the MSE procedure 6 67 75.12 Multicategory GeneralizationsnMulticategory casenLinearly-separable for Multicategory case there exists a set of weight vectors 8 8n

8、 nKeslersKeslers Construction Construction suppose that ,we have suppose that ,we have 9 9n nRemark:it allows us to convert many Remark:it allows us to convert many muylticategorymuylticategory error-correction procedures to two-category error-correction procedures to two-category procedures for the

9、 purpose of obtaining a procedures for the purpose of obtaining a convergence proofconvergence proofn nFixed-Increment RuleFixed-Increment RuleLet denote the Let denote the kthkth sample requiring sample requiring correction,and suppose that ,there must correction,and suppose that ,there must be at

10、least one j i for which be at least one j i for which then the fixed-increment rule for correcting then the fixed-increment rule for correcting is is 1010n nRemark:KeslersRemark:Keslers construction can also be used construction can also be used to extend all of results for the to extend all of resu

11、lts for the PerceptronPerceptron and and relaxation procedure to the relaxation procedure to the multicategorymulticategory casecase1111n nGeneralization for MSE ProcedureGeneralization for MSE Procedure consider consider multicategorymulticategory case as a set of c two-case as a set of c two-class problemclass problem1212