TLBO优化最小二乘支持向量机参数LSSVM.pdf

《TLBO优化最小二乘支持向量机参数LSSVM.pdf》由会员分享，可在线阅读，更多相关《TLBO优化最小二乘支持向量机参数LSSVM.pdf（10页珍藏版）》请在淘文阁 - 分享文档赚钱的网站上搜索。

1、Model NOx emissions by least squares support vector machine withtuning based on ameliorated teachinglearning-based optimizationGuoqiang Lia,b,Peifeng Niua,b,Weiping Zhanga,b,Yongchao Liua,baKey Lab of Industrial Computer Control Engineering of Hebei Province,Yanshan University,Qinhuangdao 066004,Chi

2、nabNational Engineering Research Center for Equipment and Technology of Cold Strip Rolling,Qinhuangdao 066004,Chinaa b s t r a c ta r t i c l ei n f oArticle history:Received 14 November 2012Received in revised form 19 April 2013Accepted 24 April 2013Available online 2 May 2013Keywords:Teachinglearn

3、ing-based optimizationLeast squares support vector machineNOx emissionsCoal-fired boilerThe teachinglearning-based optimization(TLBO)is a new efficient optimization algorithm.To improve thesolution quality and to quicken the convergence speed and running time of TLBO,this paper proposes anameliorate

4、d TLBO called A-TLBO and test it by classical numerical function optimizations.Compared withother several optimization methods,A-TLBO shows better search performance.In addition,the A-TLBO isadopted to adjust the hyper-parameters of least squares support vector machine(LS-SVM)in order tobuild NOx em

5、issions model of a 330MW coal-fired boiler and obtain a well-generalized model.Experimentalresults show that the tuned LS-SVM model by A-TLBO has well regression precision and generalizationability.2013 Elsevier B.V.All rights reserved.1.IntroductionWith the increase of energy consumption worldwide

6、and im-proved awareness of environmental protection,boiler combustionoptimization problem of power plants attracts the attention of techni-cal staffs and managers.The boiler combustion optimization technol-ogy is used to ensure boiler efficiency,and simultaneously to reducepollutant emissions,where

7、the NOx emissions are the main compo-nents.So the core task is to cut down NOx emissions.However,thefirst work of controlling NOx emissions is to set up a high precisionprediction model.So building an accurate system model is very im-portant for monitoring and optimizing the operations of powerplant

8、s.In the past ten years,many research works on how to modeland forecast the NOx emissions of high capacity coal-fired boilershave been published 15.However,the traditional statistical analy-sis and forecasting methods are always based on large sample data,and many of the prediction methods,such as a

9、rtificial neural net-works,have theoretical assurance all just under a large sample.Dueto various limits in actual circumstances,it is very difficult to gatherlarge sample data.For this problem,the least squares support vectormachine(LS-SVM)6,which is suited for small sample data,isadopted to model

10、and predict NOx emissions.The LS-SVM is a reformulation to the standard support vector ma-chine(SVM)79,which simplifies the standard SVM model in agreat extent by applyinglinear least squares criteria to the loss functionto replace traditional quadratic programming method.The simplicityand inherited

11、 advantages of SVM such as excellent generalization abili-ty and a unique solution promote the application of LS-SVM in manypattern recognition and regression problems.The regressionaccuracyand generalization abilityof LS-SVMare ex-tremely dependent on two hyper-parameters:the regularization pa-rame

12、ter and the kernel parameter 2.So choosing appropriatehyper-parameters is very important for obtaining excellent generaliza-tion ability.Parameter choosing of the LS-SVM model could be thoughtas an essential optimization task.This calls for the use of advancedmeta-heuristic approaches,such as evolut

13、ionary or population-basedmethods.The teachinglearning-based optimization algorithm 10,11 is anew and efficient meta-heuristic optimization method based on thephilosophy of teaching and learning,which is proposed by Rao et al.Like other population-based optimization techniques such as particleswarm

14、optimization(PSO)12,evolutionary optimization(DE)13,14,artificial bee colony(ABC)1519,Gravitational Search Algo-rithm(GSA)20,and Coupled Simulated Annealing(CSA)21,theTLBOisalsoapopulation-basedoptimizationmethodandadoptsapop-ulationofsolutionstoproceedtotheglobalsolution.Insomeresearches2224,the pe

15、rformance of TLBO has already been compared withother search optimization techniques such as genetic algorithm(GA)25,26,Bee algorithm(BA)27,and grenade explosion method(GEM)28.In addition,the TLBO has been applied to some complexcomputational problems,such as data clustering,mechanical design,electr

16、ochemicaldischargemachining,anddesignofplanarsteelframes.In this paper,in order to improve the solution quality and toquicken the convergence speed of TLBO,an ameliorated teachinglearning-based optimization algorithm called A-TLBO is proposed.InChemometrics and Intelligent Laboratory Systems 126(201

17、3)1120 Correspondingauthorat:KeyLabofIndustrialComputerControlEngineeringofHebeiProvince,Yanshan University,Qinhuangdao 066004,China.Tel.:+86 13230355970;fax:+86 335 8072979.E-mail address:(P.Niu).0169-7439/$see front matter 2013 Elsevier B.V.All rights reserved.http:/dx.doi.org/10.1016/j.chemolab.2

18、013.04.012Contents lists available at SciVerse ScienceDirectChemometrics and Intelligent Laboratory Systemsjournal homepage: are three major differences:the greedy selection mech-anism is not adopted but the elitist strategy,an inertia weight func-tion and an acceleration coefficient function are in

19、troduced toquicken the processes of Teaching and Learning.In order to testthe validity of the proposed method,it is adopted to optimize manyclassical numerical optimization functions and compared with othermethods.Experiment results show that the A-TLBO could find bettersolutions and have much faste

20、r convergence speed.In addition,theA-TLBO is also used to adjust two hyper-parameters of LS-SVM inorder to obtain a well-generalized model of NOx emissions for a330MW coal-fired boiler.Results show that the tuned LS-SVMmodel by A-TLBO has well regression precision and generalizationability.The rest

21、of the paper is arranged as follows.In the next section,abrief literature review is presented.The ameliorated teachinglearning-based optimization is proposed in Section 3.In Section 4,the A-TLBO is applied to optimize some classical numerical optimiza-tion functions and compared with GSA,ABC and TLB

22、O.In Section 5,the A-TLBO is employed to adjust the hyper-parameters of LS-SVMto model NOx emissions of a 330MW coal-fired boiler.Finally,Section 6 concludes the paper.2.Review of related works2.1.Teachinglearning-based optimizationThe teachinglearning-based optimization(TLBO)algorithm pro-posed by

23、Rao is inspired by the effect of the influence of a teacheron the output of learners in a class.In TLBO,there are two vital com-ponents,Teacher phase and Learner phase,which indicate two dif-ferent learning modes.2.1.1.Teacher phaseIn this phase,Learners learn from a teacher,who is considered asthe

24、most knowledgeable person in the society and would bringlearners up to his or her level in terms of knowledge.That is to say,the teacher would put effort to move the mean of a class up to hisor her level depending on his or her ability.Suppose,in the ith itera-tion,Miis the mean of marks obtained by

25、 learners in a class,and Tiisthe mark of the teacher.And the best learner could be mimicked asthe teacher,namely:Ti Xmin f X:1The teacher would put effort to move the mean value Mitowardsitself.Namely,the new mean Mnewwill be Ti.The learners would learn and update their knowledge according tothe fol

26、lowing form:Xnew;i Xold;i riMnewTFMi2where Xold,iand Xnew,irespectively denote the ith learners mark be-fore or after learning from the teacher;riis a random number be-tween 0 and 1;and TFis a teaching factor which controls themovement of the mean value.The value of TFis either 1 or 2,whichis a heur

27、istic step and decided randomly by:TF round 1 rand 0;1 21?:32.1.2.Learner phaseIn this phase,the learners could increase their knowledge by in-teractions among themselves.A learner could communicate random-ly with other learners in order to improve his or her knowledge withthe help of group discussi

28、ons,formal communications,presentations,etc.A learner learns something new from the learners who have moreknowledge than him or her.The learning process could be describedas follows:In the ith iteration,for a learner Xi,randomly select another learn-er Xjwhere i jXnew;iXold;i riXiXj?if f Xif Xj?Xold

29、;i riXjXi?if f Xi f Xj?:8:5where is the regularization parameter,and iand iare the slackvariables.Compared with SVM,the least squares support vector machine(LS-SVM)applies linear least squares criteria to the loss function toreplace the inequality constraints with equality constraints.So theoptimiza

30、tion problem can be defined as:min;e 12T 12Xli1e2isubject to:yi wT xi b ei;i 1;2;l:6The Lagrangian function can be set up by:L;b;e;eXli1iT xi b eiyi?7where iis the ith Lagrange multiplier.The KarushKuhnTucker(KKT)conditions for optimality are de-scribed as follows:L 0 Xli1i xi Lb 0Xli1i 0Lei 0i CeiL

31、i 0T xi b ei yi:8:8After eliminating variables and ei,the optimization problemcould be transformed into the following form:1Il1l1Tl0264375b?y0?912G.Li et al./Chemometrics and Intelligent Laboratory Systems 126(2013)1120where y y1;yl?T;1l 1;1?T;1;l?Tand =ijl l.ij K xi;xj?xi T xj?:10Then,the explicit

32、solution of Eq.(9)can be expressed as follows:b 1Tl 1Il?1y1Tl 1Il?11l 1Il?1y1lb:11According to Mercers theorem,the output result could be calcu-lated by:f x Xli1iK x;xi b12K x;xi expxxi222!13where is the Gaussian kernel,and the kernel function K(x,xi)is se-lected as the radial basis function(RBF)in

33、this paper.3.Ameliorated teachinglearning-based optimizationIn this section,an ameliorated TLBO algorithm called A-TLBO isproposed to quicken the convergence speed and simultaneously toimprove the search accuracy of the original TLBO.In the A-TLBO,there are three major differences:by introducing the

34、 elitist strategyto replace the greedy selection mechanism,the inertia weight and ac-celeration coefficients to quicken the processes of Teaching andLearning.The A-TLBO is described as follows in detail.3.1.Initial parametersIn A-TLBO,the parameters:the number of learners and the maxi-mum iteration

35、number are set in this step like that of TLBO.3.2.Teacher phaseIn this step,the new obtained mark by a learner mainly depends ontwoparts:thepreviousmarkXold,iandthedifference(Mnew TFMi)be-tweentheexistingandthenewmean.Inordertoimprovethequalityofteaching and simultaneously quicken the process of tea

36、ching,the solu-tion search equation described by Eq.(2)introduces two important pa-rameters which decide the influence degree of the two parts,and ismodified as follows:Xnew;i iXold;i iMnewTFMi14where iis the inertia weightthat controls impact of the previous solu-tion Xold,i;iis the acceleration co

37、efficient between 0 and 1,which de-cidesthemaximumstepsize.However,iftheobtainedmarkisverylow,there is a big gap between the learner and the teacher.So a big correc-tion is needed in order to improve the learners mark.Conversely,if theobtained mark is very high,only a small modification is needed.So

38、 inorder to further improve the learning efficiency of the learner,inertiaweight and the acceleration coefficient are defined as functions of thefitness in the learning process.These parameters are given as follows:i 1=1 exp fit i =ap?iter15wi 1=1 exp fit i =apiter?16where fit(i)is the fitness of th

39、e ith learner;ap is the maximum fitnessvalues among learners in the first iteration;iter is the current iteration.NoYesFig.1.The flowchart of the A-TLBO algorithm.13G.Li et al./Chemometrics and Intelligent Laboratory Systems 126(2013)1120In addition,after all the learners update their marks by Eq.(1

40、4),they calculate their fitness values together in the form of a matrixby the objective function.The new marks and fitness values wouldnttake place the previous marks and fitness values but are respectivelymerged together with the previous ones as two new sets.And thenthe fitness set would be reorde

41、red as the sort ascends,and the markset would be reordered depending on the order of the fitness set.Fi-nally,the first half of fitness set would be assigned to the all learnersas their fitness values,and the corresponding values of the mark areset as the marks of all learners.Namely,during the upda

42、ting marks,the greed selection mechanism is not adopted but an elitist strategy.This strategy could decrease a lot of time which could be expendedin comparison with each other,and simultaneously increase learningefficiency.3.3.Learner phaseIn this step,the learners increase their knowledge by commun

43、i-cating each other.Here a learner learns something new from somelearners who have more knowledge than him or her.A learnercould update their mark by the following form:In ith iteration,for a learner Xi,randomly select another learner Xjwhere i jXnew;iXold;i iXjXi?if f Xif Xj?Xold;i iXbestXiif f Xi

44、f Xj?8:17i 1exp fit Xj?fit Xi?18i 1exp fit Xbestfit Xi19where Xbestis best learner in a class;iand iare acceleration coeffi-cients which decide the step size depending on the difference be-tween two learners.When all learners update their marks by Eq.(17),they could cal-culate their fitness values t

45、ogether.And then the elitist strategywould be adopted like the teaching phase in order to obtain newmarks and fitness values for all learners.3.4.Main steps of A-TLBOBased on the above explanation of initializing population,theTeacher phase,and the Learner phase,the flowchart of the A-TLBO al-gorith

46、m is shown in Fig.1.4.Experimental study and discussion4.1.Experimental setupIn this subsection,in order to test the performance of A-TLBO,tenfamous benchmark optimization problems are adopted,which aredescribed in Table 1.In Table 1,n denotes the dimension,S denotesa subset of Rn,and synchronously

47、the global optimum solutions andlocations of classical functions are also given in Table 1.Moreover,the A-TLBO is compared with other population-based optimizationtechniques:GSA,ABC and the original TLBO.The parameter settingof each optimization algorithm is given in Table 2.All the experiments are

48、carried out under Windows 2000 andMatlab 2009 with an AMD Athlon(tm)64 X2 Dual Core Processor5000+,2.61 GHz and 2 GB RAM.4.2.Comparison with other methodsIn this subsection,a series of experiments are presented to showthe superiority of A-TLBO in convergence speed and quality solutionswith GSA,ABC a

49、nd the original TLBO.For each function we make 30independent experiments,determine the runtime and the conver-gence time,and evaluate the best solution found and the standard de-viation.The results are shown in Tables 34 and Figs.2,3,4,and 5.Table 2Parameter setting.MethodPopulationsizeMaximumiterat

50、ionThe dim ofeach objectOtherGSA40100010,30,50,100G0=100,=20ABC40100010,30,50,100Limit=200TLBO40100010,30,50,100A-TLBO40100010,30,50,100Table 1Classical benchmark functions.Test functionGlobal optimaOptimum locationSf1x?Xni1x2i00n100,100nf2x?Xni1xij j ni1xij j00n10,10nf3x?Xni1Xij1xj01A200n100,100nf4