《人工智能与数据挖掘教学课件》l.ppt

《《人工智能与数据挖掘教学课件》l.ppt》由会员分享，可在线阅读，更多相关《《人工智能与数据挖掘教学课件》l.ppt（30页珍藏版）》请在淘文阁 - 分享文档赚钱的网站上搜索。

1、2023/2/122023/2/12AI&DMAI&DM1 1Chapter 8Neural NetworksPart III:Advance Data Mining Techniques2023/2/122023/2/12AI&DMAI&DM2 21.What&Why ANN(8.1 Feed forward Neural Network)2.How ANN works-working principle(8.2.1 Supervised Learning)3.Most popular ANN-Backpropagation Network(8.5.1 The Backpropagation

2、 Algorithm:An example)Content2023/2/122023/2/12AI&DMAI&DM3 31.What&Why ANN:Artificial Neural Networks(ANN)ANN is an information processing technology that emulates a biological neural network.Neuron(Neuron(神经元神经元)vs Node(Transformation)vs Node(Transformation)Dendrite(Dendrite(树突树突)vs Input)vs Input

3、Axon(Axon(轴突轴突)vs Output)vs Output Synapse(Synapse(神经键神经键)vs Weight)vs WeightStarts in 1970s,become very popular in 1990s,because of the advancement of computer technology.2023/2/122023/2/12AI&DMAI&DM4 42023/2/122023/2/12AI&DMAI&DM5 52023/2/122023/2/12AI&DMAI&DM6 6What is ANN:Basics Types of ANNType

4、s of ANN Network structure,e.g.Figure 17.9&17.10(Turban,2000,version 5,Network structure,e.g.Figure 17.9&17.10(Turban,2000,version 5,p663)p663)Number of hidden layersNumber of hidden layers Number of hidden nodesNumber of hidden nodes Feed forward and feed backward(time dependent problems)Feed forwa

5、rd and feed backward(time dependent problems)Links between nodes(exist or absent of links)Links between nodes(exist or absent of links)The ultimate objectives of training:obtain a set of weights that makes all the The ultimate objectives of training:obtain a set of weights that makes all the instanc

6、es in the training data predicted as correctly as possible.instances in the training data predicted as correctly as possible.Back-propagation is one type of ANN which can be used for classification Back-propagation is one type of ANN which can be used for classification and estimationand estimation

7、multi-layer:Input layer,Hidden layer(s),Output layermulti-layer:Input layer,Hidden layer(s),Output layer Fully connected Fully connected Feed forwardFeed forward Error back-propagationError back-propagation2023/2/122023/2/12AI&DMAI&DM7 71.What&Why ANN(8.1 Feed forward Neural Network)2.How ANN works-

8、working principle(8.2.1 Supervised Learning)3.Most popular ANN-Backpropagation Network(8.5.1 The Backpropagation Algorithm:An example)Content2023/2/122023/2/12AI&DMAI&DM8 82.How ANN:working principle(I)Step 1Step 1:Collect data:Collect dataStep 2Step 2:Separate data into training and test:Separate d

9、ata into training and test sets for network training and validation sets for network training and validation respectivelyrespectivelyStep 3Step 3:Select network structure,learning:Select network structure,learning algorithm,and parametersalgorithm,and parameters Set the Set the initial weightsinitia

10、l weights either by rules or randomly either by rules or randomly Rate of learningRate of learning(pace to adjust weights)(pace to adjust weights)Select learning algorithm(Select learning algorithm(More than a hundred More than a hundred learning algorithms available for various situations and learn

11、ing algorithms available for various situations and configurations)configurations)2023/2/122023/2/12AI&DMAI&DM9 92.ANN working principle(II)Step 4Step 4:Train the network:Train the network Compute outputsCompute outputs Compare outputs with desired targets.The difference Compare outputs with desired

12、 targets.The difference between the outputs and the desired targets is called deltabetween the outputs and the desired targets is called delta Adjust the weights and repeat the process to minimize the Adjust the weights and repeat the process to minimize the delta.delta.The objective of training is

13、to The objective of training is to MinimizeMinimize the Delta(Error).the Delta(Error).The final result of training is a set of weights.The final result of training is a set of weights.Step 5Step 5:Test the network:Test the network Use test set:comparing test results to historical results,to Use test

14、 set:comparing test results to historical results,to find out the accuracy of the networkfind out the accuracy of the network Step 6Step 6:Deploy developed network application if the:Deploy developed network application if the test accuracy is acceptabletest accuracy is acceptable2023/2/122023/2/12A

15、I&DMAI&DM10102.ANN working principle(III):ExampleExample 1:OR operation(see table below)Two input elements,X1 and X2InputsCaseX1X2Desired Results1 0002011(positive)3101(positive)4111(positive)2023/2/122023/2/12AI&DMAI&DM11112.ANN working principle(IV):Example Network structure:one layer(see next pag

16、e)Network structure:one layer(see next page)Learning algorithmLearning algorithm Weighted sum-summation function:Weighted sum-summation function:Y1=Y1=XiWiXiWi Transformation(transfer)function:Y1 less than threshold,Y=0;Transformation(transfer)function:Y1 less than threshold,Y=0;otherwise Y=1otherwi

17、se Y=1 Delta=Z-YDelta=Z-Y Wi(final)=Wi(initial)+Alpha*Delta*XiWi(final)=Wi(initial)+Alpha*Delta*Xi Initial Parameters:Initial Parameters:Rate of learning:alpha=0.2Rate of learning:alpha=0.2Threshold=0.5;Threshold=0.5;Initial weight:0.1,0.3Initial weight:0.1,0.3 Notes:Notes:Weights are initially rand

18、om Weights are initially random The value of The value of learning rate-learning rate-alpha,is set low first.alpha,is set low first.2023/2/122023/2/12AI&DMAI&DM1212Processing Informationin an Artificial Neuronx x1 1w w1j1jx x2 2Y Yj jw w2j2jNeuron jNeuron j w wij ij x xi iWeightsWeightsOutputOutputI

19、nputsInputsSummationsSummationsTransfer functionTransfer function2023/2/122023/2/12AI&DMAI&DM13131.What&Why ANN(8.1 Feed forward Neural Network)2.How ANN works-working principle(8.2.1 Supervised Learning)3.Most popular ANN-Backpropagation Network(8.5.1 The Backpropagation Algorithm:An example)Conten

20、t2023/2/122023/2/12AI&DMAI&DM14143.Back-propagation NetworkNetwork Topologymulti-layer:Input layer,Hidden layer(s),Output multi-layer:Input layer,Hidden layer(s),Output layerlayerFully connected Fully connected Feed forwardFeed forwardError back-propagationError back-propagationInitialize weights wi

21、th random values2023/2/122023/2/12AI&DMAI&DM1515Back-propagation NetworkOutput nodesOutput nodesInput nodesInput nodesHidden nodesHidden nodesOutput vectorOutput vectorInput vector:Input vector:x xi iw wij ij2023/2/122023/2/12AI&DMAI&DM16163.Back-propagation NetworkFor each nodeFor each node1.Comput

22、e the 1.Compute the net inputnet input to the unit using summation function to the unit using summation function 2.Compute the 2.Compute the output valueoutput value using the activation function(i.e.using the activation function(i.e.sigmoid function)sigmoid function)3.Compute the 3.Compute the erro

23、rerror4.Update the 4.Update the weights(and the bias)weights(and the bias)based on the error based on the error5.Terminating conditions:5.Terminating conditions:all all w wij ij in the previous epoch(in the previous epoch(周期周期)were so small as to be below some were so small as to be below some speci

24、fied thresholdspecified threshold the percentage of samples misclassified in the previous epoch is below some the percentage of samples misclassified in the previous epoch is below some thresholdthreshold a pre-specified number of epoch has expireda pre-specified number of epoch has expired2023/2/12

25、2023/2/12AI&DMAI&DM1717Backpropagation Error Output Layer2023/2/122023/2/12AI&DMAI&DM1818Backpropagation Error Hidden Layer2023/2/122023/2/12AI&DMAI&DM1919The Delta Rule2023/2/122023/2/12AI&DMAI&DM2020Root Mean Squared Error2023/2/122023/2/12AI&DMAI&DM21213.Back-propagation(cont.)Increase network ac

26、curacy and training speedIncrease network accuracy and training speed Network topologyNetwork topology number of nodes in input layernumber of nodes in input layer number of hidden layers(usually is one,no more than two)number of hidden layers(usually is one,no more than two)number of nodes in each

27、hidden layernumber of nodes in each hidden layer number of nodes in output layernumber of nodes in output layer Change initial weights,learning parameter,terminating conditionChange initial weights,learning parameter,terminating condition Training process:Training process:Feed the training instances

28、Feed the training instances Determine the output errorDetermine the output error Update the weightsUpdate the weights Repeat until the terminating condition is metRepeat until the terminating condition is met2023/2/122023/2/12AI&DMAI&DM2222Supervised Learning with Feed-Forward Networks Backpropagati

29、on Learning2023/2/122023/2/12AI&DMAI&DM2323Summary:Decisions the builder must makeNetwork Topology:number of hidden layers,number of nodes in each layer,and feedbackLearning algorithms Parameters:initial weight,learning rateSize of training and test dataStructure and parameters determine the length

30、ofStructure and parameters determine the length oftraining time and the accuracy of the networktraining time and the accuracy of the network2023/2/122023/2/12AI&DMAI&DM2424Neural Network Input Format(Normalization:categorical to numerical)1.All input and output must numerical and between 0,1All inpu

31、t and output must numerical and between 0,12.Categorical Attributes.Categorical Attributes.e.g.attribute with 4 possible valuese.g.attribute with 4 possible valuesOrdinal:Set to 0,0.33,0.66,1Ordinal:Set to 0,0.33,0.66,1Nominal:Set to 0,0,0,1,1,0.1,1 Nominal:Set to 0,0,0,1,1,0.1,1 3.Numerical Attribu

32、tes:Numerical Attributes:2023/2/122023/2/12AI&DMAI&DM2525Neural Network Output Format1.Categorical Attributes:(Numerical to(Numerical to categorical)categorical)Type 0&1Type 0.452.Numerical Attributes:(0,1 to ordinary value)(0,1 to ordinary value)Min+X*(Max-min)2023/2/122023/2/12AI&DMAI&DM2626Homewo

33、rkP264,Computational Questions-2 r=0.5,Tk=0.65Adjust all weights for one epoch2023/2/122023/2/12AI&DMAI&DM2727 Case StudyExample:Bankruptcy Prediction with Neural NetworksStructure:Three-layer network,back-propagationTraining data:Small set of well-known financial ratiosData available on bankruptcy

34、outcomes Supervised network2023/2/122023/2/12AI&DMAI&DM2828Architecture of the Bankruptcy Prediction Neural NetworkX4X4X3X3X5X5X1X1X2X2Bankrupt 0Bankrupt 0Not bankrupt 1Not bankrupt 12023/2/122023/2/12AI&DMAI&DM2929Bankruptcy Prediction:Network architectureBankruptcy Prediction:Network architecture

35、Five Input NodesFive Input NodesX1:Working capital/total assets X1:Working capital/total assets X2:Retained earnings/total assetsX2:Retained earnings/total assetsX3:Earnings before interest and taxes/total X3:Earnings before interest and taxes/total assetsassetsX4:Market value of equity/total debtX4

36、:Market value of equity/total debtX5:Sales/total assetsX5:Sales/total assets Single Output NodeSingle Output Node:Final classification for each firm:Final classification for each firm Bankruptcy or Bankruptcy or NonbankruptcyNonbankruptcy Development Tool:NeuroShellDevelopment Tool:NeuroShell2023/2/

37、122023/2/12AI&DMAI&DM3030 DevelopmentDevelopment Three-layer network with Three-layer network with back-error propagation back-error propagation(Turban,figure(Turban,figure 17.12,p669)17.12,p669)Continuous valued inputContinuous valued input Single output node:0=bankrupt,1=not bankrupt(Single output

38、 node:0=bankrupt,1=not bankrupt(Nonbankruptcy)Nonbankruptcy)TrainingTraining Data Set:129 firmsData Set:129 firms Training Set:74 firms:38 bankrupt,36 notTraining Set:74 firms:38 bankrupt,36 not TestingTesting Test data set:55 firms:27 bankrupt firms,28 nonbankrupt firmsTest data set:55 firms:27 bankrupt firms,28 nonbankrupt firms The neural network correctly predicted:The neural network correctly predicted:81.5 percent bankrupt cases 81.5 percent bankrupt cases 82.1 percent nonbankrupt cases82.1 percent nonbankrupt cases