《根据贝氏定理为基础》由会员分享，可在线阅读，更多相关《根据贝氏定理为基础（47页珍藏版）》请在金锄头文库上搜索。
1、Chapter 5 Statistical MethodsBy Jinn-Yi Yeh Ph.D. 4/7/2009Outline5.1 STATISTICAL INFERENCE 5.2 ASSESSING DIFFERENCES IN DATA SETS 5.3 BAYESIAN INFERENCE 5.4 PREDICTIVE REGRESSION 5.5 ANALYSIS OF VARIANCE 5.6 LOGISTIC REGRESSION 5.7 LOG-LINEAR MODELS 5.8 LINEAR DISCRIMINANT ANALYSIS5.1 STATISTICAL IN
2、FERENCE lDescriptive statistics V.SStatistical inferencelPopulation, Sample, Data setlParameter V.S StatisticlInference methods : estimation, and tests of hypotheses5.1 STATISTICAL INFERENCE (cont.)lEstimation: The goal is to gain information from a data set T in order to estimate one or more parame
3、ters w belonging to the model of the real-world system f(X, w)5.1 STATISTICAL INFERENCE (cont.)lstatistical testing: to decide whether a hypothesis concerning the value of the population characteristic should be accepted or rejected lnull hypothesis V.S alternative hypothesis5.2 ASSESSING DIFFERENCE
4、S IN DATA SETS lcentral tendency l1l2l35.2 ASSESSING DIFFERENCES IN DATA SETS (cont.)ldata dispersion l1l2 5.2 ASSESSING DIFFERENCES IN DATA SETS (cont.)lBoxplotlIn many statistical software tools, a popularly used visualization tool of descriptive statistical measures for central tendency and dispe
5、rsion 5.3 BAYESIAN INFERENCE lNave Bayesian Classification Process (Simple Bayesian Classified)l根據貝氏定理為基礎，用以判斷未知類別的 資料應該最接近哪一個類別l監督式學習方式(需訓練資料)lP(H/X):事後機率lP(H):事前機率5.3 BAYESIAN INFERENCE (cont.)lGiven an additional data sample X (its class is unknown), it is possible to predict the class for X usin
6、g the highest conditional probability P(Ci/X)lP(X):constant for all classes，only the product P(X/Ci) P(Ci) needs to be maximized lP(Ci):Ci/m (m is total number of training samples)5.3 BAYESIAN INFERENCE - exampleTable 5.1: Training data set for a classification using Nave Bayesian Classifier SampleA
7、ttribute 1 A1 Attribute 2 A2 Attribute 3 A3 Class C112112001132122412125012162222710115.3 BAYESIAN INFERENCE example (cont.)lGoal：to predict classification of the new sample X = 1, 2, 2, class = ?lmaximize the product P(X/Ci) P(Ci) for i = 1,2 lStep1:compute prior probabilities P(Ci) 5.3 BAYESIAN IN
8、FERENCE example (cont.)lStep2:compute conditional probabilities P(xt/Ci) for every attribute value given in the new sample X = 1, 2, 2, C = ?5.3 BAYESIAN INFERENCE example (cont.)lStep3:Under the assumption of conditional independence of attributes, compute conditional probabilities P(X/Ci)l 5.3 BAY
9、ESIAN INFERENCE example (cont.)lFinally: multiplying these conditional probabilities with corresponding priori probabilitieslobtain values proportional () to P(Ci/X) and find the maximum 5.4 PREDICTIVE REGRESSIONlThe prediction of continuous values can be modeled by a statistical technique called re
10、gression.lRegression analysis is the process of determining how a variable Y is related to one or more other variables X1, X2, , Xn. lModeling this type of relationship is often called linear regression. lThe relationship that fits a set of data is characterized by a prediction model called a regres
11、sion equation. The most widely used form of the regression model is the general linear model formally written as Y=+1X1+2X2 +3X3+ +nXnSimple regressionlSimple regression：Y = + XlSSE：l and ：Multiple regressionlMultiple regression： Y = + 1X1 + 2X2 + 3X3 + + nXnlSSE = (Y - X) (Y - X)l(SSE)/=0 =(XX)-1(X
12、Y)correlation coefficientcorrelation coefficient (cont.)lA correlation coefficient r = 0.85 indicates a good linear relationship between two variables. Additional interpretation is possible. Because r2 = 0.72, we can say that approximately 72% of the variations in the values of Y is accounted for by
13、 a linear relationship with X. 5.5 ANALYSIS OF VARIANCElOften the problem of analyzing the quality of the estimated regression line and the influence of the independent variables on the final regression is handled through an analysis-of- variance approach. lThe size of the residuals, for all m sampl
14、es in a data set, is related to the size of variance 2 and it can be estimated by:lThe numerator is called the residual sum while the denominator is called the residual degree of freedom. lThe criteria are basic decision steps in the ANOVA algorithm in which we analyze the influence of input variabl
15、es on a final model. lFirst, we start with all inputs and compute S2 for this model. Then, we omit inputs from the model one by one. lF = S2new / S2oldMultivariate analysis of variancelMultivariate analysis of variance is a generalization of the previously explained ANOVA analysis. Yj=+1X1j+2X2j +3X
16、3j+nXnj+j J=1,2,mlThe corresponding residuals for each dimension will be (Yj Yj).lClassical multivariate analysis also includes the method of principal component analysis. This method has been explained in Chapter 3 when we were talking about data reduction and data transformation as preprocessing phases for data mining.5.6 Logistic RegressionlT
