《模式识别Chapter 3》由会员分享，可在线阅读，更多相关《模式识别Chapter 3（56页珍藏版）》请在金锄头文库上搜索。
1、Pattern Recognition北京交通大学北京交通大学电子信息工程学院电子信息工程学院2021/6/161Character RecognitionCorresponding to a;Corresponding to b;2021/6/162Feature space字符的圆形度字符的圆形度字符的字符的 长宽比长宽比Mapping the input pattern onto points in a feature spaceFeature Space2021/6/163Character RecognitionOnce mapping the input pattern onto
2、points in a feature space, the purpose of classification is to : assign each point in the space with a class label2021/6/164模式识别系统模式识别系统2021/6/165宽度宽度 特特 征征 空空 间间长度长度decision regionsdecision boundariesdiscriminant function2021/6/166Pattern Recognition Assign each point in the space with a class labe
3、l statistical methods artificial neural network methods structure methods2021/6/167Bayes theorem in general 2021/6/168Bayes Decision: error minimumThe probability of misclassification is minimized by selecting the class having the largest posterior probability2021/6/169Bayes Decision: risk minimum I
4、f the likelihood ratio of class and exceeds a threshold value (that is independent of the input pattern ), the optimal action is to decide . 2021/6/1610Discriminant functionsn Bayesian decisionn Normal densityHow about the discriminant function?2021/6/1611Discriminant functionsoCase 1Linear Discrimi
5、nant Function(LDF)2021/6/1612Discriminant functionsoCase 2Linear Discriminant Function(LDF)2021/6/1613Discriminant functionsoCase 3Quadratic Discriminant Function(QDF)2021/6/1614Overviewn Pattern classification problemn Feature space, feature point in spacen Classification - Bayesian decision theory
6、 - Discriminant function - Decision region, Decision boundary2021/6/1615Exampleo分别写出在以下两种情况下的最小错误率贝叶斯决分别写出在以下两种情况下的最小错误率贝叶斯决策规则：策规则： A： (1) If (2) If 2021/6/1616Exampleo对两类问题，证明最小风险贝叶斯决策规则可表示对两类问题，证明最小风险贝叶斯决策规则可表示为：为： A： 2021/6/1617Exampleo对两类问题，证明最小风险贝叶斯决策规则可表示对两类问题，证明最小风险贝叶斯决策规则可表示为：为： A： 2021/6/1
7、618Exampleo对两类问题，证明最小风险贝叶斯决策规则可表示对两类问题，证明最小风险贝叶斯决策规则可表示为：为： A： 2021/6/1619Exampleo说明在说明在0-1损失的情况下最小风险贝叶斯决策规则与损失的情况下最小风险贝叶斯决策规则与最小错误率贝叶斯决策规则相同。最小错误率贝叶斯决策规则相同。 A： MinimumMaximum2021/6/1620Chapter 3 Maximum-likelihood and Bayesian parameter estimationo Introductiono Maximum-likelihoodo Bayesian paramet
8、er estimationo Gaussian classifier 2021/6/1621Introductionowe could design an optional classifier if we knew the priori probabilities and the class-conditional densities Unfortunately, we rarely, if ever, have this kind of completely knowledge about the probabilistic structure of the problem 2021/6/
9、1622Probability Density EstimationoThree alternative approaches to density estimation - parametric methods - non- parametric methods - semi- parametric methods2021/6/1623Probability Density EstimationoParametric methods - a specific functional form is assumed - a number of parameters which are then
10、optimized by fitting the model of the data seto Drawbacks - form of function chosen might be incapable 2021/6/1624Probability Density EstimationoNon- parametric methods - not assume the form - to be determined entirely by the dataoDrawbacks - the number of parameters grows with the size of the data
11、- slow 2021/6/1625Probability Density EstimationoSemi- parametric methods - use the very general function form - the number of adaptive parameters can be increased in a systematic way - build a more flexible models oNeural network 2021/6/1626Probability Density EstimationoParametric methods - Maximu
12、m likelihood estimation - Bayesian estimation2021/6/1627Maximum Likelihood Estimation oc Data sets samples having been drawn independently according to the probability law oWe assume that has known parametric form, and therefore determined uniquely by the value of a parameter vector . 2021/6/1628Max
13、imum Likelihood Estimation o Suppose that D contains n samples . o Because the samples are drawn independently, we have: 2021/6/1629Maximum Likelihood Estimation o is called as the likelihood of . The maximum-likelihood estimation of is the value that maximize . 2021/6/1630Maximum Likelihood Estimat
14、ion 2021/6/1631Example oThe Gaussian case: Unknown 2021/6/1632oThe Gaussian case: Unknown Example 2021/6/1633Example oThe Gaussian case: Unknown and 2021/6/1634Example oThe Gaussian case: Unknown and 2021/6/1635Example oThe Gaussian case: Unknown and 2021/6/16362021/6/16372021/6/16382021/6/16392021/
15、6/16402021/6/1641Gaussian Mixture2021/6/1642Bayes Estimation oWhereas in maximum-likelihood methods, we view the true parameter vector to be fixed, in Bayesian method, we consider to be a random variable, and the training data allows us to convert a distribution on this variable into a posterior pro
16、bability density. 2021/6/1643模式识别系统模式识别系统Gaussian densityGaussian ClassifierEstimate mean vector and covariance matrix2021/6/1644Gaussian Classifierso概率密度函数o分类函数2021/6/1645n假设独立等方差Nearest distance (nearest mean)同时也是线性鉴别函数 (LDF)Gaussian Classifiers2021/6/1646o假设等协方差矩阵nLinear discriminant function (LD
17、F)Gaussian Classifiers2021/6/1647Gaussian Classifierso假设任意协方差矩阵且等先验概率nQuadratic discriminant function (QDF)nDecision surface2021/6/1648oParameter Estimation of Gaussian DensitynMaximum Likelihood (ML)Gaussian Classifiers2021/6/1649oParameter Estimation of Gaussian DensityGaussian Classifiers2021/6/1
18、650共享协方差距阵的情况Gaussian Classifiers2021/6/1651nParametric分类器不好用吗 - 实际中很多类别的概率分布近似Gaussian - 即使概率分布偏离Gaussian比较大，当特征维数高而训练样本少(Curse of dimensionality)时，Parametric分类器仍然比较好n有时LDF甚至比QDF更好nML估计的好处： - 训练计算量小（与类别数和样本数成线性关系） - 高维情况下降维（特征选择、变换）经常是有益的Gaussian Classifiers2021/6/1652oGaussian分类器的改进nQDF的问题o参数太多：与维
19、数的平方成正比o训练样本少时协方差矩阵奇异o即使不奇异ML估计的泛化性能也不好nRegularized discriminant analysis (RDA)o通过平滑协方差矩阵克服奇异，同时提高泛化性能Gaussian Classifiers2021/6/1653 We could design an optional classifier if we knew the prior probabilities and conditional densities. One approach is use the samples to estimate the unknown probabilities and densities, and then the resulting estimates as if they were the true values. Chapter 32021/6/1654 1 . 设设 为为来来正正态态分分布布 的的样样本本集集，试试求求参参数数 的的最最大大似似然然估估计计量量 。 作业四 2 . 设设 为为来来自自点点二二项项分分布布的的样样本本集集，即即 试求参数试求参数P的最大似然估计量的最大似然估计量 。 2021/6/1655 结束语结束语若有不当之处，请指正，谢谢！若有不当之处，请指正，谢谢！
