《Linear Discrimination--线性判别式.ppt》由会员分享，可在线阅读，更多相关《Linear Discrimination--线性判别式.ppt（12页珍藏版）》请在金锄头文库上搜索。
1、LinearDiscrimination outline 10 7LogisticDiscrimination10 7 1TwoClasses10 7 2MultipleClasses10 8DiscriminationbyRegression GradientDescent TwoClassesWedonotmodeltheclass conditionaldensities p x Ci butrathertheirratio Twoclassesandassumethattheloglikelihoodratioislinear UsingBayes ruleandinference w
2、ehavethesigmoidfunction whereHowwecanlearnand 10 7Logisticdiscrimination Howwecanlearnand Givenasampleoftwoclasses X where 1 ifx and 0ifx Assume given isBernoulli Thesamplelikelihoodis Likelihoodfunctiontomaximize minimizedasItsderivativeisgivenas 10 7 1TwoClasses ItisbesttoinitializeWjwithrandomval
3、uescloseto0 generallytheyaredrawnuniformlyfromtheinterval 0 01 0 01 10 7 1TwoClasses 计算y值 LetusnowgeneralizetoK 2classes Wetakeoneoftheclasses forexample asthereferenceclassandassumethat Inference Thesoftmaxfunction Totreatallclassesuniformly wecanwriteguaranteesthat 10 7 2MultipleClasses Howwecanle
4、arnand InthiscaseofK 2classes eachsamplepointisamultinomialtrialwithonedraw thatis where Thesamplelikelihoodis Theerrorfunctionisagaincross entropy Define whichis1ifi jand0ifij 10 7 2MultipleClasses Duringtesting wecalculateall k 1 KandchooseifAgainwedonotneedtocontinuetrainingtominimizecrossentropy
5、asmuchaspossible wetrainonlyuntilthecorrectclasshasthehighestweightedsum andthereforewecanstoptrainingearlierbycheckingthenumberofmisclassifications Whendataarenormallydistributed thelogisticdiscriminanthascomparableerrorratetotheparametric normal basedlineardiscriminant McLachlan1992 Logisticdiscri
6、minationcanstillbeusedwhentheclass conditionaldensitiesarenonnormalorwhentheyarenotunimodal aslongasclassesarelinearlyseparable 10 7 2MultipleClasses Inregression theprobabilisticmodelis TwoClassesAssumingalinearmodelandtwoclasses wehaveThenthesamplelikelihoodinregression assumingMaximizingtheloglikelihoodisminimizingthesumofsquareerrors 10 8DiscriminationbyRegression Usinggradient descent wegetMultipleClasses 10 8DiscriminationbyRegression
