《精品阳明大学放射医学科学研究所》由会员分享，可在线阅读，更多相关《精品阳明大学放射医学科学研究所（40页珍藏版）》请在金锄头文库上搜索。
1、An Introduction to Independent Component Analysis (ICA),吳育德 陽明大學放射醫學科學研究所 台北榮總整合性腦功能實驗室,The Principle of ICA: a cocktail-party problem,x1(t)=a11 s1(t) +a12 s2(t) +a13 s3(t) x2(t)=a21 s1(t) +a22 s2(t) +a12 s3(t) x3(t)=a31 s1(t) +a32 s2(t) +a33 s3(t),Independent Component Analysis,Reference : A. Hyvri
2、nen, J. Karhunen, E. Oja (2001) John Wiley & Sons. Independent Component Analysis,Central limit theorem,The distribution of a sum of independent random variables tends toward a Gaussian distribution,Observed signal,=,IC1,IC2,ICn,m1,+ m2,.+ mn,toward Gaussian,Non-Gaussian,Non-Gaussian,Non-Gaussian,Ce
3、ntral Limit Theorem,Partial sum of a sequence zi of independent and identically distributed random variables zi,Partial sum of a sequence zi of independent and identically distributed random variables zi,How to estimate ICA model,Principle for estimating the model of ICA,Maximization of NonGaussiani
4、ty,Measures for NonGaussianity,Kurtosis,Super-Gaussian kurtosis 0,Gaussian kurtosis = 0,Sub-Gaussian kurtosis 0,Kurtosis : E(x- )4-3*E(x-)2 2,kurt(x1+x2)= kurt(x1) + kurt(x2) kurt(x1) =4kurt(x1),Assume measurement,Whitening process,is zero mean and,Then is a whitening matrix,I,ss,=,T,E,s,x,A,=,T,E,D
5、,V,2,1,-,=,T,T,T,E,E,V,xx,V,zz,=,2,1,2,1,-,-,=,ED,EDE,E,D,T,T,I,=,T,T,E,EDE,xx,=,Let D and E be the eigenvalues and eigenvector matrix of covariance matrix of x, i.e.,Importance of whitening,For the whitened data z, find a vector w such that the linear combination y=wTz has maximum nongaussianity un
6、der the constrain,Maximize | kurt(wTz)| under the simpler constraint that |w|=1,Then,Constrained Optimization,max F(w), |w|2=1,At the stable point, the gradient of F(w) must point in the direction of w, i.e. equal to w multiplied by a scalar.,0,),(,=,w,w,l,L,0,2,),(,=,+,w,w,w,l,F,1,),1,(,),(,),(,2,2
7、,=,=,-,+,=,w,w,w,w,w,w,T,F,L,l,l,w,w,w,-,=,l,2,),(,F,Gradient of kurtosis,),(,3,),(,),(,),(,2,2,4,z,w,z,w,z,w,w,T,T,T,E,E,kurt,F,-,=,=,w,w,w,z,w,w,z,w,z,w,w,w,-,=,-,=,=,2,1,4,2,4,),(,3,),(,(,1,),(,3,),(,),(,T,T,t,T,T,T,t,T,E,E,F,),)(,(,2,*,3,),(,),(,4,1,3,w,w,w,w,z,w,z,+,-,=,=,T,T,t,T,t,t,T,3,),(,(,
8、4,2,3,w,w,z,w,z,z,w,-,=,T,T,E,kurt,sign,=,=,T,t,t,T,E,1,),(,1,y,y,Q,Fixed-point algorithm using kurtosis,wk+1 = wk + ,Note that adding the gradient to wk does not change its direction, since,Convergence : |=1 since wk and wk+1 are unit vectors,),(,wk,F,wk,= ( + ,-1,l,2,),),(,wk,F,wk,2,3,3,w,w,z,w,z,
9、Therefore, w,-,T,E,w,w,w,/,wk+1 = wk - (,wk ),l,2,= (1- 2,l,),wk,Fixed-point algorithm using kurtosis,Centering Whitening Choose m, No. of ICs to estimate. Set counter p 1 Choose an initial guess of unit norm for wp, eg. randomly. Let Do deflation decorrelation Let wp wp/|wp| If wp has not converged
10、 (| 1), go to step 5. Set p p+1. If p m, go back to step 4.,One-by-one Estimation,Fixed-point iteration,Fixed-point algorithm using negentropy,The kurtosis is very sensitive to outliers, which may be erroneous or irrelevant observations,Need to find a more robust measure for nongaussianity,Approxima
11、tion of negentropy,ex. r.v. with sample size=1000, mean=0, variance=1, contains one value = 10 kurtosis at least equal to 104/1000-3=7,kurtosis : Ex4-3,Fixed-point algorithm using negentropy,Entropy,Entropy,Negentropy,Approximation of negentropy,w Ezg(wTz) Eg (wTz) w w w/|w|,Fixed-point algorithm us
12、ing negentropy,Convergence : |=1,Max J(y),Fixed-point algorithm using negentropy,Centering Whitening Choose m, No. of ICs to estimate. Set counter p 1 Choose an initial guess of unit norm for wp, eg. randomly. Let Do deflation decorrelation Let wp wp/|wp| If wp has not converged, go back to step 5.
13、Set p p+1. If p m, go back to step 4.,One-by-one Estimation,Fixed-point iteration,Implantations,Create two uniform sources,Implantations,Create two uniform sources,Implantations,Two mixed observed signals,Implantations,Two mixed observed signals,Implantations,Centering,Implantations,Centering,Implan
14、tations,Whitening,Implantations,Whitening,Implantations,Fixed-point iteration using kurtosis,Implantations,Fixed-point iteration using kurtosis,Implantations,Fixed-point iteration using kurtosis,Implantations,Fixed-point iteration using negentropy,Implantations,Fixed-point iteration using negentropy
15、,Implantations,Fixed-point iteration using negentropy,Implantations,Fixed-point iteration using negentropy,Implantations,Fixed-point iteration using negentropy,Fixed-point algorithm using negentropy,Entropy,Fixed-point algorithm using negentropy,High-order cumulant approximation,Its quite common that most r.v. have approximately symmetric dist.,Its quite common that most r.v. have approximately symmetric dist.,Fixed-point algorithm using negentropy,According to Lagrange multiplier the gradient must poi
