Confused about Karhunen-Loeve Transform

Hi all, I've read lots of documents about KLT, but still get confused on how to correctly apply it to a data set. My references:

http://fourier.eng.hmc.edu/e161/lectures/klt/node3.html

http://reference.wolfram.com/mathematica/ref/KarhunenLoeveDecomposition.html

To sum up about the first link, the step to calculate the KLT of a matrix x = [x1(t); x2(t); x3(t), ... xn(t)] in which xn(t) is a time series

Step1: auto-correlation matrix--> R = x*x';

Step2: calculate transformation matrix Phi_H--> [V,D] = eig( R); Phi_H = V';

Step3: transformed matrix--> y = Phi_H * x

Then apply this algorithm to the example at link2, to transform a matrix x = [1 2 4; 2 3 10]; The transformation matrix is not right according to that done by Mathematica, neither does the final result.

Could anyone please revise my algorithm applying to transform a time-series matrix?

Thank you for any suggestions Kyle

[Daniel Shub]

I always confuse the differences between KLT, SVD, and PCA. Do you need to subtract the mean in KLT?

[Kyle]

They are really similar, wiki states that KLT is "also called" SVD. In some code SVD removes the mean, but I think KLT doesn't need this step.

Answer 1

[UJJWAL]

Ok.... Let me put it down one by one...

Karhunen Loeve Transform relies on the covariance matrix of a set of observation vectors. The basic Algorithm is clearly explained in the first link you have posted. I am below giving the code for the KLT for the same example as given in the MATHEMATICA Example which you have mentioned :-

clc
clear all
y=[1,2,4;2,3,10];
y=y' %Reasons for transposing will become clear when you will read the second point given below.
[V,D]=eig(cov(y))
KLT = V' * y';

Note the following points :-

a) For a given Eigenvalue there can be multiple EigenVectors which will be multiples of each other. Now if you run the program then please observe the following points :-

(i) Take a look at the transformation matrix returned by mathematica and the matrix V(which is the same as the transformation matrix returned by MATLAB). They are essentially the same except that one of the eigenvectors has been multiplied by -1 which is perfectly acceptable because even by multiplication by -1 it is still an eigenvector. It is this difference in the transformation matrices used by the two softwares that results into different values of the KVT. Also note that one of the eigenvectors in both the softwares is the same and so is one of the rows in the KVT. So Essentially the difference is there just because of the multiplication by -1 and there is nothing to worry about these results. If mathematics allows it then nothing can refute that.

(ii) Also note that in MATHEMATICA the observations are taken row-wise while in MATLAB they are taken Column-wise.It is because of that I have transposed y in the above code.

Please remember that each software calculates a quantity by a certain algorithm and minute differences may arise. The only important thing is mathematical consistency. Hope this clears your doubt.

Happy To help.

UJJWAL

[SGUNITN]

I was reading some paper where KLT is used for template protection.

Could you please share your view if it can be used for protecting a template like below.

[481.28 404.27 845.21 311.36 363.94 92.913 375.61 2647.8 2479.8 99.213 13.647 407.16 795.75 3121]

[Jaime De La Mota Sanchis]

As far as I understand, the KLT should be orthogonal, but as far as I can tell, they aren't. Am I wrong?