Implementation support vector machine nonlinear case with quadprog function in matlab.

Hello

I just try implement nonlinear support vector machine in matlab on my own (without using svmtrain) and I get some serious problems. I find such tutorial http://www.robots.ox.ac.uk/~az/lectures/ml/matlab2.pdf which works fine for linear case Unfortunately I don't know how to modify it to nonlinear case. This is the modified code from linear to unlinear kernel. In places where is "%%%%%" I change linear kernel to (what I think should be) nonlinear. Of course it is not working properly. Where is the error ?

Best regards. Jan.

function Matlab2()
% Genereting two sets of points with normal distribution
n = 25;
data1 = normrnd(0,1,[n 2]);
data2 = normrnd(5,1,[n 2]);
xdata = [data1;data2];
n = size(xdata,1);
names = [];
for i=1:(size(xdata)/2)
    names(i) = 1;
end
for i=(size(xdata)/2)+1:size(xdata)
    names(i) = -1;
end
names = names.';
TRAINDATA= [xdata names];

% Find the number of examples and attributes used in the data
numOfExamples = size(TRAINDATA,1)
numOfAttributes = size(TRAINDATA,2)-1
% Extract the attribute matrix X and label vector Y
X = TRAINDATA(:,1:numOfAttributes)
Y = TRAINDATA(:,numOfAttributes+1)
% Split the data into no and yes play days
X_YES_DAYS = X(find(Y==1),:)
X_NO_DAYS = X(find(Y==-1),:)
hold on
% Plot the yes days
plot(X_YES_DAYS(:,1),X_YES_DAYS(:,2),'or')
% Plot the no days
plot(X_NO_DAYS(:,1),X_NO_DAYS(:,2),'+b')
%%%Code to find the SVM hyperplane will go here! %%%%
H=eye(numOfAttributes+1)
H(numOfAttributes+1,numOfAttributes+1)=0
f=zeros(numOfAttributes+1,1)
Z = [X ones(numOfExamples,1)]

%%%%%%%%%%In linear case there should be 
%%%%%%%%%%A=-diag(Y)*Z 
%%%%%%%%%%but because I use nonlinear kernel then I change A to

dotproduct = (-diag(Y)*Z);
A = dotproduct.*(1 + dotproduct); 

c=-1*ones(numOfExamples,1)
w=quadprog(H,f,A,c)
%%%Code to plot the SVM separating hyperplane will go here! %%%%
X1= xdata;
w1=w(1,1);
w2=w(2,1);
b=w(3,1);
%%%%%%%%%%In linear case there should be 
%%%%%%%%%%Y1=-(w1*X1+b)/w2;
%%%%%%%%%%but because I use nonlinear kernel then I change A to
dotproduct = (w1*X1);
A = dotproduct.*(1 + dotproduct); 

Y1=-(A+b)/w2; %Seperating hyperplane

plot(X1,Y1,'k-')
end

[Jonathan]

See short answer on this thread: https://se.mathworks.com/matlabcentral/newsreader/view_thread/296517

Answer 1

[Matt J]

Check your constraints. There are no points w satisfying

 A*w<=c

[Matt J]

Were you talking to me?

Answer 2

[poprostuJanek]

In SVM in linear case there is a coinstrains that for every y there should be

y(w*x-b)>=1

in order to use "quadprog" function I multiply by -1 and get

-y(w*x-b)<=-1

in code this is the part

A=-diag(Y)*Z
c=-1*ones(numOfExamples,1)
w=quadprog(H,f,A,c)

but because I would like to use nonlinear kernel then I should get

-y(kernel(w*x)-b)<=-1

am I right ? because this was my idea behind the code

dotproduct = (-diag(Y)*Z);
A = dotproduct.*(1 + dotproduct); 
c=-1*ones(numOfExamples,1)
w=quadprog(H,f,A,c)

:)

[deepti khanduja]

I am trying implement a simple linear SVM using linear Algebra without quadprog that could be verified mathematically. Can you please guide me understand the working of quadprog or suggest a way to implement linear SVM.