SVM Separating hyperplane and its margin

I am solving a classification problem using SVM specifically fitcsvm. I want to know how do I get the equation of the (a) separating hyperplane and (b) and equations of the margin.

My goal is to know whether a new sample (data_new, not added to the training set) is within the margin of separating hyperplane. BTW, im using gaussian kernel.

clc,clear; format compact; format shortG;
seed  = 1;
rng(seed);

r = sqrt(rand(100,1)); % Radius
t = 2*pi*rand(100,1);  % Angle
data1 = [r.*cos(t), r.*sin(t)]; % Points

r2 = sqrt(3*rand(100,1)+1); % Radius
t2 = 2*pi*rand(100,1);      % Angle
data2 = [r2.*cos(t2), r2.*sin(t2)]; % points

figure;
plot(data1(:,1),data1(:,2),'r.','MarkerSize',15); hold on
plot(data2(:,1),data2(:,2),'b.','MarkerSize',15)
ezpolar(@(x)1);ezpolar(@(x)2);
ylim([-2.5,2.5]); xlim([-2.5,2.5]); hold off
axis equal

data3 = [data1;data2];
theclass = ones(200,1);
theclass(1:100) = -1;

%Train the SVM Classifier
cl = fitcsvm(data3,theclass,'KernelFunction','rbf',...
    'BoxConstraint',Inf,'ClassNames',[-1,1]);

% Predict scores over the grid
d = 0.02;
[x1Grid,x2Grid] = meshgrid(min(data3(:,1)):d:max(data3(:,1)),...
    min(data3(:,2)):d:max(data3(:,2)));
xGrid = [x1Grid(:),x2Grid(:)];
[~,scores] = predict(cl,xGrid);


r_new = sqrt(1.0*rand(20,1)+0.5); % Radius
t_new = 2*pi*rand(20,1);      % Angle
data_new = [r_new.*cos(t_new), r_new.*sin(t_new)]; % points

% Plot the data and the decision boundary
figure;
h(1:2) = gscatter(data3(:,1),data3(:,2),theclass,'rb','.'); hold on
ezpolar(@(x)1); ezpolar(@(x)2);
h(3) = plot(data3(cl.IsSupportVector,1),data3(cl.IsSupportVector,2),'ko'); hold on
h(4) = gscatter(data_new(:,1),data_new(:,2),ones(20,1),'g','.');

contour(x1Grid,x2Grid,reshape(scores(:,2),size(x1Grid)),[0 0],'k');
legend(h,{'-1','+1','Support Vectors','New samples'}, Location="northeast");
axis equal
ylim([-2.5,2.5]); xlim([-2.5,2.5]); 
hold off

I am not exactly sure if this 'margin' function is proper to use.

Can anybody help or suggest how to do this?

Answer 1

[Githin George]

Hello,

I understand you are trying to find the equations of the margins and separating hyperplanes for the SVM classifier, and you are using a ‘gaussian’ kernel. I also understand that the final objective here is to check whether a new sample point lies within the margin of separating hyperplane or not.

It is in fact a difficult task to obtain the equations of the separating hyperplane and margins, because the output of “fitcsvm” function will not have the beta values for a non-linear kernel like ‘gaussian’.

Alternatively, you could use “margin” function to compute the classification margins for each new sample and compare it to the max-margin of support vectors to filter out samples lying within the margin as shown in the code below:

labels = predict(cl,data_new);
m = margin(cl,data_new,labels);
maxMargin = max(margin(cl,cl.SupportVectors,cl.SupportVectorLabels));

indices = find(m < maxmargin);
figure;
h(1:2) = gscatter(data3(:,1),data3(:,2),theclass,'rb','.'); hold on
ezpolar(@(x)1); ezpolar(@(x)2);
h(3) = plot(data3(cl.IsSupportVector,1),data3(cl.IsSupportVector,2),'ko'); hold on

% PLOTTING NEW SAMPLES WITHIN THE 'MARGIN'
gscatter(data_new(indices,1),data_new(indices,2),labels(indices),'cm','.',20);

contour(x1Grid,x2Grid,reshape(scores(:,2),size(x1Grid)),[0 0],'k');
legend off;
axis equal
ylim([-2.5,2.5]); xlim([-2.5,2.5]); 
hold off;

In the image below, I’ve plotted samples lying inside and outside the margins in separate figures.

1. The green dots in Figure 21 represents new samples.
2. In Figure 23, only new samples lying outside the margin is plotted (samples in cyan and magenta)
3. In Figure 22, only new samples lying inside the margin is plotted.

You can clearly observe that “margin” function is proving helpful in this task.

I hope this helps.