Supplied objective function must return a scalar value

I am trying to code a ML algorithm in Matlab. These are my different functions:

sigmoid.m:

    function g = sigmoid(z)
    g = zeros(size(z));
    g = 1 ./ (1+exp(z));

costFunction.m

   function [J, grad ] = costFunction(theta, X, y)
            
        
    m = length(y); % number of training examples
    z = -X * theta;
    g = sigmoid(z);
    J = 1/m * ((-y * log(g)') - ((1 - y) * log(1 - g)'));
        
    grad = zeros(size(theta'));
    grad = (1/m) * (X' * (g - y));
    
   end

ex2.m (This is the main file of my project and I put the relative lines I get this error message)

    options = optimset('GradObj', 'on', 'MaxIter', 400);
    
    [theta, cost] = ...
    	fminunc(@(t)(costFunction(t, X, y)), initial_theta, options);

The error message:

> Error using fminunc (line 348) Supplied objective function must return Supplied objective function must return
> a scalar value.
> 
> Error in ex2 (line 97) 	fminunc(@(t)(costFunction(t, X, y)),
> initial_theta, options);

I don't know is there enough information above or not? If not, let me know to add extra information.

[Walter Roberson]

what are the sizes of the inputs?

[Jay Guwalani]

% Try using this in cost function || I guess the issue is with the (.)
% elementwise ops
H=sigmoid(X*theta);
J=(sum(-y.*log(H)-(1-y).*log(1-H))/m);
grad=(1/m)*(X'*(H-y));      

Answer 1

[reza]

I changed the following line of code:

J = 1/m * ((-y * log(g)') - ((1 - y) * log(1 - g)'));

To the following line of code:

J = 1/m * (((-y)' * log(g)) - ((1 - y)' * log(1 - g)));

And problem solved!

The y and g were 100*1 matrices and with previous code I had J=100*100 matrix, but with new code I have J=1*1 matrix or scalar number and problem solved!

[Rik]

Since it is the conclusion of my answer that you needed to change something to achieve a scalar output, I would say this derives from my answer, but anyway, happy to help.

This also shows the value of comments: I had no way of knowing what your code does, so I could only point you in a general direction. If you want your code understandable to others, make sure to write comments and function headers. Some people go as far as saying you should have a line of comment for every line of code, but I would say that is being excessive.

Answer 2

[Rik]

Depending on your input variable sizes, your code is not guaranteed to return the same size output as the input size. The documentation of fminunc states the following:

"fun is a function that accepts a vector or array x and returns a real scalar f, the objective function evaluated at x."

The initial_theta determines the size of the theta that the solver will use.

Also, I can't find a sigmoid function in the Matlab doc. Do you mean this FEX entry?

%to more easily explain what's going on, assign some names to dimensions:
[a,b]=size(X);[c,d]=size(y);
[e,f]=size(initial_theta);

And now to your function:

function [J, grad ] = costFunction(theta, X, y)
m = length(y); % number of training examples
%m is either c or d
z = -X * theta;
%z is [a,f], unless theta is a scalar, then z is [a,b]
g = sigmoid(z);
%same as z (so [a,b] or [a,f])
J = 1/m * ((-y * log(g)') - ((1 - y) * log(1 - g)'));
%1/m is a scalar
%(-y * log(g)') is [c,d]*[f,a] or [c,d]*[b,a], so [c,a], but will error if d and b (or d and f) are not equal
%((1 - y) * log(1 - g)') is the same size as the other term
%so J is [c,a], unless this line returns an error

grad = zeros(size(theta'));
%this line doesn't do anything, as grad gets overwritten
grad = (1/m) * (X' * (g - y));
%1/m is scalar
%(X' * (g - y)) is [b,a]*[c,d] so [b,d] and asserts that a==c and that [a,b] (or [a,f]) is the same as [c,d]
end

So this function places quite a lot of implicit restrictions on your inputs.

To successfully evaluate, the following must all be true:

[a,b]=size(X);[c,d]=size(y);
[e,f]=size(initial_theta);
theta_is_scalar=numel(theta)==1;

%J restrictions
assert(a==1)
assert(c==1)
if theta_is_scalar
    assert(d==f)%so assert(d==1)
else
    assert(b==e)
    assert(b==d)
end
%grad restrictions
assert(b==1)
assert(c==1)
assert(a==c)
if theta_is_scalar
    %[a,b] is [c,d]
    assert(b==d)
else
    %[a,f] is [c,d]
    assert(f==d)
end

Concluding this:

%scalar theta:
X must be [1,1]
y must be [1,1]
%non-scalar theta:
X must be [1,1]
y must be [1,1]
initial_theta must be [1,1]

So all the inputs must be scalar.

[reza]

I don't know how to say thank you. I could solve my problem with only one simple change as I wrote in my answer, but again I say thank you very much. I learned a lot from your answer.

[Walter Roberson]

(User supplied the source for sigmoid in the Question.)

[Sanyam Maheshwari]

Hi Rik I am also getting same error when I am writing my code.

This is the Data file

global SL V p r k m n C Z s 
SL = 0.75;
V = 94100;
p = [0.07,0.18,0.2,0.3];
r = [55,47,45,49];
k = [33,28,29,30];
m = 4;
n= 4;
C = [78,69,70,73;
    64,68,56,59;
    34,39,42,41;
    52,47,48,45];
Z =[250	250	250	250;
    320	320	320	320;
    440	440	440	440;
    350	350	350	350];
s = [110,95,99,100;
    110,95,99,100;
    110,95,99,100;
    110,95,99,100];

bguess = randi(4,4);
bguess = (bguess<2);
Q0 = [zeros(4,4) bguess];
% call the solver
xopt = fmincon(@EP, Q0,[],[],[],[],[],[]);

This is the obective function file.

function G =  EP(Q,b)
global s r k C m n 
rng;
y = rand(4);
b = randi(4,4);
b = (b<2);
q = 0;
for i = 1:m
    for j = 1:n
        q = q + y(i,j)*Q(i,j);
    end
end

x = sym('x',[1,4]); %x as symbolic variable
f1 = 0;
for i = 1:m
    for j = i:n 
        f1  = f1 + s(i,j)*b(i,j)*x(j);
    end
end       
f2 = 0;
for i = 1:m
    for j = 1:n
        f2 = f2+C(i,j)*b(i,j)*y(i,j)*Q(i,j);
    end
end
f3 = 0;
        f3a = 0;
        for i = 1:m
            for j = 1:n
                f3a = f3a + y(i,j)*Q(i,j)-x(j);
            end
        end
for i  = 1:m
    for j = 1:n
        f3 = f3 + b(i,j)*r(j)*f3a;
    end
end
f4 = 0;
for i = 1:m
    for j = 1:n
        f4 = f4+ s(j)*b(i,j)*y(i,j)*Q(i,j);
    end
end
f5 = 0;
        f5a = 0;
        for i = 1:m
            for j = 1:n
                f5a = f5a + x(j) -y(i,j)*Q(i,j);
            end
        end
    
for i  = 1:m
    for j = 1:n
        f5 = f5 + b(i,j)*k(j)*f5a;
    end
end

F1 = int(((f1 - f2 + f3).*normpdf(x, 400, 100)),0,q);
F2 = int(((f4 - f2 - f5).*normpdf(x, 400, 100)),q,Inf);
G = -(F1 +F2);
end

I am getting this error. I don't how to solve it please help.

[Rik]

Why are you using globals?

And if the error is the same, it is likely that the solution is also the same: make sure your objective function returns a scalar.

[Walter Roberson]

The user posted this elsewhere, and I responded there. The syms x [1 4] is a problem and it cannot be fixed without redefining the calculation.