Why testing accuracy is 0.000?

I am attaching the matlab code and input file. This code is not giving any error and running very smoothly. However, It is giving testing accuracy 0.000

Out-of-Bag Error: 0.0412

Out-of-Bag Classification Error: 0.6729

Training Accuracy: 0.9588

Testing Accuracy: 0.0000

is it okey or some problem in computing testing error?

please suggest me.

Sanchit

Answer 1

[Aniketh]

Hi Sanchit,

It seems the problem in your code is in the respective datatypes of y_test.Var11 and y_pred.

To address this problem, let's examine the code and make the necessary corrections. In the original code, the line test_accuracy = sum(strcmp(y_pred, y_test.Var11)) / numel(y_test.Var11); is used to calculate the testing accuracy. However, based on the given information, it appears that y_test.Var11 contains numeric values, while y_pred contains cell arrays of strings.

y_test_str = cellfun(@num2str, num2cell(y_test.Var11), 'UniformOutput', false);
test_accuracy = sum(strcmp(y_pred, y_test_str)) / numel(y_test_str);
fprintf('Testing Accuracy: %.4f\n', test_accuracy);

With this change testing accuracy is being printed correctly, you could employ other methods as well to change the datatypes.

Hope this helped!

[Sanchit]

Thank you very much. It has worked very well. I further request you to kindly provide me an equvalent function of following

oob_mse = mean_squared_error(y_train, predict(rf_classifier, X_train));

Sanchit

[Sanchit]

I am using below matlab code for mean square error. It is giving mse 0.000 for three different data sets. Therefore, there is some error in code. I request you to kindly have a look on it and suggest me how to fix it.

Sanchit

% Create a Random Forest classifier

rf_classifier = TreeBagger(100, X_train, y_train, 'OOBPrediction', 'On');

predicted = predict(rf_classifier, X_train);

YY = categorical(predicted)

ZZ = str2num(char(YY(:)))

Z = table2array(y_train)

oob_mse = immse(Z,ZZ);

disp(sprintf('Out-of-Bag Mean Square Error: %.4f', oob_mse(end)));

[Aniketh]

You're welcome! I'm glad to hear that the solution worked for you.

An equivalent of mse can be done in MATLAB as:

y_train_pred = predict(rf_classifier, X_train);
oob_mse = mean((y_train - y_train_pred).^2);

For the sceond part can't say for sure without looking at the complete code but if you are evaluating on the same dataset that you used for training, it is possible the classifier has learned to predict the training data very well and you are getting a loss as 0.000