ANN not giving quite favorable results with the new dataset

The performance of the Artificial Neural Network (ANN) seems promising based on the training,validation and test data, but it's not giving quite favorable results with the new dataset. Any insights into why this might be happening?

[Walter Roberson]

Over-fitting.

[the cyclist]

As @Walter Roberson says, overfitting is certainly a possibility. It could also just be sampling error. It's not really possible to diagnose without more specifics.

At the very least, sharing how many features you have, and how many observations you have in your training, validation, test and new datasets would be needed to make a better guess.

[Sunita]

4 inputs, 2 outputs, 80 :20:20 , total 110 datasets,

[Sam Chak]

@Sunita, can you provide the datasets and show how you selected the values for the hyperparameters used in the ANN training process?

Answer 1

[the cyclist]

This could easily be sampling error. Even if your theoretical classification accuracy is 90%, your empirical classification accuracy (on just 20 observations?) can have a wide range.