Fitceoc for selection of feedforward Neural network parameters

How to select best combination of hyperpapermeters using fitcecoc function. The optimizabale parameters are given below:

optimizableVariables

epochs = [5, 10]

learning rate= [0.1, 1]

optimizers = ['rmsprop', 'adam']

activation1 = ['relu','sigmoid']

I have seen that fitcecoc function is used for SVM and other classifiers but how to tune the parameters of feedforward neural network paparameters.

clc
clear all

load fisheriris
t = templateSVM();
results = fitcecoc(meas, species,'Learners',t,'OptimizeHyperparameters','auto',...
    'HyperparameterOptimizationOptions',struct('Optimizer',...
    'randomsearch'))

Answer 1

[Shivam Singh]

Yes, there is an automatic hyperparameter optimization in fitcecoc function, which is used in various algorithm like SVM etc.

For the neural networks like feedforward neutral network, there is an effective hyperparameter optimization technique called Bayesian optimization.

Below links are examples which apply Bayesian optimization for finding optimal network hyperparameters and training options for convolutional neural networks. Similar to this, optimal hyperparameters for feedforward neural network can also be implemented using this approach.

• Deep Learning using Bayesian Optimization
• Tune Experiment Hyperparameters by Using Bayesian Optimization