Optimization of hidden layer dimension in neural network (number of neurons)

6 visualizzazioni (ultimi 30 giorni)
I am using GA optimization to train my neural network (to try to find the best weights set):
% Neural Network trained by Genetic Algorithm (GA)
% Reference: http://www.mathworks.com/matlabcentral/answers/100323
% Thx Greg!
rng('default')
[x, t] = simplefit_dataset;
[I, ~] = size(x);
[O, N] = size(t);
% Reference MSE: Average Target Variance
var_t = mean(var(t,1,2));
% Hidden Node Choice
H = 4;
Nw = (I + 1)*H + (H + 1)*O;
% Create Regression/Curve-Fitting Neural Network:
net = feedforwardnet(H);
net.divideFcn = 'dividetrain';
% Configure the Net for the Simplefit Dataset
net = configure(net, x, t);
% Initial Weights and Errors
wb = getwb(net)';
% Create handle to the error function,
fun = @(wb) nmse(wb, net, x, t); % NMSE
% Set the Genetic Algorithm tolerance for minimum change in fitness function
% before terminating algorithm to 1e-4 and display each iteration's results.
opts = optimoptions('ga', 'FunctionTolerance', 1e-4, 'Display', 'iter');
tic
[wbopt, fval] = ga(fun, Nw, opts);
totaltime = toc;
% Assign the weights to net
net = setwb(net, wbopt');
% Simulate the output
y = sim(net,x);
Where the function nmse is:
function nmse_calc = nmse(wb, net, input, target)
% 'wb' contains the weights and biases vector
% in row vector form as passed to it by the
% genetic algorithm. This must be transposed
% when being set as the weights and biases
% vector for the network.
% Reference MSE
var_t = mean(var(target,1,2));
% To set the weights and biases vector to the
% one given as input
net = setwb(net, wb');
% To evaluate the ouputs based on the given
% weights and biases vector
y = net(input);
% Calculating the Normalised Mean Squared Error (NMSE)
nmse_calc = mean((target(:) - y(:)).^2) / var_t;
end
My question is:
Is there any way to optimize the number of hidden nodes H in the same way? Maybe with multiobjective GA optimization... optimizing the weights set and the number of hidden nodes at once.
Thank you very much!

Risposta accettata

Greg Heath
Greg Heath il 20 Dic 2017
Modificato: Greg Heath il 20 Dic 2017
If you search the NEWSGROUP and ANSWERS using
greg genetic
you should conclude that I don't recommend GA for NN design.
Nevertheless, if you insist, I recommend a double loop design:
j =0
for h = Hmin:dH:Hmax
j=j+1
standard stuff
for i = 1:Ntrials
GA
end
end
Hope this helps,
Thank you for formally accepting my answer
Greg
  1 Commento
David Franco
David Franco il 28 Dic 2017
Modificato: David Franco il 28 Dic 2017
But in this way, there is no optimization of the number of Hidden nodes through a global optimization technique, just an exhaustive check of all Hidden nodes possibilities.
The number o Hidden nodes must be optimized too (as an input for GA, for instance)...

Accedi per commentare.

Più risposte (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by