# Optimization of hidden layer dimension in neural network (number of neurons)

22 views (last 30 days)
David Franco on 14 Dec 2017
Edited: PIYUSHA GARG on 12 Jun 2020
I am using GA optimization to train my neural network (to try to find the best weights set):
% Neural Network trained by Genetic Algorithm (GA)
% Thx Greg!
rng('default')
[x, t] = simplefit_dataset;
[I, ~] = size(x);
[O, N] = size(t);
% Reference MSE: Average Target Variance
var_t = mean(var(t,1,2));
% Hidden Node Choice
H = 4;
Nw = (I + 1)*H + (H + 1)*O;
% Create Regression/Curve-Fitting Neural Network:
net = feedforwardnet(H);
net.divideFcn = 'dividetrain';
% Configure the Net for the Simplefit Dataset
net = configure(net, x, t);
% Initial Weights and Errors
wb = getwb(net)';
% Create handle to the error function,
fun = @(wb) nmse(wb, net, x, t); % NMSE
% Set the Genetic Algorithm tolerance for minimum change in fitness function
% before terminating algorithm to 1e-4 and display each iteration's results.
opts = optimoptions('ga', 'FunctionTolerance', 1e-4, 'Display', 'iter');
tic
[wbopt, fval] = ga(fun, Nw, opts);
totaltime = toc;
% Assign the weights to net
net = setwb(net, wbopt');
% Simulate the output
y = sim(net,x);
Where the function nmse is:
function nmse_calc = nmse(wb, net, input, target)
% 'wb' contains the weights and biases vector
% in row vector form as passed to it by the
% genetic algorithm. This must be transposed
% when being set as the weights and biases
% vector for the network.
% Reference MSE
var_t = mean(var(target,1,2));
% To set the weights and biases vector to the
% one given as input
net = setwb(net, wb');
% To evaluate the ouputs based on the given
% weights and biases vector
y = net(input);
% Calculating the Normalised Mean Squared Error (NMSE)
nmse_calc = mean((target(:) - y(:)).^2) / var_t;
end
My question is:
Is there any way to optimize the number of hidden nodes H in the same way? Maybe with multiobjective GA optimization... optimizing the weights set and the number of hidden nodes at once.
Thank you very much!
PIYUSHA GARG on 12 Jun 2020
Edited: PIYUSHA GARG on 12 Jun 2020
please tell how to write Nw for multiple hidden layers.

Greg Heath on 20 Dec 2017
Edited: Greg Heath on 20 Dec 2017
If you search the NEWSGROUP and ANSWERS using
greg genetic
you should conclude that I don't recommend GA for NN design.
Nevertheless, if you insist, I recommend a double loop design:
j =0
for h = Hmin:dH:Hmax
j=j+1
standard stuff
for i = 1:Ntrials
GA
end
end
Hope this helps,
Thank you for formally accepting my answer
Greg
David Franco on 28 Dec 2017
Edited: David Franco on 28 Dec 2017
But in this way, there is no optimization of the number of Hidden nodes through a global optimization technique, just an exhaustive check of all Hidden nodes possibilities.
The number o Hidden nodes must be optimized too (as an input for GA, for instance)...