Avoid training certain neurons

Using the Deep Learning Toolbox, I wish to construct a simple feed-forward network for a simulation, however assume I have already trained one of the hidden neurons (out of several) with the correct weights and biases and I don't want them to change during training. How can I make this single specific neuron be "constant" and not get retrained with new wights and biases while the rest of the network is being trained?

Answer 1

[Hiro Yoshino]

There is an option to keep specific layers' learning rates low so you can fix them as they are.

for example

fullyConnectedLayer(<outputsize>, 'WeightLearnRateFactor', 0, 'BiasLearnRateFactor', 0)

This way, you would multiply zero to the global learning rate, which is set via trainingOptions function, and thus the learning rates of the weights in the fully-connected-layer are set as zero.

Check this out - this may help you out too.

[Hamid Moazed]

Thanks for the quick and thorough response; yes, that makes total sense, why didn't I think of that! Thans again for getting me unstuck.