How to define a custom transferFcn for recurrent neural network ?

Dear all,

I'm currently using Recurrent Neural Networks (layrecnet), and I am trying to use a custom transfer function, in order to try a "Leaky ReLU" approach. I read some topics discussing way to add custom transfer functions (or performance functions) :

https://fr.mathworks.com/matlabcentral/answers/118968-how-to-create-custom-transfer-function-for-neural-network-in-matlab

https://fr.mathworks.com/matlabcentral/answers/56137-how-to-use-a-custom-transfer-function-in-neural-net-training

And I did this :

1. Went into my folder .../MATLAB/R2018b/toolbox/nnet/nnet/nntransfer/
2. I copied poslin.m (as reference) and pasted it as "lkposlin.m"
3. In "lkposlin.m", I replaced both "poslin" with "lkposlin"
4. I copied the folder "+poslin" and pasted it as "+lkposlin"
5. All the permissions are the same as default files

I did not make any other changes, just to try it out. And then, when I called in my network using "net.layers{1}.transferFcn='lkposlin'", my code sends me error an error saying "lkposlin" is an unknown function or variable. What am I missing ?

Answer 1

[Vijaya Lakshmi Chagi]

Hi Nicolas,

The function files and packages to use for defining custom versions of other Neural Network components can be found by executing the following in MATLAB:

doc nncustom

For custom transfer functions, would recommend you to use +tansig package as reference.

To summarize the workflow for creating the custom transfer fucntion,

1. Navigate to the folder containing the NN transfer functions (MATLAB\R2017a\toolbox\nnet\nnet\nntransfer in Windows)
2. Copy the file "tansig.m" and the directory "+tansig".
3. Paste the copied file and directory into your working directory
4. Modify the name of "tansig.m" and "+tansig" to reflect the new transfer function name, for example "myFcn.m" and "+myFcn".
5. Modify the code in the relevant files of "+myFcn"to properly calculate the new transfer function and its derivative.
6. For the layer in the neural network that will use this transfer function, change the "transferFcn" property to 'myFcn'.

Hope the information helps you.

[Nicolas M.]

Thank you for your answer.

I tried again using the tansig as refrence as you suggested, and did the following steps:

1. Went to the toolbox/nnet/nnet/nntransfer/ folder
2. Copied tansig.m, renamed it leakyrelu.m, and changed all instances of "tansig" into "leakyrelu" in the file.
3. Copies +tansig, renamed it +leakyrelu. I changed nothing inside, as I just wanted to try if the function was recognized.
4. Restarted my computer.

And when I try " net.layers{1}.transferFcn='leakyrelu' ", I get the same error saying the function does not exist... It's like matlab is completly ignoring my custom functions.

Permissions of the files are the same as the others:

drwxr-xr-x 2 root root 4096 janv. 14 18:22 +leakyrelu
-r--r--r-- 1 root root 1465 mars  18 07:25 leakyrelu.m
drwxr-xr-x 2 root root 4096 janv. 14 18:22 +logsig
-r--r--r-- 1 root root 1562 avril  4  2018 logsig.m

And here's the content of leakyrelu.m :

function a = leakyrelu(n,varargin)

if nargin > 0
    n = convertStringsToChars(n);
end

if ischar(n)
  a = nnet7.transfer_fcn(mfilename,n,varargin{:});
  return
end

% Apply
a = leakyrelu.apply(n);

I don't understand were my problem is coming from. Am I missing something ?

Have you encountered something similar before ?

[Illia]

For those who's googling: new files should be either copied to a working directory (where your script is placed), or the directory should be placed to path (see path function). nntransfer folder is not initially on path (at least in my case).

Answer 2

[a1a1 b1b1]

hello dear nicolas

i had your problem too . i find solution of that and send it here .

to solve this problem you should copy and paste your mfile (myfunction.m) and the respective folder (+myfunction) to current directory (current folder) of your matlab codes;

i hope this become helpful.

thanks