View weights used in a neural network created after using the GUI

I new to the Neural Network toolbox and am creating a network using the fitting tool with up to six inputs and one output. I have created the network using the fitting tool in the GUI and want to see which inputs are the most inportant in the training of the network. Is there a way to view the individual weights applied to each input in the network? Thanks for any help

Answer 1

[Greg Heath]

If you want to rank input importance, looking at the input weights is not sufficient.

1. Standardize input and target variables (zero-mean/unit-variance)

2. Train the net using all of the variables

3. The simplest way to obtain input rankings is to rank the individual increases in MSE when each input variable row is replaced by the zero variance variable row with all values equal to the mean value of the original row.

replx(i,:) = repmat(mean(x(i,:),2), 1, N);

4. The next simplest way is to continue training the original net after the constant row replacement before measuring the increase in MSE.

5. A more convincing method is to replace each input variable row with a random permutation of it's values. This can be repeated Ntrial (e.g., Ntrials = 20) times for each row.

replx(i,:) = repmat(x(i,randperm(1:N)));

6. Same as 5 except that training is continued after each random permutation.

7. Each ranking procedure can be extended to a sequential reduction of variables by discarding the lowest ranked input after each step.

Other toolboxes contain nonneural statistical methods for ranking and reduction variables. However, using a neural approach for a neural model usually yields the best results.

Hope this helps.

Greg

[Sarah]

Hi Greg, Thanks for the advice, your method certainly seems more robust at identifying the ranking of inputs. I have a few questions about some of the points if you don't mind..

For this method:

3.The simplest way to obtain input rankings is to rank the individual increases in MSE when each input variable row is replaced by the zero variance variable row with all values equal to the mean value of the original row.

Does this require that I start an entirely new network each time with no prior training?

As i replace the input rows with the mean of the row the smaller the MSE the more accurate the input and target and so this would be the most highly ranked input? Do I have the correct idea here?

4. The next simplest way is to continue training the original net after the constant row replacement before measuring the increase in MSE.

Would I continue the training when given the option to train on a larger dataset for example in the GUI (unlike above where I begin all over again)?

As I said I am only starting out with neural networks so I apologise if I have missed the point and my questions seem basic.

Thank you for your help

[Greg Heath]

What I described assumes a single data set with no validation set. However, I've never used validation stopping for this task so I cannot say for certain that it will not work.

What I described involves no restarts from the same or a new set of random initial weights. The point is to

use the original design weights to determine the effectiveness of each input variable.

Each time start with the original trained network. If you have I inputs you will create I more input matrices where each new matrix differs from the original by the replacement of one variable row by a constant row containing the mean value of the original row. The option in 4 involves the continuation of training with the constant rows.

Hope this helps.

Greg

[Sarah]

Thank you for the detailed description. One final question if I might? When I have my network trained using my original values and then use:

test1=sim(net,EXP9a_1) ;

where EXP9a_1 is the first of my input matrices in which the first input has been replaced by the mean of the input. How do I then access the MSE value to determine if it is increasing or decreasing?

Many thanks again

Sarah

[Greg Heath]

mse(targetEXP9a_1-test1)

[Greg Heath]

If the input variables are standardized, the mean of the rows are zero. Therefore, just use

replx(i,:) = zeros( 1, N);

instead of

replx(i,:) = repmat(mean(x(i,:),2), 1, N);

Answer 2

[Georgios Mavromatidis]

Hello Greg and Sara.

One quick question. If you have binary variables i.e. 0 or 1 should I normalise them as well?

[Greg Heath]

If you have binary inputs use {-1,1} and tansig hidden node activation units.

If you have independent binary outputs use {0,1} and logsig hidden node activation units.

If you have unit sum binary outputs (e.g.,classification)use {0,1} and softmax hidden node activation units.