not getting good results while simulation in NNTOOL

hello sir, i am using nntool of matlab for my prediction my application is to forecast water demand based on previous consumption. year month day net supply 2003 1 1 2900 2003 1 1 2915 . .

      2010       12         31      3400.
my input matrix has 3 inputs month day and date and output matrix is simply net supply.i have trained my network using
nntool i got good results in training,validation and testing.ie(0.93).but when i try to simulate the network using nntool to forecast the consumption 
for 2011 year i am not getting good results.
i mean if the actual consumption on 1st jan 2011 is 3400 i am getting it as 2900.
can you give me any sugession how to improve the results.

Answer 1

[Greg Heath]

continued ...

 You should scale and center the data.  zero-mean/unit-variance inputs  with tansig hidden node activation units are recommended..

 help prestd

 doc prestd

 similarly for trastd and poststd.

 >%creating network

 Initialize the uniform random number generator RAND so that 
 you can duplicate the results if necessary.

 > net=newff(minmax(x),[3,12,1],{'logsig','logsig','purelin'},'trainlm');

 Do not include the number of input nodes. That info is obtained 
 from minmax(x)

 >net.performFcn = 'mse';

 Delete. It is a default value.

 >net.trainParam.goal = 1e-4;

 Use a better informed choice that takes the data scaling into account..

 >net.trainParam.min_grad = 1e-20;

 >net.trainParam.epochs = 5000;

 Where did you get these values?

 Use as many defaults of newff  as possible.

 help newff

 doc newff

 >net=init(net);

 Delete. NEWFF is self-initializing.

 >net=train(net,x,y);
 >z=sim(net,x);

 Use the multiple output option.

 >test=sim(net,a);

 %%%%%%%%%%%%%%%%%%%%%%

 MSEtrn00    = mse(y-mean(y))    % Naive Constant Model

 MSEgoal      = MSEtrn00/100           % R^2 >= 0.99

 state0      = 0                             % Any integer < 2^32-1

 rand('state',state0)                     % help RAND

 net = newff(minmax(x),[12 3]);

 net.trainParam.goal = MSEgoal;

 [ net tr Y E] = train(net,x,y);       % help TRAIN, help TRAINLM

 MSEtrn = tr.perf(end)                 % = mse(y-Y)

 R2trn    = 1-MSEtrn/MSEtrn00

 MSEtst00 = mse(b-mean(b))

 tst             = sim(net,a);                   % test is not a good name

 MSEtst      = mse(b-tst);

 R2tst         = 1-MSEtst/MSEtst00

 % %evaluating errors

 -----SNIP

 >i tried with 10-30 neurons but the result is not good 
 >can you tell where its going wrong

 Implement a double loop over choice of H and creation of a 
 number of nets for each value of H.

 Hope this helps.

 Greg

Answer 2

[Greg Heath]

It is difficult to give advice when you haven't fully specified your problem.

How much data do you have per year? Is it evenly spaced?

Clarify the ranges of day and date (1to7, 1to31 or 1to366?)

To better understand your data, I recommend you color code and overlay plots of the 8 output series vs each input. Is the 2010 data similar to that of the previous years? Is it similar to the average?

I would think that day of the year and day of the week should be sufficient inputs.

What net, node topology and parameters did you use?

Any other important info?

Hope this helps.

Greg

[niranjan sane]

hello gary

my input matrix has three inputs

year month day netsupply

2003 1 1 2900

2003 1 2 2913

.

.

.

.2010 12 31 3400

where year,month and day are inputs and netsupply is output

range of day is 1-31

month is 1-12

year is 2003-2010

i am having the net supply for each day from 2003-2010.

i have used nntool with defaults settings,ie training by back propotion algorithm,no of neurons as 10,and i have trained the network with this data i got overall regression plot as 0.93.

but when i try to simulate(predict)the output for 2011 results are not good.

if my actual consumption on 1st jan2011 is 3400

i am getting it as 2900 usng nntool.

Answer 3

[niranjan sane]

hello gary my input matrix has three inputs year month day netsupply 2003 1 1 2900 2003 1 2 2913 . . . .2010 12 31 3400 where year,month and day are inputs and netsupply is output range of day is 1-31 month is 1-12 year is 2003-2010 i am having the net supply for each day from 2003-2010. i have used nntool with defaults settings,ie training by back propotion algorithm,no of neurons as 10,and i have trained the network with this data i got overall regression plot as 0.93. but when i try to simulate(predict)the output for 2011 results are not good. if my actual consumption on 1st jan2011 is 3400 i am getting it as 2900 usng nntool.

Answer 4

[Greg Heath]

My nane is Greg; not Gary.

To better understand your data, I recommend you color code and overlay plots of the 9 output series vs each input. Are the 2010 and 2011 data similar to that of the previous years? Are they similar to the average? If so, the day of the year and day of the week should be sufficient inputs. Do those two plots look better than the three that you already have?

You are using 8 years for design. In order to try to optimize the design, I recommend REPEATEDLY using the first 7 years for training, and the 8th year for validation. I typically loop over 10 candidate designs obtained with random intial weights automatically determined by newff for each of 10 candidate values for No of hidden nodes, H. The net with the minimum of the 100 validation set errors is chosen.

For an I-H-O = 3-H-1 topology with Ntrn = 2+7*365 = 2557, the number of training equations is Neq = Ntrn*O = 2557 and the number of unknown weights is Nw = (I+1)*H+(H+1)*O = O +(I+O+1)*H = 1+5*H.

For robust estimation of weights it is desired that Neq >> Nw or

H << (Neq-O)/(I+O+1) = 2556/5 ~ 511

I suggest two runs:

1. A coarse search of 50 designs for H = 10:10:50 to determine a smaller range Hmin:Hmax 2. A finer search of 100 designs over the smaller range.

Hope this helps.

Greg

Answer 5

[niranjan sane]

thanks sir,actually i am very new to neural network so not able to understand wha you are tring to convey. yes the data is nearly same for 2010 and 2011 year with little change in it. i have constructed a neural network having 3 layers. input layer having 3inputs and one output layer i used 6yeras data for training and 2years data for testing.nntool is giving me good results but poor prediction. so i developed one code but this dode is giving me worst reults. can you tell me wats wrong in code. clc; clear qll; %preparing inpput data

%training datas
data3=xlsread('training');
data3=data3'
x1=data3(1,:);
x2=data3(2,:);
x3=data3(3,:);
x=[x1;x2;x3];
y=data3(4,:);

%testing data
data4=xlsread('testing');
data4=data4';
a1=data4(1,:);
a2=data4(2,:);
a3=data4(3,:);
a=[a1;a2;a3];
b=data4(4,:);

%creating network
net=newff(minmax(x),[3,12,1],{'logsig','logsig','purelin'},'trainlm');
net.performFcn = 'mse';
net.trainParam.goal = 1e-4;
net.trainParam.min_grad = 1e-20;
net.trainParam.epochs = 5000;
net=init(net);
net=train(net,x,y);
z=sim(net,x);
test=sim(net,a);

% %evaluating errors
% %--training
% error=(abs(z-y)./y)*100;
% n=0;
% for i=1:1460
%     if error(i)<10;
%         n=n+1;
%     end
% end
% P=(n/1460);
% %--testing
% error2=(abs(test-b)./b)*100;
% n=0;
% for i=1:730     
%     if error2(i)<10;
%         n=n+1;
%     end
% end
% P2=(n/730);
i tried with 10-30 neurons but the result is not good can you tell where its going wrong

Answer 6

[Greg Heath]

 >thanks sir,actually i am very new to neural network so not able to  >understand wha you are tring to convey. yes the data is nearly same  >for 2010 and 2011 year with little change in it.

 I asked about the similarity of 2010 and 2011 to the 7 earlier years

 > i have constructed a neural network having 3 layers. 

 No. NNs are characterized by the number of weight layers ; NOT the  number of node layers. You have a 2-layer net.

 Since all classification and regression nets have inputs and outputs, 
 It is better to characterize the nets by the number of hidden layers.  Then there is no source of confusion.

 >input layer having 3 inputs and one 
 >output layer i used 6yeras data for training and 2years data for  >testing.

 WHY?? I told you previously that you needed a validation set to  determine the best of many candidate designs! The test set is used  once and only once on the best candidate chosen by the validation  set. That is a proper way to get an UNBIASED prediction of  performance on nondesign data when the single best or a best group of candidate designs are chosen.

 >nntool is giving me good results but poor prediction. 

 I'm not surprised. When training to convergence without regularization,  good performance on training data is not sufficient for expecting good  performance on nontraining data. That is why you need a validation set.

 > so i developed one code but this dode is giving me worst reults. can  > you tell me wats wrong in code. 

 Basically, you did not use a validation set to choose the best of many  candidate designs.

>clc; clear qll; %preparing inpput data

>%training datas

 -----SNIP

 >%testing data

 -----SNIP