Could you look over the modified Autoencoder MATLAB code for regression?

Hi there,

I am new in Deep Learning. I would like to predict my target variable (time to 1st break) using Autoencoder Neural network. So I modified the Autoencoder example code, which was originally fit for a classification work in the MathWorks. The modified code is as below:

% Data
x = dataSPANN;
t = Timeto1stbreak;

% set the random number generator seed
rng('default')

% set the size of the hidden layer for the 1st autoencoder
hiddenSize1 = 10;

% Specify the values for the regularizers to learn a sparse representation in the 1st layer 
autoenc1 = trainAutoencoder(x,hiddenSize1, ...
    'MaxEpochs',1000, ...
    'DecoderTransferFunction','purelin', ...
    'L2WeightRegularization',0.004, ...
    'SparsityRegularization',4, ...
    'SparsityProportion',0.05, ...
    'ScaleData', false);

% xReconstructed1 = predict(autoenc1, x);

% view a diagram of the autoencoder
view(autoenc1)

% Set up Division of Data for Training, Validation, Testing
% net.divideParam.trainRatio = 70/100;
% net.divideParam.valRatio = 15/100;
% net.divideParam.testRatio = 15/100;

% Train the 1st autoencoder
feat1 = encode(autoenc1,x);

% set the size of the hidden layer for the 2nd autoencoder
hiddenSize2 = 7;

% Specify the values for the regularizers to learn a sparse representation in the 2nd layer
autoenc2 = trainAutoencoder(feat1,hiddenSize2, ...
    'MaxEpochs',400, ...
    'DecoderTransferFunction','purelin', ...
    'L2WeightRegularization',0.002, ...
    'SparsityRegularization',4, ...
    'SparsityProportion',0.05, ...
    'ScaleData', false);

% xReconstructed2 = predict(autoenc2, feat1);

% Train the 2nd autoencoder
feat2 = encode(autoenc2,feat1);

% Train the final softmax layer
% softnet = trainSoftmaxLayer(feat2,t,'MaxEpochs',400);
% softnet = trainSoftmaxLayer(feat2,t,'LossFunction','mse');

% Choose a Training Function
% trainFcn = 'trainlm';  % Levenberg-Marquardt backpropagation.

% Create a Fitting Network
% net = fitnet(hiddenSize2,trainFcn);
% net = feedforwardnet(hiddenSize2);

net = network;                                  % create network
net.numInputs = 1;                              % set number of inputs
net.inputs{1}.size = 7;                         % assign 2 to input size
net.numLayers = 1;                              % add 1 layer to network
net.layers{1}.size = 1;                         % assign number of neurons in layer
net.inputConnect(1) = 1;                        % connet input to layer 1
net.biasConnect(1) = 1;                         % connect bias to layer 1
net.biases{1}.learnFcn = 'learnp';              % set bias learning function
net.outputConnect(1) = 1;
net.layers{1}.transferFcn = 'purelin';          % set layer transfer function [hard limit]
net.inputWeights{1}.learnFcn = 'learnp';        % set input weight learning function
net.trainFcn = 'trainlm';                        % set network training function
net.performFcn = 'mse';                         % set network perf evaluation function
view(net)

net = train(net,feat2,t);
% net.performFcn = 'mse';  % Mean Squared Error

% view a diagram of the softmax layer
% view(net)

% view(autoenc1)
% view(autoenc2)

% Form a stacked neural netw
deepnet = stack(autoenc1,autoenc2,net);

% view a diagram of the stacked network
view(deepnet)

% Train the stacked network
deepnet = train(deepnet,x,t);

outputs = deepnet(x);

errors = gsubtract(outputs,t);
performance = perform(deepnet,t,outputs);

figure, plotfit(deepnet,t,outputs)
figure, plotregression(t,outputs)
figure, ploterrhist(errors)

However, the result (correlation) seems to have something wrong and less than the method (shallow Neural network) as shown:

Could you advise me, please?

[Machine Learning Enthusiast]

If you working on Regression problem with Autoencoders,you can contact me at [e-mail] for further details.

[Dr Nagwan Abdulsamie]

I am researcher and working with matlab. I am workin on genes expression matrix not images and I am trying to predict the input using Deep networks. I am trying to build a stack of autoencoders followed by a regression layer. However, in matlab, I have to use a trained network such as the softmax layer in the stack.