Bad classification accuracy... What could be making my pattern recognition network perform so poorly?
1 visualizzazione (ultimi 30 giorni)
Mostra commenti meno recenti
Hello Matlab Community. I have run into an issue with my neural network.
My network contains 2000 samples evenly distributed among 4 classes. Each sample contains 100 attributes. The data is best described as particular points in a time sequence (for example, one sample contains 100 data points over 100 seconds--or a 1*100 array). My inputs are a 100*2000 double, and my target is a 4*2000 binary matrix where ones describe the instance of a particular class.
Everything seems to work well, until I get my result.
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/165957/image.png)
My confusion matrices have been showing about a 25% accuracy rate.. and it varies every time I retrain. It looks like my neural network is clumping most of my data points into one class in the training phase, which is causing problems down the line. I would also like to note that each time I retrain, a bulk of the training data gets clumped into different classes. Sometimes, it's class 4 (as shown above), but other times it groups everything into class 1,2, or 3.
Here is what my data looks like plotted, each plot represents each of my four classes. I feel like these would be pretty easy to distinguish given their size alone, but that does not seem to be the case.
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/165958/image.png)
I have tried a number of things.. changing the training function, number of layers, layer sizes, ect... I'm still resulting in poor classification. Does anyone know what may be causing this? I would also like to note that I am using nnstart to work my way through the network design, then pulling the script from nntraintool. I have it set at the standard 70% train, 15% test, 15% train. Thanks
Edit: My code is below.
% Solve a Pattern Recognition Problem with a Neural Network
% Script generated by Neural Pattern Recognition app
% Created 11-Jul-2017 14:40:12
%
% This script assumes these variables are defined:
%
% LargeRadar - input data.
% LargeRadarTarget - target data.
x = LargeRadar;
t = LargeRadarTarget;
% Choose a Training Function
trainFcn = 'trainscg'; % Scaled conjugate gradient backpropagation.
% Create a Pattern Recognition Network
hiddenLayerSize = 15;
net = patternnet(hiddenLayerSize, trainFcn);
% Choose Input and Output Pre/Post-Processing Functions
net.input.processFcns = {'removeconstantrows','mapminmax'};
net.output.processFcns = {'removeconstantrows','mapminmax'};
% Setup Division of Data for Training, Validation, Testing
net.divideFcn = 'dividerand'; % Divide data randomly
net.divideMode = 'sample'; % Divide up every sample
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
% Choose a Performance Function
net.performFcn = 'crossentropy'; % Cross-Entropy
% Choose Plot Functions
net.plotFcns = {'plotperform','plottrainstate','ploterrhist', ...
'plotconfusion', 'plotroc'};
% Train the Network
[net,tr] = train(net,x,t);
% Test the Network
y = net(x);
e = gsubtract(t,y);
performance = perform(net,t,y)
tind = vec2ind(t);
yind = vec2ind(y);
percentErrors = sum(tind ~= yind)/numel(tind);
% Recalculate Training, Validation and Test Performance
trainTargets = t .* tr.trainMask{1};
valTargets = t .* tr.valMask{1};
testTargets = t .* tr.testMask{1};
trainPerformance = perform(net,trainTargets,y)
valPerformance = perform(net,valTargets,y)
testPerformance = perform(net,testTargets,y)
% View the Network
view(net)
genFunction(net,'myNeuralNetworkFunction');
y = myNeuralNetworkFunction(x);
end
Greg, I have re-uploaded my question with code included. Hope this helps
0 Commenti
Risposte (1)
Mukul Rao
il 19 Lug 2017
Hi,
You can try steps mentioned in the following links to improve neural network performance :
As far as I understand there is no recipe to make a good network, it takes some trial and error. I would also be vary of my dataset, does it make sense in terms of having adequate unique information for my network to learn well?
You could consider using "processpca" to check if there are redundant attributes and reduce the size of the training dataset.
1 Commento
Greg Heath
il 19 Lug 2017
Classes 2 and 4 appear to have completely different scales
Classes one and three seem to be contaminated with outliers.
Sorry, have to go!
Vedere anche
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!