How to split matrix data into training, validation and testing data based on a categorical label?
4 visualizzazioni (ultimi 30 giorni)
Mostra commenti meno recenti
Kimberly Cardillo
il 20 Ago 2020
Risposto: Srivardhan Gadila
il 23 Ago 2020
I have a 184 x 1 cell array called aefm_4 and each cell is 1 x 1599. Each row (184) has a corresponding categorical label in a separate variable called f and its either labeled 'crack-related' or 'noise'. I want to split up my data so that I have 70% training data, 10% validation data and 20% testing data. I want about even amounts of noise and crack related signals in each training, validation and testing dataset. How would I do that?
Right now, I have been manually separating them as seen below but I will be running this for more data and it is too time consuming to do this.
dataTrain = aefm_4([1:9,14:23,28:47,55:75,87:90,93:101,108:152,169:177],:); %127 hits (70%)
fTrain = f([1:9,14:23,28:47,55:75,87:90,93:101,108:152,169:177],:);
dataVal = aefm_4([12:13,26:27,53:54,83:86,92,105:107,163:168],:); %20 hits (10%)
fVal = f([12:13,26:27,53:54,83:86,92,105:107,163:168],:);
dataTest = aefm_4([10:11,24:25,48:52,76:82,91,102:104,151:162,178:184],:); %39 hits (20%)
fTest = aefm_4([10:11,24:25,48:52,76:82,91,102:104,151:162,178:184],:);
0 Commenti
Risposta accettata
Srivardhan Gadila
il 23 Ago 2020
You can do something like below:
%% Create categorical label data
numLabels = 18
labels = randi([1 2],numLabels,1);
labelsCat = categorical(labels,[1 2],{'crack-related','noise'});
%% find lndices w.r.t each label
crackInd = find(labelsCat==categorical("crack-related"));
noiseInd = find(labelsCat==categorical("noise"));
%% divide into train, val, test for each label
[trainCrack,valCrack,testCrack] = dividerand(numel(crackInd),0.7,0.1,0.2);
[trainNoise,valNoise,testNoise] = dividerand(numel(noiseInd),0.7,0.1,0.2);
%% get final indices
trainInd = [crackInd(trainCrack); noiseInd(trainNoise)];
valInd = [crackInd(valCrack); noiseInd(valNoise)];
testInd = [crackInd(testCrack); noiseInd(testNoise)];
You can optimize the above code based on your requirement like, dividing the indices randomly or not. Refer to Divide Data for Optimal Neural Network Training, dividerand, divideblock, divideint, divideind for more information.
trainInd = trainInd(randperm(numel(trainInd)));
0 Commenti
Più risposte (0)
Vedere anche
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!