Training data and Training target in Neural Networks

Hi,

I am having a signal in form of vector (1*25000). I want to split this signal into four parts x_train, y_train, x_test and y_test (according to 70-30% training and testing method) in MATLAB. Can anyone help me how to split this vector form signal into these four parts?

Thanks

[BN]

This is a general question. you haven't told me how method do you want to split your data? do you want to split them randomly (from first, middle, and end of the dataset) or you want to use the first part as train and end part as a test?

Anyway, You don't need to do this manually, Matlab can automatically divide your data set in which way you want to split (random or not random). You can get more information Here. Or type:

help nndivide

You can choose the function do you want then using it in the code. For example, I want to split my data according to which way I want (first part for train and middle for validation and end part for test). So first I split my data in the indTrain, indVal, indTest by index them, then using divideind.

% For a list of all data division functions type: help nndivide
net.divideFcn = 'divideind';  % Divide data %Divide targets into three sets using specified indices%
net.divideMode = 'sample';  % Divide up every sample
net.divideParam.trainInd = indTrain; % 70% of the data set was used for train
net.divideParam.valInd = indVal; % 15% of the data set was used for validation
net.divideParam.testInd = indTest; % 15% of the data set was used for test

[MAT-Magic]

Thanks for the answer. Here, I am confused becuase I just have recorded respiratory signal in this array 1*25000, which is unlabled data according to my understanding without any training target. But for neural networks training on training data, I need to have the corresponding training target, which is I am not having right now.

Anyways, I did it in the following way. Can you please review the code, and tell me whether I am in the right track or not?

signal = data(1:25000);
[m,n] = size(signal);
P = 0.70 ;
idx = randperm(m);
train_data = signal(idx(1:round(P*m)));            %% 17500*1 (dimension)
test_data = signal(idx(round(P*m)+1:end));         %% 7500*1  (dimension)
%% For training data:
colnr_1 = 2;
rownr_1 = 17500/2;
mat_1 = reshape(train_data, [rownr_1, colnr_1]); 
x_train = mat_1(:,1);
y_train = mat_1(:,2);
%% For testing data:
colnr_2 = 2;
rownr_2 = 7500/2;
mat_2 = reshape(test_data, [rownr_2, colnr_2]); 
x_test = mat_2(:,1);
y_test = mat_2(:,2);

Waiting for the positive feedback. Correct me If I am wrong anywhere. Thanks.

Please go through this below URL, it might be related to my problem.

Stackoverflow

Answer 1

[Greg Heath]

You cannot make any intelligent decisions until you have examined a plot of the data!!!

(WRONG!!! Plotting the data first is the ultimate beginning decision!!!)

Hope this helps.

Greg

Answer 2

[Mahesh Taparia]

Hi

You have correctly divided the data using randperm. Since you didn’t have ground truth, you are taking last 8750 as ground truth as per following code:

mat_1 = reshape(train_data, [rownr_1, colnr_1]);  
x_train = mat_1(:,1); 
y_train = mat_1(:,2); 

which is incorrect. Select the correct ground truth.

[MAT-Magic]

Dear Mr. Mahesh Taparia,

Thank you so much for your kind reply.

FYI, I have recorded respiratory data which is having time and data (amplitude) information in form of two vectors having obviuosly same dimension (1*25000). Accroding to you, untill randperm, I am right. Then how will I take x_train and y_train for training according to given information, because you told me taking last 8750 as ground truth will be incorrect strategy?

Waiting for the your positive feedback. Thanks

[Mahesh Taparia]

Hi

You mentioned earlier that your dataset is unlabeled, y_train would be the labels of x_train. Taking y_train (labels of x_train) as half of the data (which is amplitude) is illogical.

For supervised learning, there is a need of ground truth so collect the labels. Or else you can try with unsupervised learning approach like clusteriung.