Sequence-to-Label Classification Using 1-D Convolutions

I'm trying to build a Time Convolutional Network for sequence classification which can perform the same task of an LSTM network with 'output mode' equal to 'last'. In fact, all the time points of a sequence have the same label and different sequences may have different labels.

How can I modify the code proposed for Sequence-to-Sequence Classification Using 1-D Convolutions (https://www.mathworks.com/help/deeplearning/ug/sequence-to-sequence-classification-using-1-d-convolutions.html) in order to perform a sequence-to-one task?

When using a LSTM network with 'output mode' equal to 'last' the lstm layer returns the hidden state vector of the last time step only and provides it to the fully connected layer.

I guess I have to work on the output of the convolution operation (using the function dlconv) of the last convolutional layer before the fully connected layer. Suppose to apply the convolutional operation to a batch of 6 observations and the length of each sequence is equal to 1440 and the number of features is 20. The output of the convolutional operation (dlconv) is a dlarray of size NumFilters × 6 ×1440.

However, I don't know how to collapse time dimension of the output of the convolutional operation to only value. Is the same for LSTM networks, I mean I have to consider the output of the last time step only?

Thank you in advance

[James Z]

If this issue is not resolved yet, adding a classifier layer at the end may help.

Answer 1

[Aditya]

For a sequence-to-one classification task using a Time Convolutional Network (TCN) rather than an LSTM, you indeed need to modify how the network handles the output from the convolutional layers before passing it to the fully connected layer. Specifically, you want to collapse or reduce the temporal dimension such that each sequence's representation is condensed into a single vector, akin to taking the output of the last time step in an LSTM configured with 'output mode' set to 'last'.

Given a convolutional layer's output dlconv as a dlarray of size NumFilters × SequenceLength × BatchSize (assuming 'SBC' format for simplicity), you can achieve the sequence-to-one task by selecting the output of the last time step for each sequence. Here's how you can modify the output to achieve this:

Example Approach

If the output of your last convolutional operation is a dlarray named convOutput with the size NumFilters × SequenceLength × BatchSize, you can select the last time step's output using slicing:

% Assuming convOutput is the output of the convolutional layer
% and its size is NumFilters × SequenceLength × BatchSize
% We want to select the last time step of each sequence

% First, ensure convOutput is a formatted dlarray with 'SBC' format
% S: Spatial (or Sequence Length), B: Batch, C: Channel (or NumFilters)
if ~isdlarray(convOutput)
    convOutput = dlarray(convOutput, 'SBC');
end

% Select the last time step's output for each sequence in the batch
lastTimeStepOutput = convOutput(end, :, :);

% Now lastTimeStepOutput has size 1 × BatchSize × NumFilters
% You might need to permute dimensions to fit the input requirement of the fully connected layer
lastTimeStepOutput = permute(lastTimeStepOutput, [3, 2, 1]);

% Now lastTimeStepOutput is ready to be fed into the fully connected layer

[Enzo Losi]

Thank you so much for your help!

Enzo