How to multiply and sum extracted array elements based array indices held in another array?

5 views (last 30 days)
I am trying to perform the following vectorized operation.
I have a large matrix of raw data: RawData with size (A,B). I have computed a set of weights (Weights) which are stored in a matrix of size (ns,nl,t*2). I have also calculated which elements of Raw Data need to be multipled with the weight matrix and these indices are stored in another 3D matrix DataIndex with size (ns,nl,t*2) (one index for the A and B dimensions of RawData). I would like to loop through t*2, pulling the appropriate elements of RawData using the indices in DataIndex, multipling by the weight matrix and accumulating the sum in a final array FResult with size (ns,nl).
My solution looks something like this.
RawData = rand(3000,256 ,'single');
Weights = rand(512,256,158, 'single');
DataIndex = single(ceil(rand(512,256,158)*100)); % some dummy values
FResult = zeros(512,256,'single');
% trying to operate on one 'slice' at a time
for SumEvent = 1:2:size(Weights,3)
% DataIndex is used to reference the x and y indices of RawData
% x index differs only by 1, y index is the same
FResult = FResult + Weights(:,:,SumEvent).*RawData(DataIndex(:,:,SumEvent),DataIndex(:,:,SumEvent+1));
FResult = FResult + Weights(:,:,SumEvent+1).*RawData(DataIndex(:,:,SumEvent+1),DataIndex(:,:,SumEvent+1));
end
Requested 131072x131072 (64.0GB) array exceeds maximum array size preference (30.9GB). This might cause MATLAB to become unresponsive.
% FAILS
% Requested 131072x131072 (64.0GB) array exceeds maximum array size preference (15.9GB)
% fails are matlab tries to create a new array for each element in
% DataIndex (131072 = 256 x 256 x 2)
Is there any way to directly input the array positions and extract the appropriate elements of RawData? I think the for loop can be removed and replaced with some kind of bsxfun command as well.
These matrices are quite large so I'm not quite sure if this is the best approach.
Thanks

Answers (1)

DGM
DGM on 20 Jul 2022
Edited: DGM on 20 Jul 2022
I'm not sure about the index offsets you're using, but this is why you don't try doing scattered indexing using subscripts. If your indexes are scattered, try using linear indexing instead. Something like this might be a start:
RawData = rand(3000,256 ,'single');
Weights = rand(512,256,158, 'single');
DataIndex = single(ceil(rand(512,256,158)*100)); % some dummy values
FResult = zeros(512,256,'single');
% trying to operate on one 'slice' at a time
for SumEvent = 1:2:size(Weights,3)
% DataIndex is used to reference the x and y indices of RawData
% x index differs only by 1, y index is the same
% this uses DataIndex(:,:,SE) and DataIndex(:,:,SE + 1)
Didx = sub2ind(size(RawData),DataIndex(:,:,SumEvent),DataIndex(:,:,SumEvent+1));
FResult = FResult + Weights(:,:,SumEvent).*RawData(Didx);
% this uses DataIndex(:,:,SE + 1) and DataIndex(:,:,SE + 1)
Didx = sub2ind(size(RawData),DataIndex(:,:,SumEvent+1),DataIndex(:,:,SumEvent+1));
FResult = FResult + Weights(:,:,SumEvent+1).*RawData(Didx);
end
The lack of symmetry regarding the +1 offset seems strange to me. Bear in mind that the comment says "x index differs only by 1, y index is the same", but the code suggests the opposite. You might want to verify that's intended.
  1 Comment
Bub
Bub on 20 Jul 2022
Edited: Bub on 20 Jul 2022
The current solution runs but doesn't appear to be correct.
The RawData is essentially a number of channels which are sampling in time. I am just taking a weighted average of two temporally adjacent samples from each channel. I mixed up the comment sorry.
The current solution is rather slow as the array indices must first be extracted and really doesn't appear to be faster than jsut calculating the weights and data indices manually.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by