How to convert rows of matrices to entries in multidimensional struct.

Hi,

I have a large matrices with consistant number of rows. I wish to create a multidimensional struct with each row contained in its own dimension of the struct.

As I see it, I have two options.

1. Dynamically resize the struct each time:

for i = 1:size(data,1)
    myStruct(i).data = data(i,:);
    myStruct(i).data2 = data2(i,:);
    myStruct(i).data3 = data3(i,:);
end

2. First convert the data into cells containing each row via num2cell and then creat the struct:

%Split each row into a new cell
data = num2cell(data,2);
data2 = num2cell(data2,2);
data3 = num2cell(data3,2);

%Each cell of the inputs gets placed into the next index of the struct
myStruct = struct('data',data,'data2',data2,'data3',data3)

I suppose my question is really about the efficiency of these two methods.

Which one of these two methods should run quicker over large data sets?

Is there a better way to convert matrices to a struct of this fashion that I have not thought of?

Thanks in advance!!

Answer 1

[Rajani Mishra]

I think first option of Dynamically resizing the struct each time will run faster as compared to the second option of convert data into cells and then creating struct as in the second option there is an overhead of num2cell() function call.

Also, I tried both of your options with random data of size 100000 * 100000.

%First option -> 4.891826527670000e+04 milliseconds
for i = 1:size(data,1)
    myStruct(i).data = data(i,:);
    myStruct(i).data2 = data2(i,:);
    myStruct(i).data3 = data3(i,:);
end

%Second option -> 1.429934315767000e+05 milliseconds
%Split each row into a new cell
data = num2cell(data,2);
data2 = num2cell(data2,2);
data3 = num2cell(data3,2);
%Each cell of the inputs gets placed into the next index of the struct
myStruct = struct('data',data,'data2',data2,'data3',data3)

Also, second option of creating cells and then struct will hamper optimization while code generation in MATLAB.

[Nicholas Ayres]

Thank you!

I was thinking that, depending on how the structure is created in the "struct" command, this may have been quicker. Otherwise, I don't really see the use of the "struct" command itself over dynamic creation.

It's really helpful to see the numbers attached to this.

Thanks again,

Nicholas.

[Guillaume]

Note that with the loop, you should still preallocate the structure before the loop

mystruct = struct('data', cell(size(data, 1), 1), 'data2', [], 'data3', []);  %preallocate structure
for i = 1:size(data, 1)
    %...
end

I'm surprise that num2cell is so much slower than the loop. I can't test with 1e6x1e6 arrays (this require 7 TB of memory per array! What machine was it tested on?), on a more reasonable 1e4x1e4 arrays, the loop takes about 4.5 s on my computer and num2cell about 9 s.

For the record, this is the code I used for testing:

function testtiming
    data = rand(1e4);
    data2 = rand(1e4);
    data3 = rand(1e4);
    fprintf('With a loop: ');
    tic; 
    s1 = method1(data, data2, data3);
    toc
    fprintf('With num2cell: ');
    tic;
    s2 = method2(data, data2, data3);
    toc;
    assert(isequal(s1, s2), 'Something went wrong');
end

function s = method1(data, data2, data3)
    s = struct('data', cell(size(data, 1), 1), 'data2', [], 'data3', []);
    for i = 1:size(data, 1)
        s(i).data = data(i, :);
        s(i).data2 = data2(i, :);
        s(i).data3 = data3(i, :);
    end
end

function s = method2(data, data2, data3)
    s = struct('data', num2cell(data, 2), 'data2', num2cell(data2, 2), 'data3', num2cell(data3, 2));
end