replacing the for loop please

How can I replace the for loops?

L=(length(time)-1);
 for q=1:numPT
    for  h=1:L
    x(q,h)=SortDate{q+numPT*(h-1),6};
    y(q,h)=SortDate{q+numPT*(h-1),7};
    c(q,h)=SortDate{q+numPT*(h-1),5};
    end
end

[pg]

As I have to process a large amount of data, so I need to find ways to replace the for loops. Thanks

[Image Analyst]

What does "large" mean to you? Like 1000, or 100 million? What are typical values for L and numPT?

[pg]

around 10 million, so it's better with I get replace the for loops.

[Stephen23]

"so it's better with I get replace the for loops"

Why? What is the problem with them?

Loops are not slow. Badly written code is slow, but as you have not shown us much of your code then we don't have enough information to know what changes your code might need, if any. For example, are the output arrays preallocated?. The most useful information for you is contained on this page, and the pages it links to:

https://www.mathworks.com/help/matlab/matlab_prog/techniques-for-improving-performance.html

Answer 1

[Jan]

Before you start to vectorize the code, did you pre-allocate the output before the loops? This is essential:

L = (length(time)-1);
x = zeros(numPT, L);
y = zeros(numPT, L);
c = zeros(numPT, L);
for q = 1:numPT
  for h=1:L
    x(q,h) = SortDate{q+numPT*(h-1),6};
    y(q,h) = SortDate{q+numPT*(h-1),7};
    c(q,h) = SortDate{q+numPT*(h-1),5};
  end
end

Now compare the timings. Then in the next step remove the inner loop:

L = (length(time)-1);
x = zeros(numPT, L);
y = zeros(numPT, L);
c = zeros(numPT, L);
for q = 1:numPT
  x(q, :) = [SortDate{q+numPT*(0:L-1), 6}];  % Faster than: x(q, 1:L) = ...
  y(q, :) = [SortDate{q+numPT*(0:L-1), 7}];
  c(q, :) = [SortDate{q+numPT*(0:L-1), 5}];
end

And now the outer loop - and then a pre-allocation is not required:

L = (length(time)-1);
index = (1:numPT).' + (0:numPT:numPT*(L-1)));  % >= R2016b
x = reshape(cell2mat(SortDate(index, 6)), size(index));
y = reshape(cell2mat(SortDate(index, 7)), size(index));
c = reshape(cell2mat(SortDate(index, 5)), size(index));

Do you see how the indices are moved from the for loop into the vector indices?

If you have an older Matlab version, you need bsxfun instead of the modern auto-expanding:

index = bsxfun(@plus, (1:numPT).', (0:numPT:numPT*(L-1)));

If this is still the bottleneck of you code, try to use the faster FEX: Cell2Vec, which seems to have a better memory management. Perhaps it is faster to convert the complete SortDate array at once:

L     = (length(time)-1);
index = (1:numPT).' + (0:numPT:numPT*(L-1)));  % >= R2016b
siz   = size(index);
M     = Cell2Vec(SortDate(:, 5:7));
x     =  reshape(M(index, 2), siz);
y     =  reshape(M(index, 3), siz);
z     =  reshape(M(index, 1), siz);

[pg]

Thanks so much. This is absolutely useful.