Create a growing array

Hi, all. I'm a Matlab novice, so your help is greatly appreciated. I'm attempting to create an array of data as I proceed through a loop. I'll be doing a calculation between lat/long points to figure out distances between each and every pair. If the distance is small enough, I want to write several columns of information to some sort of array. This will include numbers and strings. Since I have no idea how many pairs will qualify, I don't know how big this array needs to be. Can you help me find a solution to this growing array problem? Thanks in advance.

Answer 1

[Guillaume]

Since you want to store both strings and numbers you have to use a cell array (or a table). As for matrices, continuously growing a cell array gets expensive after a certain size in term of processing time as matlab needs to reallocate a new chunk of memory and copy the old array there each time.

You have two options, accept the cost, in which case:

storage = {}; %start with empty cell array
for iter = 1 : 10000  %e.g.
   if rand > 0.5      %whatever condition you have
      storage = [storage; {'aaaa', pi}]; %add a new row of data
   end
end

The other option is to allocate a cell array bigger than you'll ever need and trim at the end. If you know an upper bound for the number of rows in the array (the number of points maybe?), it's easy:

npoints = 10000;
storage = cell(npoints, 2); %predeclare array bigger than needed
currentrow = 1; %keep track of where we are with filling
for iter = 1 : npoints
  if rand > 0.5
    storage(currentrow, :) = {'aaaaa', pi};
    currentrow = currentrow + 1;
  end
end
storage(currentrow:end, :) = []; %remove unused rows at the end

[Jascha Polet]

I tried this:

storage = {}; %start with empty cell array

for i = [1:NumStations]

for j = [1:NumEvents]

[arclen,az] = distance(StLat(i),StLong(i),EvLat(j),EvLong(j))

distkm = distdim(arclen,'deg','km')

          if distkm <= 500   %%%%%%%%%%%%%%%%%This value needs to be adjusted... only so high for test

              EventID(k) = Events.EventID(j)

              Evla(k) = Events.Latitude(j)

              Evlo(k) = Events.Longitude(j)

              Station(k) = Stations.Station(i)

              Stla(k) = Stations.Latitude(i)

              Stlo(k) = Stations.Longitude(i)

              Dist(k) = distkm

              storage = [storage; {EventID,Evla,Evlo,Station,Stla,Stlo,Dist}]; %add a new row of data

          end

     end

end

Where Station is a string that is 3 letters long. The result array only has 1 letter in that cell. Any other suggestions?

[Walter Roberson]

You do not initialize or change k anywhere so you are always writing to th same place for those arrays.

[Guillaume]

To answer your immediate problem we need to know what is

class Stations.Station
%and
size(Stations.Station)

However, you may be able to do away with the loops entirely. Can distance and distdim operate on vectors or matrices or are they just restricted to scalar?

Even if they are, you can take the if operation out of the loop:

 [statsidx, eventsidx] = ndgrid(1:NumStations, 1:NumEvents);
 distkm = zeros(numel(statsidx), 1); %this needs to be a column vector for the last line to work correctly

 %if distance can take matrices the loop is not even required
 %if distdim can take matrices the call can be taken out of the loop
  for idx = 1 : numel(statsidx)
     arclen = distance(StLat(statsidx(idx)), StLong(statsidx(idx)), EvLat(eventsidx(idx)), EvLong(eventsidx(idx)));
     distkm(idx) = distdim(arclen,'deg','km');
  end

%filtering everything at once
validdistance = distkm <= 500; %%%%%%%%%%%%%%%%%This value needs to be adjusted... only so high for test

%the following may not work since you've got an error in your original code
%needs to know the class and size of the structure fields to write it correctly
valideventidx = eventsidx(validdistance);
valideventid = Events.EventID(valideventidx);
valideventlat = Events.Latitude(valideventidx);
valideventlong = Events.Longitude(valideventidx);
validstationidx = statsidx(validdistance);
validstation = Stations.Station(validstationidx );
validstatlat = Stations.Latitude(validstationidx );
validstatlong = Stations.Longitude(validstationidx );
storage = [num2cell(valideventid(:)), ...
           num2cell(valideventlat(:)), ...
           num2cell(valideventlong(:)), ...
           num2cell(validstation(:)), ...  %num2cell may not be necessary
           num2cell(validstatlat(:)), ...
           num2cell(validstatlong(:)), ...
           num2cell(distkm(validdistance))];

Answer 2

[KSSV]

x = [] ;
for i = 1:100
x = [x ; i] ;
end

Answer 3

[Walter Roberson]

Use pdist2 to calculate all the pairwise distances. You can then find() the ones that meet the distance limits. The number of matches tells you how much space you need to allocate.

[Jascha Polet]

I looked up pdist2 and don't see a way to use it with arclen or otherwise use degrees. Do you know how to do this?

[Walter Roberson]

You can pass a function handle as the distance type:

map_dist_km = @(oneplace, manyplaces) arrayfun(@(location) distdim( distance(oneplace(1), oneplace(2), location(1), location(2)), 'deg', 'km'), manyplaces);

distkm = pdist2([StLat(:), StLong(:)], [EvLat(:), EvLong(:)], map_dist_km);

[r, c] = find( distkm <= 500 );

numres = size(r,1);

EventID = zeros(numres,1);
Evla = zeros(numres,1);
Evlo = zeros(numres,1);
Station = zeros(numres,1);
Stla = zeros(numres,1);
Stlo = zeros(numres,1);
Dist = zeros(numres,1);

for k = 1 : numres
  i = r(k);
  j = c(k);
  EventID(k) = Events.EventID(j);
  Evla(k) = Events.Latitude(j);
  Evlo(k) = Events.Longitude(j);
  Station(k) = Stations.Station(i);
  Stla(k) = Stations.Latitude(i);
  Stlo(k) = Stations.Longitude(i);
  Dist(k) = distkm(i,j);
end