Building an array of values from multiple .txt files

Hi, I have several hundreds of files which I need to read, take out 3 columns of data (numbers) and then create a cell of the data so that I can use it within the workspace.

For each of the 3 columns, the data of each file need to be added to the end of the previous column.

So far I have created and ordered an array of the file names to be read and used fopen to open each file in the correct order within a loop. I then use textscan to build a cell array of the data. My problem is actually adding new values to this cell. I have tried using ‘cat’ to concatenate the previous and new cells but this just adds new cells within my C_abxyz cell.

Here is my code so far for the loop:

for i = 1:length(files)
   file = char(files(i));
   fid = fopen(file);
   C_abxyz{:,i} = textscan(fid,'%f%f%f','Delimiter',',','Headerlines',1);
   fclose(fid);

     if i>1
     C_abxyz{i} = cat(1,C_abxyz{i-1}, C_abxyz{i});
  %    C_abxyz{i} = [C_abxyz{i-1} C_abxyz{i}];
  %    XYZ = [C_abxyz{i-1};C_abxyz{i}];
     end
  end

This builds C_abxyz as a 1x154 cell (as there are 154 files). Te cells within C_abxyz then start at a 1x3 cell, then a 2x3 cell, 3x3 cell up to a 154x 3 cell due to the re-definition of C_abxyz at each loop iteration. The 154x3 cell almost does what I need, in the sense that the values of the columns for each file are added below their previous respective columns. But these are all within cells. What I require is 3, 15554x1 cells (101*154 = 15554), containing all of the column data to be created inside the 1x3 C_abxyz cell.

I hope that makes sense. I would very much appreciate any help!

Thanks,

Earle

Answer 1

[Oleg Komarov]

EDIT

% Use vertical allocation and CollectOutput:

C_abxyz(i,:) = textscan(fid,'%f%f%f','Delimiter',',','Headerlines',1,'CollectOutput',1); % Note round brackets

% Then to consolidate all of the data (outside of the loop):

C_abxyz = cat(1,C_abxyz{:});

[Earle]

Thanks Oleg! So from your suggestion, here's the code:

C_abxyz = {}; % Initialize variable

for i = 1:length(files) % Loop

file = char(files(i));

fid = fopen(file);

C_abxyz{i,:} = textscan(fid,'%f%f%f','Delimiter',',','Headerlines',1,'CollectOutput',1);

C_abxyz = cat(1,C_abxyz{:});

fclose(fid);

end

Is this what you were suggesting?

This code returns the error:

"??? The left hand side is initialized and has an empty range of indices.

However, the right hand side returned one or more results.

Error in ==> Open_txt at 9

C_abxyz{i,:} = textscan(fid,'%f%f%f','Delimiter',',','Headerlines',1,'CollectOutput',1);"

Have I done something wrong?

Thanks

[Oleg Komarov]

Get rid of the first C_abxyz = {}; % Initialize variable

Only preallocate it if you know the number of elements:

C_abxyz = cell(20,1);

The in the loop use:

C_abxyz(i,:) = textscan(fid,'%f%f%f','Delimiter',',','HeaderLines',1,'CollectOutput',1);

And outside of the loop:

C_abxyz = cat(1,C_abxyz{:});

[Earle]

ok thats very close - the values are all populated correctly, but its now creating an array of doubles, rather than a cell. C_abxyz needs to be a 1x3 cell containing these 3 columns of doubles.

Thanks,

Earle

[Oleg Komarov]

Why do you need it to be a cell? It's inefficient, just refer to the columns as C_abxyz(:,1).

Use num2cell(C_...,1) to cellify the columns.

[Sean de Wolski]

cellify - word of the day!

[Oleg Komarov]

@Sean: :P

[Earle]

Its working! I built it as a cell because thats how the existing code works. Thanks for your help!

Answer 2

[Bob Hamans]

Hi Earle, something like this should work:

C_abxyz = {}; % Initialize variable
for i = 1:length(files) % Loop
 file = char(files(i));
 fid = fopen(file);
 C_temp = textscan(fid,'%f%f%f','Delimiter',',','Headerlines',1);
 C_abxyz = cat(1,C_abxyz,C_temp); % C_abxyz, growing in loop
 fclose(fid);
end

[Earle]

Thanks Bob! I've run your code but this builds C_abxyz as a 154x3 cell. For every loop iteration i need the new file data to be added to the bottom of each of the 3 cells for C_abxyz.

Any idea how to do this?

Thanks,

Earle

[Bob Hamans]

Could you post like the first five lines of one of your data sets? That would make things so much easier.

[Earle]

Thanks. I've tried that but it's still building C_abxyz as a 154x3 cell. The difference between this version and yur previous suggestion is that the 101x1 double in the leftmost column has been replaced by a 1x3 cell. I feel like its very close, but I'm not sure about quite how to get it to work!

[Earle]

Sure, is this ok? -

21973.7805241942, 121371.107863843, -16359.999999422

21973.7707782736, 121371.163139238, -16385.482754882

21973.7621599047, 121371.212019561, -16410.9655238176

21973.7546690887, 121371.25450481, -16436.448304574

21973.7483058265, 121371.290594981, -16461.9310954964