How to import multiple ascii files into an array using textscan or dlmread?

I am trying to import 1000+ ascii data files of the format:

++++++ Detector-2-1.Spe ++++++

|*DETDESC# PHPC243 MCB 131

AP# Maestro Version 6.08

$DATE_MEA:

05/06/2014 16:30:43

$MEAS_TIM:

40 41

$DATA:

0 1023

       0
       0
       0
       0
       0
       0
       0
       0
       0
      39
      40
      11
      20

$ROI:

$PRESETS: Real Time|

++++++ END ++++++

The key data is the column of numbers (13 rows, data 0 to 20). As there are strings at the top and bottom (header/footer) I have had to use either:

++++++ dlmread ++++++

numfiles = 2; mydata = cell(11, numfiles);

for k = 1:numfiles

   myfilename = sprintf('Detector-2-%d.Spe', k);

    M = dlmread(myfilename, '\t', 'A13..A26'); 

    mydata{1,k} = M;

end

++++++ END ++++++

or

++++++ textscan ++++++

numfiles = 2; mydata = cell(11, numfiles);

for k = 1:numfiles myfilename = sprintf('Detector-2-%d.Spe', k); delimiter = ' '; if nargin<=2 startRow = 13; endRow = 26; end

formatSpec = '%f%*s%*s%*s%*s%[^\n\r]';

fileID = fopen(myfilename,'r');

dataArray = textscan(fileID, formatSpec, endRow(1)-startRow(1)+1, 'Delimiter', delimiter, 'MultipleDelimsAsOne', true, 'HeaderLines', startRow(1)-1, 'ReturnOnError', false);

    for block=2:length(startRow)
        frewind(fileID);
        dataArrayBlock = textscan(fileID, formatSpec, endRow(block)-startRow(block)+1, 'Delimiter', delimiter, 'MultipleDelimsAsOne', true, 'HeaderLines', startRow(block)-1, 'ReturnOnError', false);
        dataArray{k} = [dataArray{k};dataArrayBlock{k}];
    end

mydata{1,k} = [dataArray{1:end-1}];

fclose(fileID);

end

++++++ END ++++++

Both these methods work quite well for importing the required block of data - "load" and "import data" did not. Both are able to import multiple data files - BUT I cannot get the correct output.

I am looking for 1 column for each data input . The various things I've tried have resulted in:

• the last data input only [i.e. over written],
• [0;0;0;0;0;0;0;0;0;39;40;11;20] in R1C1 and similar in R1C2, but not filling every row [if you double click, you enter the individual array],
• something like "<11x2 double>" in the first row of each column,
• all the data written into column 1 [i.e. underneath each other]

I've seen 'dereferencing' being mentioned, but I'm not sure where it goes or what exactly it means. It also seems that using 'EVAL' is not ideal?

Apologies for the messy coding and lack of comments. I'm sure it's something simple I'm missing but I've been going round in circles for about a day now and I just can't see it anymore!

[Image Analyst]

Can you try to make it easy for us to help you by attaching at least two of your files with the paper clip icon?

[Michael]

Hi,

I've attached two "examples" of the files. Please note:

• It would not allow me to attached the original '.spe' file so I have attached it as a '.txt'.
• These are slightly longer than the example I pasted. I was trying to simplify my development by grabbing 13 lines from two files, but I will eventually need to get around 1023 lines from 1000+ files.

Thanks for your help

Answer 1

[Michael Haderlein]

I think the dlmread version is fine here. No need to make things more complicated than necessary. Please change the following:

mydata = cell(11, numfiles);

to

mydata = zeros(11, numfiles);

and

mydata{1,k} = M;

to

mydata(:,k)=M;

[Michael]

Hi,

Thanks for the help, however I receive the following message when I try this:

++++++

 Subscripted assignment dimension mismatch.

 Error in Multi2 (line 11)

    mydata(:,k)=M;

+++++++

The code I'm running is called 'Multi2'.

[Michael Haderlein]

Please set a breakpoint in the respective line and check the sizes of mydata and M. Does the size of M vary between the different files?

[Michael]

Sorry for the delay. M doesn't vary, but I needed to specify the length of the file for each input into the array. Seems to have solved it cheers.

Answer 2

[Michael]

Solved it. I needed to define the length of the rows for each import, the "M(1:end)". See below:

numfiles = 2;

    for k = 0:numfiles

        myfilename = sprintf('Detector-2-%03d.Spe', k);

        M = dlmread(myfilename, '\t', 'A13..A26');

        mydata(:,k+1) = M(1:end);

    end

Note, my files are actually referenced 001.Spe, 002.Spe etc - hence the "%03d" part of the 'sprintf' command, slightly different to my first posting.