Select rows with textscan

Hi

I use the below text file:

ETE 04/01/2010 10170000 18.54 430 Big Cap

ABC 04/01/2010 10190000 18.34 200 Big Cap

YYY 04/01/2010 10200000 18.34 100 Big Cap

How can I use textscan to import rows with respect to values of column 1? So to import only the rows of 'ETE' for instance and the file to like :

ETE 04/01/2010 10170000 18.54 430 Big Cap

Panos

Answer 1

[Oleg Komarov]

1st approach: bulk import - select

% Import all
fid  = fopen('C:\Users\Oleg\Desktop\test.txt');
data = textscan(fid, '%s%s%f%f%f%s%s','CollectOutput',1);
fid  = fclose(fid);

% Select only line that begin with 'ETE'
idx  = strcmp(data{1}(:,1),'ETE');
data = cellfun(@(x) x(idx,:),data,'un',0);

2nd approach: line by line conditional import

fid  = fopen('C:\Users\Oleg\Desktop\test.txt');
data = [];
while ~feof(fid)
    tmp = textscan(fid,'%s%s%f%f%f%s%s',1);
    if strcmp(tmp{1},'ETE')
        data = [data; tmp];
    end
end
fid = fclose(fid);

PROS/CONS:

The 1st approach is faster and achieves a discrete memory management for the output without further manipulation of the data BUT it can go out of memory if the file you're importing is huge.

The 2nd approach is slower since there's no preallocation of the output array. From the memory point of view it is inefficient in the way data is stored but at the same time it is efficient since you select only the rows you need. With this approach you can manipulate data inside the IF statement to optimize memory storage to the extreme (ex: instead of big cap you could store numbers to indicate the feature).

[Pap]

Many Thanks Oleg