Binning data into 1 hour average in cell array?

Hi,

I have large number of .txt files; each file corresponds to day of year. Data in each file is like say (1348x4, not same always); year, day,ut, rainfall. I know binning data into 1 hour average for a single file. I have problem with extending the code to large number of files using cell. My code as follows (I am using Matlab2015a).

delimiterIn = ' ';
headerlinesIn =48;%%something something written on all files for 1-48 rows
dinfo = dir('*.txt');
for k=1:length(dinfo)
  thisfilename=dinfo(k).name;  
  myStructure{k}= importdata(thisfilename,delimiterIn,headerlinesIn);
end
for k=1:length(dinfo)
T{k}=myStructure{k};
end
for k=1:length(dinfo);
    a{k}=T{k}.data;
end
data=a{1};%single file
%%%FOR MULTIPLE FILES
%%%data=[a{1}' a{2}' a{3}' a{4}' a{5}' a{6}' ..]';
%%BINNING FOR A SINGLE FILE (IT WORKS FINE)
% % ab=round(max(ut(end)));%ut=data(:,3)
% % [N,edges,bins] = histcounts(ut,ab);
% % for n=1:ab
% %   bin_means(:,n) = nanmean(rainfall(bins==n,:))';%%railfall=data(:,end)
% % end
%%%BINNING FOR ALL FILES (DOESNOT WORK)
for k=1:length(dinfo); 
 ax{k}=size(a{k},1); %row size of each file
 ab{k}=round(max(data(1:ax{k},3)));%%for each file, 3rd column is UT
 [N{k},edges{k},bins{k}] = histcounts(data(1:ax{k},3),ab{k});
end
for k=1:length(dinfo);
    for n=1:ab{k}
  bin_means{k}(:,n) = nanmean(data(bins{k}==n,end))';%rainfall is last column of all files.
    end
end

What I think (I am noy sure though) that might be causing unexpected results is the loop; for n=1:ab{k} (or something that starts from 1,ex. ab{k} or data(1:ax{k},3) etc ), because it starts from 1 everytime. Need help.

[Image Analyst]

Attach one of your data files with the paperclip icon after you read this:

TUTORIAL: How to ask a question (on Answers) and get a fast answer

[Madan Kumar]

Oops. I am sorry, I thought I have aready attached. Here is the data.

Answer 1

[Karim]

Hello,

If you know how to proces the data for a single file, you can use the same logic in a loop. At the begining of the loop you extract the data you need, in the loop you treat it as if it is a single file and at the end of the loop you can save the data you need into the cell.

Note that you do not need the multiple for loops, you can do all this in a single loop.

Also, it's best to always allocate the cell array.

See below for the code, hope it helps.

delimiterIn = ' ';
headerlinesIn = 48;
dinfo = dir('*.txt');

numFiles = length(dinfo);

% allocate the cells
data = cell(numFiles,1);
bin_means = cell(numFiles,1);

for k = 1:numFiles
    
    % read the current file
    currData = importdata(dinfo(k).name, delimiterIn, headerlinesIn);
    data{k} = currData.data;
    
    % extract data for currect file
    ut = data{k}(:,3);
    rainfall = data{k}(:,4);
    
    % process the current file
    ab = round(max(ut));
    [N,edges,bins] = histcounts(ut,ab);
    bin_means_tmp = zeros(1,ab);
    for n = 1:ab
        bin_means_tmp(:,n) = nanmean( rainfall(bins==n,:) )';
    end
    
    % save the 'bin' data in a new cell
    bin_means{k} = bin_means_tmp;
    
end

% display the data
bin_means{1}

ans = 1×24

2.3896 0.9336 1.6308 0.3671 1.5527 0.9333 0.7050 1.3149 0.4243 1.3023 1.3473 0.6649 1.5455 0.5846 1.2249 1.2973 0.4566 1.1825 0.5678 1.0098 1.1596 0.3322 1.0665 0.1995

bin_means{2}

ans = 1×24

2.3896 0.9336 1.6308 0.3671 1.5527 0.9333 0.7050 1.3149 0.4243 1.3023 1.3473 0.6649 1.5455 0.5846 1.2249 1.2973 0.4566 1.1825 0.5678 1.0098 1.1596 0.3322 1.0665 0.1995

[Madan Kumar]

Thank you so much.