Fail to process multiselect files by using readtable function....

3 visualizzazioni (ultimi 30 giorni)
Hi everyone. I just so confused about how to process and combine multifile datas by using readtable function. As you mention to my code below, iam using the readtable function because there is so many numheaderlines (consisted by random data type like numeric, string, etc) / heterogenous data which is should to be removed or skipped to get reach the header lines.
Format IAGA-2002 |%This headerlines should be removed or skipped
Source of Data Istituto Nazionale Geofisica e Vulcanologia |%This headerlines should be removed or skipped
Station Name Castello Tesino |%This headerlines should be removed or skipped
IAGA Code CTS |%This headerlines should be removed or skipped
Geodetic Latitude 46.047 |%This headerlines should be removed or skipped
Geodetic Longitude 11.649 |%This headerlines should be removed or skipped
Elevation 1175 |%This headerlines should be removed or skipped
Reported HDZF |%This headerlines should be removed or skipped
Sensor Orientation HDZ |%This headerlines should be removed or skipped
Digital Sampling 1 second |%This headerlines should be removed or skipped
Data Interval Type Filtered 1-minute (00:15-01:45) |%This headerlines should be removed or skipped
Data Type variation |%This headerlines should be removed or skipped
# MAGO 4.13.4 |%This headerlines should be removed or skipped
# SyS 1 |%This headerlines should be removed or skipped
# H, Z and F in nT, D in minutes of arc |%This headerlines should be removed or skipped
# F Digital Sampling 5 seconds |%This headerlines should be removed or skipped
# Approx H = 0 |%This headerlines should be removed or skipped
# Approx D = 0 |%This headerlines should be removed or skipped
# Approx Z = 0 |%This headerlines should be removed or skipped
# Vectorial Lemi 18 (text) - 115200 |%This headerlines should be removed or skipped
# Scalar GEM GSM 90 1 (048183.03 09) - 9600 |%This headerlines should be removed or skipped
# Scalar Tuning = 47 |%This headerlines should be removed or skipped
DATE TIME DOY CTSH CTSD CTSZ CTSF |%This is my header of data column i want to use
2021-07-05 00:00:00.000 186 22495.32 6.92 42278.17 47880.43
2021-07-05 00:01:00.000 186 22494.96 6.88 42278.31 47880.39
2021-07-05 00:02:00.000 186 22495.40 6.91 42278.35 47880.62
2021-07-05 00:03:00.000 186 22494.58 6.97 42278.27 47880.19
2021-07-05 00:04:00.000 186 22494.71 6.90 42278.35 47880.30
2021-07-05 00:05:00.000 186 22495.76 6.83 42278.47 47880.94
2021-07-05 00:06:00.000 186 22495.51 6.84 42278.35 47880.79
2021-07-05 00:07:00.000 186 22495.11 6.98 42278.14 47880.40
2021-07-05 00:08:00.000 186 22495.22 6.99 42278.19 47880.45
And by using a single file input (uigetfile) without multiselect function, iam trying to create a script to get data column of each of those header (DATE, TIME, DOY, CTSH, CTSD, CTSD, CTSZ, and CTSF) and it worked. However, i just wonder, how to manage a new script by using the readtable concept so that i can get the output file just like the single file script one. Iam trying to comprehend the document and write down the multiselect script, but it still doesnt success.... Here is my trial and error script :
[if file > 1
for k = 1 : nfiles
xx = string(full{k});
[~, namafile{k}] = fileparts(full{k});
nama = namafile{k};
judul2 = length(nama);
%Capitalize first character and lower case remaining characters.
nb2 = [upper(nama(1)), lower(nama(2:judul2))];
namas = namafile{:,1:k}
defm = contains(nb2, 'dmin');
varm = contains(nb2, 'vmin');
hord = contains(nb2, 'dhor');
stasiun = upper(extractBetween(nama,1,3));
sth = upper(extractBetween(nama,1,3))+"H";
std = upper(extractBetween(nama,1,3))+"D";
stz = upper(extractBetween(nama,1,3))+"Z";
stf = upper(extractBetween(nama,1,3))+"F";
stt = upper(extractBetween(nama,4,11));
opts = detectImportOptions(xx,'NumHeaderLines',22,'ReadVariableNames',true, 'Delimiter', ' ', 'Whitespace', ' ',...
'ConsecutiveDelimitersRule', 'join', 'EmptyLineRule', 'skip');
opts.SelectedVariableNames = ["DATE","TIME",sth,std,stz,stf];
opts.VariableNamesLine = 23; % Set variable names line
opts = setvartype(opts,{'DATE','TIME'},'string');
T2 = readtable(full{k}, opts);
DATE = T2{:,1};
TIME = T2{:,2};
DOH = T2{:,3};
DOD = T2{:,4};
DOZ = T2{:,5};
DOF = T2{:,6};
per = strcat(string(T2{1,1}));
tahun = str2double(extractBetween(per,1,4));
bulan = str2double(extractBetween(per,6,7));
hari = str2double(extractBetween(per,9,10));
D = datetime(tahun,bulan,tanggal,0,0,0,'TimeZone','Asia/Jakarta');
wak = DATE +" "+ TIME;
ntropy = string(wak);
DOH_combine(:,k) = DOH(:);
DOHn = reshape(DOH_combine,[],1);
DOFk = sqrt((DOH.^2)+(DOZ.^2));
DOFk_combine(:,k) = DOFk;
DOFk_combine(DOFk_combine == 0)=[];
DOFn = reshape(DOFk_combine,[],1);
sti = "Data Extraction of Horizontal and Total Magnetic Field IAGA Format (.Min)"+namas;
th = "Plot Of Horizontal Magnetic Field IAGA Format (.Min)";
tf = "Plot Of Horizontal Total Field IAGA Format (.Min)";
end
if judul2 == 15 || varm == 1
for n = 1: nfiles
[savefile, direct, ~] = uiputfile('*.txt', 'Save As', sti);
eval(['cd ''' direct ''';']);
fout=fopen(savefile,'w');
for i=1:length(TIME)
DOFS = sqrt((DOH.^2)+(DOZ.^2));
fprintf(fout,'%s %s %.2f %.2f\n', char(DATE(i)), char(TIME(i)), DOH(i), DOF(i));
end
fclose(fout);
end
figure('Name','Plot Of Horizontal Magnetic Field IAGA Format Vs Time Series')
plot(wak, DOHn, 'Color', 'r', 'LineWidth',1.1);
title(th, 'fontsize', 14, 'fontweight','bold');
xlabel('Deret Waktu (Time Series)','fontweight','bold','fontsize',10);
ylabel('Komponen H Magnet Data Lemi IAGA Format (nT)','fontweight','bold','fontsize',10);
legend('off');
figure('Name','Plot Of Total Magnetic Field IAGA Format Vs Time Series ')
plot(wak, DOFn, 'Color', 'b', 'LineWidth',1.1);
title(tf, 'fontsize', 14, 'fontweight','bold');
xlabel('Deret Waktu (Time Series)','fontweight','bold','fontsize',10);
ylabel('Komponen F Magnet Data Lemi IAGA Format (nT)','fontweight','bold','fontsize',10);
legend('off');
I just confused how to manage data table which re acquired from multiselect readtable onto another file format that can be processed. The error is almost always like this :
Error using teslemiiaga (line 232)
Right hand side of an assignment into a table must be another table or a cell array.
So... Would anyone lend me a hand to fix the error of the above script i made, or maybe just fix the entire concept of my script? and yes, i just wonder how it could be fixed by another function besides of readtable, maybe like readmatrix or readcell? Thank you everyone..... (i already attached my data).
Iam sorry for interrupting your days /.\ /.\ /.\

Risposta accettata

Walter Roberson
Walter Roberson il 15 Ago 2021
Modificato: Walter Roberson il 16 Ago 2021
When you have code that processe only one file from a uigetfile(), then it is of the form
[filename, directory] = uigetfile(appropriate parameters);
if isempty(filename)
return; %user cancel
end
fullname = fullfile(directory, filename);
Now do code processing the one file recorded in fullname
When you have multiselect turned on, then instead it is of the form
[filenames, directory] = uigetfile(appropriate parameters);
if isempty(filenames)
return; %user cancel
end
filenames = cellstr(filenames); %takes care of case only one file was selected
fullnames = fullfile(directory, filenames);
all_tables = {};
for K = 1 : length(fullnames)
fullname = fullnames{K};
Now do code processing the one file recorded in fullname
all_tables{K} = output table;
end
combined_table = vertcat(all_tables{:});
save combined_table in appropriate place
  6 Commenti
Tyann Hardyn
Tyann Hardyn il 16 Ago 2021
Modificato: Tyann Hardyn il 16 Ago 2021
The output of your code above is good, Sir. I have been try it....
But it only read 1 file (the last picked file by multiselect) :
2021-07-05 00:00:00.000,22495.32,42278.17,47880.43
2021-07-05 00:01:00.000,22494.96,42278.31,47880.39
2021-07-05 00:02:00.000,22495.4,42278.35,47880.62
2021-07-05 00:03:00.000,22494.58,42278.27,47880.19
2021-07-05 00:04:00.000,22494.71,42278.35,47880.3
2021-07-05 00:05:00.000,22495.76,42278.47,47880.94
2021-07-05 00:06:00.000,22495.51,42278.35,47880.79
2021-07-05 00:07:00.000,22495.11,42278.14,47880.4
2021-07-05 00:08:00.000,22495.22,42278.19,47880.45
2021-07-05 00:09:00.000,22495.39,42278.36,47880.68
2021-07-05 00:10:00.000,22495.64,42278.49,47880.92
2021-07-05 00:11:00.000,22495.6,42278.48,47880.92
2021-07-05 00:12:00.000,22496.23,42278.33,47881.1
in this case, the datas selected by uigetfile multiselect is 2021-07-03, 2021-07-04, and 2021-07-05 file, but it somehow only read the last file (2021-07-05 file)....
Tyann Hardyn
Tyann Hardyn il 16 Ago 2021
Modificato: Tyann Hardyn il 16 Ago 2021
Ohhh, alright, Sir... I am just misplacing the variable fullnames to be putted on the outside of the "for loop". When i placed the fullnames to be inside of the "for loop", the output is excellent :
2021-07-01 23:52:00.000 22492.16 42281.89 47883.04
2021-07-01 23:53:00.000 22491.89 42281.66 47882.71
2021-07-01 23:54:00.000 22491.93 42281.57 47882.66
2021-07-01 23:55:00.000 22492.14 42281.50 47882.71
2021-07-01 23:56:00.000 22492.15 42281.58 47882.82
2021-07-01 23:57:00.000 22492.20 42281.31 47882.64
2021-07-01 23:58:00.000 22492.48 42281.35 47882.77
2021-07-01 23:59:00.000 22492.94 42281.72 47883.31%It finally changed from 2021-07-01
2021-07-02 00:00:00.000 22492.80 42281.72 47883.26% To 2021-07-02 data files
2021-07-02 00:01:00.000 22492.89 42282.24 47883.72
2021-07-02 00:02:00.000 22492.06 42281.87 47883.03
2021-07-02 00:03:00.000 22491.41 42281.61 47882.49
2021-07-02 00:04:00.000 22491.17 42281.82 47882.61
2021-07-02 00:05:00.000 22491.07 42281.86 47882.57
2021-07-02 00:06:00.000 22491.43 42281.86 47882.67
2021-07-02 00:07:00.000 22491.57 42281.91 47882.79
2021-07-02 00:08:00.000 22491.64 42281.80 47882.75
2021-07-02 00:09:00.000 22492.02 42281.85 47882.88
Finally, its over.... Well done, i said thank you very much, Sir ^^

Accedi per commentare.

Più risposte (1)

Simon Chan
Simon Chan il 15 Ago 2021
Not quite understand your issue, but I guess you are not able to get the figure when the code is reading multiple files.
So I put the code which generate figures inside the loop. If you would like to save different files in similar way, you may also put the code inside the loop.
for k = 1 : nfiles
xx = string(full{k});
[~, namafile{k}] = fileparts(full{k});
nama = namafile{k};
judul2 = length(nama);
%Capitalize first character and lower case remaining characters.
nb2 = [upper(nama(1)), lower(nama(2:judul2))];
namas = namafile{:,1:k};
defm = contains(nb2, 'dmin');
varm = contains(nb2, 'vmin');
hord = contains(nb2, 'dhor');
stasiun = upper(extractBetween(nama,1,3));
sth = upper(extractBetween(nama,1,3))+"H";
std = upper(extractBetween(nama,1,3))+"D";
stz = upper(extractBetween(nama,1,3))+"Z";
stf = upper(extractBetween(nama,1,3))+"F";
stt = upper(extractBetween(nama,4,11));
opts = detectImportOptions(xx,'NumHeaderLines',22,'ReadVariableNames',true, 'Delimiter', ' ', 'Whitespace', ' ',...
'ConsecutiveDelimitersRule', 'join', 'EmptyLineRule', 'skip');
opts.SelectedVariableNames = ["DATE","TIME",sth,std,stz,stf];
opts.VariableNamesLine = 23; % Set variable names line
opts = setvartype(opts,{'DATE','TIME'},'string');
T2 = readtable(full{k}, opts);
DATE = T2{:,1};
TIME = T2{:,2};
DOH = T2{:,3};
DOD = T2{:,4};
DOZ = T2{:,5};
DOF = T2{:,6};
per = strcat(string(T2{1,1}));
tahun = str2double(extractBetween(per,1,4));
bulan = str2double(extractBetween(per,6,7));
hari = str2double(extractBetween(per,9,10));
D = datetime(tahun,bulan,hari,0,0,0,'TimeZone','Asia/Jakarta'); % Modify the avriable to 'hari'
wak = DATE +" "+ TIME;
ntropy = string(wak);
DOH_combine(:,k) = DOH(:);
DOHn = reshape(DOH_combine,[],1);
DOFk = sqrt((DOH.^2)+(DOZ.^2));
DOFk_combine(:,k) = DOFk;
DOFk_combine(DOFk_combine == 0)=[];
DOFn = reshape(DOFk_combine,[],1);
sti = "Data Extraction of Horizontal and Total Magnetic Field IAGA Format (.Min)"+namas;
th = "Plot Of Horizontal Magnetic Field IAGA Format (.Min)";
tf = "Plot Of Horizontal Total Field IAGA Format (.Min)";
%
% Put the figure generation iside the loop
%
figure(k)
subplot(2,1,1)
%figure('Name','Plot Of Horizontal Magnetic Field IAGA Format Vs Time Series')
plot(datetime(wak), DOH(:), 'Color', 'r', 'LineWidth',1.1); % Convert to datetime and select one column of data
title(th, 'fontsize', 14, 'fontweight','bold');
xlabel('Deret Waktu (Time Series)','fontweight','bold','fontsize',10);
ylabel('Komponen H Magnet Data Lemi IAGA Format (nT)','fontweight','bold','fontsize',10);
legend('off');
subplot(2,1,2)
%figure('Name','Plot Of Total Magnetic Field IAGA Format Vs Time Series ')
plot(datetime(wak), DOFk_combine(:,k), 'Color', 'b', 'LineWidth',1.1); % Convert to datetime and select one column of data
title(tf, 'fontsize', 14, 'fontweight','bold');
xlabel('Deret Waktu (Time Series)','fontweight','bold','fontsize',10);
ylabel('Komponen F Magnet Data Lemi IAGA Format (nT)','fontweight','bold','fontsize',10);
legend('off');
end
  3 Commenti
Simon Chan
Simon Chan il 16 Ago 2021
As i mentioned earlier, you can put the save file section inside the loop as well.
Please refer to the following as a reference. You can actually rearrange your code to achieve your target.
for k = 1 : nfiles
xx = string(full{k});
[~, namafile{k}] = fileparts(full{k});
nama = namafile{k};
judul2 = length(nama);
%Capitalize first character and lower case remaining characters.
nb2 = [upper(nama(1)), lower(nama(2:judul2))];
%namas = namafile{:,1:k}; % No need this line
defm = contains(nb2, 'dmin');
varm = contains(nb2, 'vmin');
hord = contains(nb2, 'dhor');
stasiun = upper(extractBetween(nama,1,3));
sth = upper(extractBetween(nama,1,3))+"H";
std = upper(extractBetween(nama,1,3))+"D";
stz = upper(extractBetween(nama,1,3))+"Z";
stf = upper(extractBetween(nama,1,3))+"F";
stt = upper(extractBetween(nama,4,11));
opts = detectImportOptions(xx,'NumHeaderLines',22,'ReadVariableNames',true, 'Delimiter', ' ', 'Whitespace', ' ',...
'ConsecutiveDelimitersRule', 'join', 'EmptyLineRule', 'skip');
opts.SelectedVariableNames = ["DATE","TIME",sth,std,stz,stf];
opts.VariableNamesLine = 23; % Set variable names line
opts = setvartype(opts,{'DATE','TIME'},'string');
T2 = readtable(full{k}, opts);
DATE = T2{:,1};
TIME = T2{:,2};
DOH = T2{:,3};
DOD = T2{:,4};
DOZ = T2{:,5};
DOF = T2{:,6};
% Rearrange the following lines before you write the txt file
sti = "Data Extraction of Horizontal and Total Magnetic Field IAGA Format (.Min)"+nama;
th = "Plot Of Horizontal Magnetic Field IAGA Format (.Min)";
tf = "Plot Of Horizontal Total Field IAGA Format (.Min)";
if judul2 == 15 || varm == 1
[savefile, direct, ~] = uiputfile('*.txt', 'Save As', sti);
eval(['cd ''' direct ''';']);
fout=fopen(savefile,'w');
for i=1:length(TIME)
fprintf(fout,'%s %s %.2f %.2f %.2f %.2f\n', DATE(i), TIME(i), DOH(i), DOD(i), DOZ(i), DOF(i));
end
fclose(fout);
end
end

Accedi per commentare.

Categorie

Scopri di più su Tables in Help Center e File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by