How can I increase timeout when using readtable to get csv data from url?

16 visualizzazioni (ultimi 30 giorni)
I want to use readtable function to read time series csv file from internet via an url. It works for smaller datasets but when I try to download longer datasets I get an error that the server is not responding and that it might be wise to increase the timeout from 5s to a larger number. How can I do it? I can do it with webread function but not with readtable.
Example that works (short dataset):
url = 'https://dataset.api.hub.geosphere.at/v1/station/historical/klima-v1-1h?parameters=RSX&start=1976-01-01T00%3A00%3A00.000Z&end=1976-12-31T23%3A00%3A00.000Z&station_ids=710&output_format=csv&filename=RH_710';
% Options for reading
opts = delimitedTextImportOptions("NumVariables",3,'Delimiter',{','});
opts.Encoding = 'latin1';
opts.VariableNames = ["dateS", "stationID","dataV"];
opts.VariableTypes = ["string","double","double"];
opts.ExtraColumnsRule = "ignore";
opts.EmptyLineRule = "read";
opts.ConsecutiveDelimitersRule = "join";
% Read data using the identified decimal separator
opts.DataLines = [2,Inf];
opts.VariableTypes = ["string","double","double"];
dataTb = readtable(url,opts);
Example that does not work (long dataset):
url = 'https://dataset.api.hub.geosphere.at/v1/station/historical/klima-v1-1h?parameters=RSX&start=1976-01-01T00%3A00%3A00.000Z&end=2020-12-31T23%3A00%3A00.000Z&station_ids=710&output_format=csv&filename=RH_710';
% Options for reading
opts = delimitedTextImportOptions("NumVariables",3,'Delimiter',{','});
opts.Encoding = 'latin1';
opts.VariableNames = ["dateS", "stationID","dataV"];
opts.VariableTypes = ["string","double","double"];
opts.ExtraColumnsRule = "ignore";
opts.EmptyLineRule = "read";
opts.ConsecutiveDelimitersRule = "join";
% Read data using the identified decimal separator
opts.DataLines = [2,Inf];
opts.VariableTypes = ["string","double","double"];
dataTb = readtable(url,opts);
Thank you

Risposta accettata

Jasper Gerritsen
Jasper Gerritsen il 12 Dic 2023
You can have webread use your readtable options and increase the Timeout variable to e.g. 20 seconds as follows:
myReadTable = @(x) readtable(x, opts);
webopts = weboptions('ContentReader', myReadTable, 'Timeout', 20);
dataTb = webread(url, webopts);
Kind regards

Più risposte (0)

Prodotti


Release

R2023b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by