Azzera filtri
Azzera filtri

Automatic Download from a Url when the file of the date is inputted by the user

14 visualizzazioni (ultimi 30 giorni)
I have written a code to download a specific data from cosmic satellite website, the code is as follows
url_https='https://data.cosmic.ucar.edu';
dataUrl=strcat(url_https,"/gnss-ro/cosmic2/nrt/level1b/2022/004/podTc2_nrt_2022_004.tar.gz");
dataFile="podTc2_nrt_2022_005.tar.gz";
FileFullPath=websave(dataFile,dataUrl);
Files=untar("F:\podtc2_data\podTc2_nrt_2022_005.tar.gz",'podTc2_nrt_2022_005');
Now this code only works for one day of the data that is provided by the data portal. is there a way such that I could create a loop that modifies a string of characters corresponent to the desired date that I am looking to work with, then the web function can read the string automatically created by MATLAB.
I was wondering if a web crawler or something similar has been done on matlab?
Overall, I have currently created a code that download a specific date of data, I am looking to code a set of loops where if I enter a specific date the code would automatically crawl the web and download the specific file that I am looking for.
I have been stuck on this section for a quite a long time, hopefully you guys could give me some advice and ideas on this matter.
  2 Commenti
Jan
Jan il 10 Ago 2022
Which part of the file name do you want to modify and how? The more details you mention, the smaller is the chance of a wrong guess.
Tianchu Lu
Tianchu Lu il 10 Ago 2022
Thank you very much for your answer benearth, so the part that should be modified would be the part of _005, as 005 means 5th of Jan 2022 and hence 001 is 1st of Jan and so on and so forth. So the pusedocode that I had in mind was that the user( myself) would enter a specific date that the I would like to investigate into, and then a loop where matlab would urlread (https://data.cosmic.ucar.edu/gnss-ro/cosmic2/nrt/level1b/2022/) and identify the date that the user want and would then download the specific zipped file of that date. so a loop where it would urlread and identify the date inputted and download the file correspond to the inputted information. But I will defo have a play with the code that you submitted below and let you know how I got on.

Accedi per commentare.

Risposta accettata

Jan
Jan il 10 Ago 2022
This can be implemented with sprintf() or compose() easily.
url_https='https://data.cosmic.ucar.edu';
for year = 2020:2022 % ??? Bold guess
for day = 1:365 % ??? Bold guess
dataUrl = sprintf('%s/gnss-ro/cosmic2/nrt/level1b/%d/%03d/podTc2_nrt_%d_%03d.tar.gz', ...
url_https, year, day, year, day);
dataFile = sprintf('podTc2_nrt_%d_%03d.tar.gz', year, day);
FileFullPath = websave(dataFile, dataUrl);
end
end
  3 Commenti
Débora Rodrigues
Débora Rodrigues il 19 Set 2022
Hello!
Sorry to bother, but i have a similar problem. I am trying to download a .grib preciptation data, from this website Precipitation. I saw this discussion and tried to follow.
However i have an error:
Error using websave (line 95)
The '' protocol specified in URL, 'MERGE_CPTEC_20100101.grib2', is not supported. Specify the URL with the protocol 'http://' or 'https://'.
Error in download_precipitacao (line 19)
data = websave(data_url,data_file);
my code is:
url = 'http://ftp1.cptec.inpe.br/modelos/tempo/MERGE/GPM/DAILY';
for ano= 2010:2021
for mes= 1:12
for dia=1:30
data_url = sprintf('%s/%3d/%02d/MERGE_CPTEC_%d%02d%02d.grib2', ...
url, ano, mes, ano, mes, dia);
data_file= sprintf('MERGE_CPTEC_%d%02d%02d.grib2', ano, mes, dia);
data = websave(data_url,data_file);
end
end
end
if someone can help me, i would be very grateful, thanks!
Débora Rodrigues
Débora Rodrigues il 20 Set 2022
Modificato: Débora Rodrigues il 20 Set 2022
Hello! So i solved my problem!
I changed the function websave for urlwrite, and it worked! I am living it in here in case someone with similar problens appear haha.
url = 'http://ftp1.cptec.inpe.br/modelos/tempo/MERGE/GPM/DAILY';
for ano= 2010:2021
for mes= 1:12
for dia=1:30
data_url = sprintf('%s/%3d/%02d/MERGE_CPTEC_%d%02d%02d.grib2', ...
url, ano, mes, ano, mes, dia);
data_file= sprintf('MERGE_CPTEC_%d%02d%02d.grib2', ano, mes, dia);
%data = websave(data_url,data_file);
a= urlwrite(data_url,data_file);
end
end
end

Accedi per commentare.

Più risposte (0)

Categorie

Scopri di più su Downloads in Help Center e File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by