Downloading Data off a URL and specifying what portion of the URL to save.

I am trying to take Data off 'http://tidesandcurrents.noaa.gov/harcon.html?id=9410170' and upload it to arrays. The problem that I am running into is whenever I websave or webread the URL the entire code comes up. I want to only download and save the Constituent #, Name, Amplitude, Phase, Speed, and Description portion of the URL and have MatLab display those variables separately as arrays, to be used later on.

[Kleber Zuza Nobrega]

Try this: pattern='<dt>Constituent #</dt>\s*<dd>(?<Cons>.*?(</dd>)?)\s*<dt>Name</dt>\s*<dd>(?<Nam>.*?)</dd>\s*<dt>Amplitude</dt>\s*<dd>(?<Amp>.*?)</dd>\s*<dt>Phase</dt>\s*<dd>(?<Pha>.*?)</dd>\s*<dt>Speed</dt>\s*<dd>(?<Spe>.*?)</dd>\s*<dt>Description</dt>\s*<dd>(?<Des>.*?)</dd>'

res=regexp(code,pattern,'names')

res.Cons %to have access to the text!! and so on res.Nam res.Amp res.Pha res.Spe res.Des

[Kleber Zuza Nobrega]

code=webread('http://tidesandcurrents.noaa.gov/harcon.html?id=9410170')

pattern='<td>(.*?)</td><td>(.*?)</td><td>(.*?)</td><td>(.*?)</td><td>(.*?)</td><td>(.*?)</td>'

res=regexp(code,pattern,'tokens')

Sorry... Misunderstood. Try this!!

Answer 1

[Tim Jackman]

Webread is designed for RESTful web services, so using it here isn't going to be ideal. You could parse the html code and grab information from the table between the thead and tbody tags, but I'd recommend giving one of these from the File Exchange a try:

http://www.mathworks.com/matlabcentral/fileexchange/22465-get-html-table-data-into-matlab

http://www.mathworks.com/matlabcentral/fileexchange/29642-get-html-table-data-into-matlab-via-urlread-and-without-builtin-browser

Answer 2

[Robert Snoeberger]

As Tim Jackman correctly pointed out, webread is designed for RESTful web services. Consuming an API provided by a web service will be much easier than screen scraping the data.

The tidesandcurrents.noaa.gov site does provide an API [1], which I found with a google search. The following is an example of using webread to read the data from the web service. The example is based on the "Sample URL requests and responses" section of the API documentation.

>> opts = weboptions('ContentType', 'json');
>> url = 'http://tidesandcurrents.noaa.gov/api/datagetter';
>> result = webread(url, opts, 'product', 'water_level', 'begin_date', '20130101', 'end_date', '20130101', 'station', '8454000', 'time_zone', 'gmt', 'units', 'metric', 'datum', 'mllw', 'format', 'json')

result = 

      metadata: [1x1 struct]
          data: [240x1 struct]

>> result.data(1)

ans = 

      t: '2013-01-01 00:00'
      v: '0.549'
      s: '0.004'
      f: '0,0,0,0'
      q: 'v'

>>

[1] https://tidesandcurrents.noaa.gov/api/