How to read netcdf files containing string arrays?

I have a netCDF file which consists of a float array and a string array as verified by ncdump. In Matlab, I am able to load the float array, but I fail in loading the string array.

In particular, ncread produces an error:

 ncread('string_test.nc','StringArray')
 Error using netcdf.getVar (line 130) 
 12 is not a recognized netCDF datatype.

"12" is the code for NC_STRING and it seems like Matlab can not handle that.

The .nc file I'm trying to load can be downloaded from http://piotr.majdak.com/temp/netcdf/ where also more description is available.

I'm using Matlab 2011b - any ideas what I can do?

Answer 1

[per isakson]

I use R2012a.

ncdisp says: Datatype: UNSUPPORTED DATATYPE. See below. Why not accept that? What is the source of string_test.nc?

HDFView 2.9 reads string_test.nc as if it was an HDF5-file. So does Matlab's h5disp and h5read.

    >> ncdisp( 'string_test.nc')
    Source:
               h:\m\cssm\string_test.nc
    Format:
               netcdf4
    Dimensions:
               test = 4
    Variables:
        FloatArray 
               Size:       4x1
               Dimensions: test
               Datatype:   single
        StringArray
               Size:       4x1
               Dimensions: test
               Datatype:   UNSUPPORTED DATATYPE

    >> str = h5read( 'h:\m\cssm\string_test.nc', '/StringArray' )
    str = 
        'NH10'
        'CI03'
        'HI17'
        'NH11'
    >> 

    >> h5disp( 'h:\m\cssm\string_test.nc', '/StringArray' )
    HDF5 string_test.nc 
    Dataset 'StringArray' 
        Size:  4
        MaxSize:  4
        Datatype:   H5T_STRING
            String Length: variable
            Padding: H5T_STR_NULLTERM
            Character Set: H5T_CSET_ASCII
            Character Type: H5T_C_S1
        ChunkSize:  []
        Filters:  none
        Attributes:
            'DIMENSION_LIST':  H5T_VLEN

[Piotr Majdak]

Thanks for your response. I try to answer your questions, maybe some ideas will come up.

@ncdisp says: Datatype: UNSUPPORTED DATATYPE. that message is equivalent to the error I receive from ncread. I think that ncdisp uses the same implementation of netcdf as ncread.

@Why not accept that? netCDF supports strings in netCDF files. Since Matlab supports netCDF, it's hard to believe that Matlab's netcdf support is limited. And we really need string arrays support in our netcdf files.

@What is the source of string_test.nc? It is a simple convention file using string arrays. The CDL definition (srting_test.txt) can be found at http://piotr.majdak.com/temp/netcdf/. The netCDF file can be created by ncgen -b -k3 string_test.txt

@HDFView 2.9 reads string_test.nc as if it was an HDF5-file. HDF5 must be able to read the netcdf4 file because netcdf4 is based on HDF5.

@So does Matlab's h5disp and h5read: Glad to see that the HDF5-support in Matlab works as expected.

I see that there could be a work around for reading the problematic files by reading the problematic variables with HDF5. However, I also need a write support...

[per isakson]

I think:

• strings can be stored in more than one way (format).
• Matlab is often slow to adapt to new versions of supported software.
• When there is a high and a low-level set of functions in Matlab some functionality is typically missing in the high-level functions. I easily believe that Matlab's high-level support is limited. However, I often find it difficult to spot the critical statements (supported or not supported) in the documentation.
• in this context one should avoid the word "must"

My guess:

• ncread (R2012a) doesn't support variable length strings

[Piotr Majdak]

I'm slowly coming to a similar conclusion. Thanks for the help.

Answer 2

[Peter Kuma]

Just to let others know, I have submitted a new bug report (since I could not find the one submitted by Erik). The answer was:

As you correctly pointed out, this is a current limitation of the
netCDF implementation. It is being actively investigated for
enhacement. 

The workaround is to utilize the fact that netCDF4 formats are a
subset of the HDF5 format and we can utilize the HDF5 functionality
to read the data

Workaround:

Instead of  ncdisp, ncread   utilize   h5disp, h5read

I hope the above workaround solves your issue. If you have any
further queries, please reach out to me.

[Anna Weeks]

The ability to write a mapped array section of values into a netCDF variable (i.e the 'NF_PUT_VARM_ type' functionality) would be highly desirable.

Answer 3

[Maxime Hervo]

Is there any update for this issue ? I still have the same problem with MATLAB R2016b !

Answer 4

[liu jibao]

Is there any update for this issue ? I still have the same problem with MATLAB R2017b !

[Erik Quaeghebeur]

It seems nobody bothered to report it as a bug, so it is not surprising nothing has changed.

I filed a bug report with Case Number 02978436.

[Natalie Perlin]

Thank you for filing the report, I wonder if anything has been done since then!

I have the same problem with Matlab not being able to read string variables in R2017a, R2018b.

[chidi]

I've tried using the h5read and hdf5read function but I'm not getting desired results

For ref. VX= h5read('d20220602.nc','.............../current velocity down')

Output comes out as [15532×1 single] . . .[15532×1 single]

The expected output are water current data and its 56x1 array

How can I resolve this issue

Thanks