HDF5 support, My work group encounters rare failures when attempting to update collections of HDF5 files. The usual symptom is that a previously valid file becomes partially broken, such that it becomes impossible to add new attributes to one particular data variable. The file otherwise reads and writes normally. Attributes can be added to other variables, and data can still be written to the suspect variable, including extending the unlimited dimension.
We normally operate on these files with the Netcdf-4 interface to HDF5, but I think I isolated the failure to pure HDF5 functionality. You can examine one of these broken files here (9 Mb): ftp://ftp.cdc.noaa.gov/Public/dallured/hdf5/precip.V1.0.2016.nc Diagnostics such as h5check, h5debug, h5dump, and h5stat all report this file as valid. h5edit demonstrates the problem: > ls -go precip.V1.0.2016.nc -rw-r--r-- 1 9024497 Nov 9 15:02 precip.V1.0.2016.nc > h5edit -c 'CREATE /precip/att12 { DATATYPE H5T_IEEE_F32LE DATASPACE SIMPLE ( 2 ) DATA { 777.0, 778.0 } } ;' precip.V1.0.2016.nc failed to create attribute (null) CREATE Attribute command failed ... and the data file is unchanged. It does not matter whether the added attribute is scalar or array. We are currently using the HDF5 library version 1.8.16 in combination with Netcdf to read and write these data sets. The version of h5edit above is 1.3.1. The broken file behaves the same way on both Mac and Linux platforms, and with several Netcdf-based attribute writers in addition to h5edit. We are investigating, but we have not yet found a good way to isolate the exact program or event that creates this broken file condition. We are suspicious that NFS connected file servers may contribute to the problem, but no strong evidence yet to back this up. Today I have these questions: 1. Can someone identify exactly what is wrong with the above sample file? 2. Can h5check or another utility be updated to diagnose this condition? 3. Has anyone else experienced partially broken files with this symptom? Thank you for any insights. --Dave A. NOAA/OAR/ESRL/PSD/CIRES
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
