Hi all (new to the list),
I am currently experiencing/investigating an issue where our usage of
HDF5 with compression is leading to some defects reported by valgrind
(apologies for not having debug symbols for line numbers). These defects
are also reported when I run valgrind on
http://www.hdfgroup.org/ftp/HDF5/examples/examples-by-api/hdf5-examples/1_8/FORTRAN/H5D/h5ex_d_gzip.f90
The valgrind output I get from the above example is:
==54559== Conditional jump or move depends on uninitialised value(s)
==54559== at 0x63664E0: inflateReset2 (in
/lib/x86_64-linux-gnu/libz.so.1.2.3.4)
==54559== by 0x63665D8: inflateInit2_ (in
/lib/x86_64-linux-gnu/libz.so.1.2.3.4)
==54559== by 0x5989E13: H5Z_filter_deflate (in
/opt/hdf5/1.8.11/gcc_4.6.3/lib/libhdf5.so.8.0.0)
==54559== by 0x59891A0: H5Z_pipeline (in
/opt/hdf5/1.8.11/gcc_4.6.3/lib/libhdf5.so.8.0.0)
==54559== by 0x57C96A1: H5D__chunk_lock (in
/opt/hdf5/1.8.11/gcc_4.6.3/lib/libhdf5.so.8.0.0)
==54559== by 0x57CA5D0: H5D__chunk_read (in
/opt/hdf5/1.8.11/gcc_4.6.3/lib/libhdf5.so.8.0.0)
==54559== by 0x57DA482: H5D__read (in
/opt/hdf5/1.8.11/gcc_4.6.3/lib/libhdf5.so.8.0.0)
==54559== by 0x57DB7B3: H5Dread (in
/opt/hdf5/1.8.11/gcc_4.6.3/lib/libhdf5.so.8.0.0)
==54559== by 0x4E5B143: h5dread_c_ (in
/opt/hdf5/1.8.11/gcc_4.6.3/lib/libhdf5_fortran.so.8.0.0)
==54559== by 0x4E69593: __h5d_provisional_MOD_h5dread_integer_2 (in
/opt/hdf5/1.8.11/gcc_4.6.3/lib/libhdf5_fortran.so.8.0.0)
==54559== by 0x40198E: MAIN__ (h5ex_d_gzip.f90:134)
==54559== by 0x401B81: main (h5ex_d_gzip.f90:16)
In my situation, when I am performing parallel reads I get anomalous
behavior, where sometimes the executable works, and sometimes it does
not. When it does not work I get the following diagnostic messages from
HDF5.
HDF5-DIAG: Error detected in HDF5 (1.8.13) MPI-process 4223:
#000: H5Dio.c line 173 in H5Dread(): can't read data
major: Dataset
minor: Read failed
#001: H5Dio.c line 545 in H5D__read(): can't read data
major: Dataset
minor: Read failed
#002: H5Dchunk.c line 1861 in H5D__chunk_read(): unable to read raw
data chunk
major: Low-level I/O
minor: Read failed
#003: H5Dchunk.c line 2891 in H5D__chunk_lock(): data pipeline read
failed
major: Data filters
minor: Filter operation failed
#004: H5Z.c line 1383 in H5Z_pipeline(): filter returned failure
during read
major: Data filters
minor: Read failed
#005: H5Zdeflate.c line 136 in H5Z_filter_deflate(): memory
allocation failed for deflate uncompression
major: Resource unavailable
minor: No space available for allocation
Which has a similar call stack, and I wonder if it's trying to allocate
using an undefined value. My gut tells me these may be related.
Is the valgrind issue I am reporting a known issue? Has anyone else run
into this?
Can anyone provide some advice on how might continue debugging this?
Any thoughts on whether this is an issue with HDF5, zlib or some
combination?
In my use case, it is extremely desirable to use compression as I am
dealing with files that are 2.2GB compressed (4.5GB uncompressed). I can
probably post some code if needed, but since the example has the same
issue perhaps it is better to work from that.
My hope is that the root cause of this issue is:
A. Something in my usage of the library
B. Something wrong with the configure/build/install of the library
(which I cannot necessarily change)
C. Something wrong with the HDF5 library
Any feedback is welcome.
Thanks,
-Brendan
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
