Hi,
Am 05.10.2016 um 02:08 schrieb Carlos Penedo Rocha
<[email protected]<mailto:[email protected]>>:
Schlumberger-Private
Hi,
I have a scenario in which my compressed h5 file needs to be updated with new
data that is coming in every, say, 5 seconds.
Approach #1: keep the file opened and just write data as they come, or write a
buffer at once.
Approach #2: open the file (RDWR), write the data (or a buffer) and then close
the file.
Approach #1 is not desirable for my case because if there’s any problem
(outage, etc), then the h5 file will likely get corrupted. Or if I want to have
a look at the file, I can’t because it’s still writing (still opened).
Approach #2 is good to address the issue above, BUT I noticed that if I
open/write/close the file every 5 seconds, the file compression gets really bad
and the file size goes up big time. Approach 1 doesn’t suffer from this problem.
So, my question is: is there an “Approach #3” that gives me the best of the two
worlds? Less likely to get me a corrupted h5 file and at the same time, a good
compression rate?
I think this may be separate issues: if the compression is bad, this may be
because your chunk size is to small. Try writing every 10 sec or such.
Then there is the approach #3 which does not close the file but just flushes
it. Flushing also ensures that the on disk structure is intact, so you are
safe against a crashing program. The call would be H5Fflush().
Regards,
Mark Könnecke
Thanks,
Carlos R.
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]<mailto:[email protected]>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
