Stuart is correct, any readers will need to close and re-open the file
after a write. See: http://www.hdfgroup.org/hdf5-quest.html#gconc1.
Anyone who is dealing with multiple reader/multiple writer issues may be
interested in trying out the HDF5 Server I've been working over the last
couple of months. It's a REST based service written in Python with an API
based on the paper Gerd wrote a while back:
http://www.hdfgroup.org/pubs/papers/RESTful_HDF5.pdf. With a service the
clients are abstracted from dealing with read/write locks, so it will
support multiple writers/multiples readers with no explicit synchronization.
I should have this available on Github in a couple of weeks.
Big provisos...
This first version doesn't do anything clever for parallelization - in
effect every call to the service is handled serially.
And - there are no client API's at this point. Interaction with the
service is just through http GET/PUT/POST/DELETE requests.
John Readey
HDF Group
On Monday, November 10, 2014 8:25:24 AM UTC-8, stuarteberg wrote:
>
> Hi Ray,
>
> In fact, I implemented something quite similar. ... to create a
>> transparent workaound for the limitation that parallel read *and* write
>> access is not possible with hdf5/h5py. ... As soon as one process wants to
>> write, all other processes accessing that file must wait until the writing
>> is done.
>>
>
> Sounds useful; looking forward to seeing your code. But beware: If I
> understand correctly, merely waiting until the writing is done isn't enough
> to avoid problems. If the file has been changed at all, the reading
> processes will probably need to close the file entirely and re-open it once
> writing is finished. In fact, I'm not sure if it is even permitted to keep
> a file open in 'r+' mode while other processes are reading it in 'r' mode.
> Perhaps an hdf5 or h5py dev can chime in on this point.
>
> Best,
> Stuart
>
>
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
