Hi Steffen, We reviewed the patch today. It looks good. Could you please send us a test? We would love to have it in HDF5 1.10.0. For your reference the issue number is HDFFV-9634.
Thanks a lot! Elena ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elena Pourmal The HDF Group http://hdfgroup.org 1800 So. Oak St., Suite 203, Champaign IL 61820 217.531.6112 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ On Dec 18, 2015, at 6:04 AM, Steffen Kieß <[email protected]<mailto:[email protected]>> wrote: Hello, the code for handling external file reads and writes currently does not handle the case when the read() or write() operation returns a number smaller than the requested amount. Linux currently transfers at most 0x7ffff000 bytes per read() or write(), meaning that if more than 2GB are read from an external file only 2GB are read and the rest is filled up with zeros (because HDF5 thinks that it has reached EOF). http://man7.org/linux/man-pages/man2/read.2.html#NOTES https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=758170#10 (Note that the problem does not occur if the files datatype and the memory datatype do not match, because in this case HDF5 will use smaller reads into a buffer for converting the values.) I've attached a patch which will restart the read() or write() operation if a number smaller than the requested amount is returned (similar to e.g. the code in H5FDcore.c). I've also added logic which will restart the syscall after EINTR. Best regards, Steffen Kieß <efl-short-read.patch>_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
