Hi Samer, Sorry I had a copy-paste malfunction there. I actually meant to use H5Fget_file_image <https://support.hdfgroup.org/HDF5/doc/RM/RM_H5F.html#File-GetFileImage> twice instead. First using a NULL pointer to retrieve the "file size", then allocating some memory and calling it again to get the actual data.
I think the purpose of setting your own callbacks is that you can handle the *realloc *call so you know when the pointer gets changed. Because even if you use the "in-place" flag, adding attributes will likely cause the image to be expanded and hence require a new buffer. But having your own realloc lets you catch that. According to the manual, callback functions are allowed to be NULL to indicate the defaults should be used. I did some work with file images a while back, but admittedly we didn't bother with in-place for our application as the copy overhead was relatively small. But maybe I can modify some old code to make an example. Cheers, Martijn On 2 January 2017 at 13:28, Samer Afach <[email protected]> wrote: > Hi Martijn: > > Thank you very much for your response. Since you split your answer into > two parts, I also have to do the same and ask two questions about these > parts, if you don't mind. > > 1. You said "you can get the file image back from two calls to > H5LTopen_file_image" But I don't get what two calls you meant. Are you > talking about the calls of "H5Pset_file_image_callbacks"? > > 2. I would love to do this as "in place", but there's a memory leak I > reported in the HDF5 library that happens when opening images as in-place > (ref. HDFFV-10019), and I'm waiting for feedback about that. But ignoring > that memory leak, are you saying this is different from the first case > (when it's not in-place)? And are you saying that to get the size of the > image, the only way is to re-implement *all* the callback functions of > H5Pset_file_image_callbacks? > > Honestly that's a little scary that I'd love to see an example about this. > > Thank you for all this information! You helped a lot :) > > Best, > Samer > > On 01.01.2017 11:16 PM, Martijn Jasperse wrote: > > Hi Samer, > Since you opened the file without H5LT_FILE_IMAGE_DONT_COPY, the library > makes a copy of the original buffer and operates on that. So once you've > created the attribute you wanted and done a flush, you can get the file > image back from two calls to H5LTopen_file_image as usual (first to get > the size and then to get the actual data). > > If you wanted to do it "in place" without making a copies you'd likely > need to look into H5Pset_file_image_callbacks to handle when the file > image is resized and hence when the buffer changes. > > Cheers, > Martijn > > On 27 December 2016 at 10:47, Samer Afach <[email protected]> wrote: > >> Hi Landon: >> >> Thanks for the response. Actually I'm not talking about file objects, but >> *file >> images*, which is basically the whole file stored in some buffer in >> memory. Imagine you use std::fstream::read() or the classic C FILE's >> fread() to read the whole file into memory, and then wanting to open it >> with the HDF5 library. This is why I'm using "H5LTopen_file_image()", >> which doesn't take a file name or file path, but takes a void* buffer, >> which is a pointer to the array where the file is stored *in memory*. >> >> Now after opening that using "H5LTopen_file_image()", we'll get an HDF5 >> file object (or file handle), the one you're talking about. My question is: >> Now after we do the changes to the file under that object, and after we >> H5Fflush(), that buffer may be changed. My question is: How do we keep >> track of that buffer (and its size) to write it back to a file eventually >> (using a std::fstream::write() or the classic C FILE's fwrite())? >> >> I would appreciate the help of anyone who knows about this. >> >> Best, >> Samer Afach >> >> On 27.12.2016 12:19 AM, Landon Clipp wrote: >> >> Hello Samer, >> >> I think I get the gist of what you're asking. If you are writing to the >> object and then want to read from the updated file, you will need to call >> H5Fflush() in order to write your data buffers to the file on your disk. >> Modifying objects on your file does not actually immediately change the >> file itself. The changes are only made when the object and/or file is >> closed or flushed. I believe that should solve your problem. >> >> Regards, >> Landon Clipp >> >> On Thu, Dec 22, 2016 at 2:39 AM, Samer Afach <[email protected]> wrote: >> >>> Hello guys: >>> >>> In a program, I use some relatively small HDF5 files multiple times and >>> transfer them then through TCP/IP, so I read them into memory as >>> `std::string`, and then read them as HDF5 using: >>> >>> fileHandle = H5LTopen_file_image((void*)&Fi >>> leImage.front(),FileImage.size(),H5LT_FILE_IMAGE_OPEN_RW); >>> *My question is:* Since I'm opening this as RW, how can I write an >>> attribute to it and retrieve the image back? >>> >>> The motivation behind my question is simple. I have a simple function >>> that can write an attribute to an object by its ID. Here it's: >>> >>> void WriteStringAttribute(hid_t dataset_id, const std::string& name, const >>> std::string& value) >>> { hid_t AttributeType = H5Tcopy(H5T_C_S1); H5Tset_size(AttributeType, >>> value.length()); hid_t dataspaceHandle = H5Screate(H5S_SCALAR); hid_t >>> AttrHandle = H5Acreate(dataset_id, name.c_str(), AttributeType, >>> dataspaceHandle, H5P_DEFAULT,H5P_DEFAULT); H5Awrite(AttrHandle, >>> AttributeType, &value.front()); >>> >>> H5Aclose (AttrHandle); H5Sclose (dataspaceHandle); >>> } >>> >>> >>> What I don't understand is that after writing the attribute using this >>> function (Is it right to do this? >>> >>> WriteStringAttribute(fileHandle, "MyAttr", "MyAttrValue"); >>> >>> ), the image size should've been changed. How can I get the new correct >>> image size and write it back to a file (or do anything else with it, for >>> that matter)? >>> >>> Thank you. >>> >>> Best, Samer Afach >>> _______________________________________________ Hdf-forum is for HDF >>> software users discussion. [email protected] >>> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org >>> Twitter: https://twitter.com/hdf5 >> >> _______________________________________________ >> Hdf-forum is for HDF software users >> [email protected]http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org >> Twitter: https://twitter.com/hdf5 >> >> _______________________________________________ Hdf-forum is for HDF >> software users discussion. [email protected] >> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org >> Twitter: https://twitter.com/hdf5 > > _______________________________________________ > Hdf-forum is for HDF software users > [email protected]http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 > > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 >
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
