Hmm. Well I have no experience with HDF5's C++ interface. My first thought when reading your description was. . . I've seen that before. It happens when I forgot to H5Xclose() all the objects I H5Xopened (groups, datasets, types, dataspaces, etc.).
However, with C++, I presume the interface is designed to close objects when they fall out of scope (e.g. deconstructor is called). So, in looking at your code, even though I don't see any explicit calls to close objects previously opened, I assume that *should* be happening when the objects fall out of scope. But, are you *certain* that *is* happening? Just before exiting main, you migth wanna make a call to H5Fget_obj_count() to get some idea how many objects HDF5 library thinks are still open in the file. If you get a large number, then that would suggest the problem is that the C++ interface isn't somehow closing objects as they fall out of scope. Thats all I can think of. Sorry if no help. Mark From: Hdf-forum <[email protected]<mailto:[email protected]>> on behalf of Jorj Pimm <[email protected]<mailto:[email protected]>> Reply-To: HDF Users Discussion List <[email protected]<mailto:[email protected]>> Date: Thursday, August 13, 2015 9:21 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: [Hdf-forum] Growing memory usage in small HDF program Hello, I am writing an application which writes large data sets to HDF5 files, in fixed size blocks, using the HDF C++ API (version 1.8.15, patch 1, built in msvc 2013 x64) I my application seems to quickly consume all the available memory on my system (win32 - around 5.9GB), and then crash whenever the system becomes stressed (windows kills it as it has no memory) I have also tested the application on a linux machine, where I saw similar results. I was under the impression that by using HDF5, the file would be brought in and out of memory in such a way that the library would only use a small working set - is this not true? I have experimented with HDF features such as flushing to disk, regularly closing and re opening, garbage collection and tuning chunking and caching settings and haven't managed to get a stable working set. I've attached a minimal example, can anyone point out my mistake? Thanks, - Jorj
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
