Hi,
I am using HDF5 storage in SWMR mode. I am open files with huge number of
datasets and then I close it but after H5Fclose the amount of memory reserved
by the process is the same.
The same effect happens when data is written to a data set. All memory reserved
by the process (for chunks, caches, etc) is not freed after file is closed
(H5Fclose).
I have checked with valgrind but no memory leak is detected. It seams there is
a free of memory before the process finishes but I need this free when file is
closed.
Is possible to have memory free after H5Fclose without finishing the process?
A simplified example of my code follows the typical sequence:
———————————————————————————————
#include "hdf5.h"
#include "hdf5_hl.h"
#include <stdlib.h>
// Number of tables to create
#define NUM_DATASETS 10000
#define CHUNK_SIZE 100
typedef struct
{
double data;
long long timestamp;
} data_t;
int main(void)
{
hid_t fid;
hid_t sid;
hid_t dcpl;
hid_t pdsets[NUM_DATASETS];
char dname[300];
hsize_t dims[2] = {1, 0}; /* Dataset starting dimensions */
hsize_t max_dims[2] = {1, H5S_UNLIMITED}; /* Dataset maximum dimensions
*/
hsize_t chunk_dims[2] = {1, CHUNK_SIZE}; /* Chunk dimensions */
int i;
printf("Creating file\n");
// Open file
fid = H5Fcreate("packet.h5", H5F_ACC_TRUNC | H5F_ACC_SWMR_WRITE,
H5P_DEFAULT, H5P_DEFAULT);
// Create compound data type
hid_t datatype = H5Tcreate(H5T_COMPOUND, sizeof(data_t));
H5Tinsert(datatype, "Data", HOFFSET(data_t, data), H5T_NATIVE_DOUBLE);
H5Tinsert(datatype, "Timestamp", HOFFSET(data_t, timestamp),
H5T_NATIVE_LLONG);
/* Create dataspace for creating datasets */
if((sid = H5Screate_simple(2, dims, max_dims)) < 0)
return 1;
/* Create dataset creation property list */
if((dcpl = H5Pcreate(H5P_DATASET_CREATE)) < 0)
return -1;
if(H5Pset_chunk(dcpl, 2, chunk_dims) < 0)
return -1;
printf("Creating %d datasets\n", NUM_DATASETS);
// Create datasets
for (i = 0; i < NUM_DATASETS; i++) {
sprintf(dname,"dset_%d",i);
if((pdsets[i] = H5Dcreate2(fid, dname, datatype, sid,
H5P_DEFAULT, dcpl, H5P_DEFAULT)) < 0)
return 1;
if(H5Dclose(pdsets[i]) < 0)
return -1;
}
printf("Closing everything\n");
if(H5Pclose(dcpl) < 0)
return -1;
if(H5Sclose(sid) < 0)
return -1;
if(H5Tclose(datatype) < 0)
return -1;
if(H5Fclose(fid) < 0)
return -1;
printf("After closing...\n");
return 0;
}
---------------------------------------------------
Thank you.
Rodrigo
----------------------------
Confidencialidad:
Este mensaje y sus ficheros adjuntos se dirige exclusivamente a su destinatario
y puede contener informaci�n privilegiada o confidencial. Si no es vd. el
destinatario indicado, queda notificado de que la utilizaci�n, divulgaci�n y/o
copia sin autorizaci�n est� prohibida en virtud de la legislaci�n vigente. Si
ha recibido este mensaje por error, le rogamos que nos lo comunique
inmediatamente respondiendo al mensaje y proceda a su destrucci�n.
Disclaimer:
This message and its attached files is intended exclusively for its recipients
and may contain confidential information. If you received this e-mail in error
you are hereby notified that any dissemination, copy or disclosure of this
communication is strictly prohibited and may be unlawful. In this case, please
notify us by a reply and delete this email and its contents immediately.
----------------------------
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
