[Hdf-forum] paging approaches

2010-12-03 Thread Philip Winston
We just added HDF5 support in our application. We are using the C API. Our datasets are 1D and 2D arrays of integers, a pretty simple structure on disk. Today we have about 5GB of data and we load the whole thing into RAM, do somewhat random reads, make changes, then overwrite the old .h5 file. I

[Hdf-forum] H5FL_malloc calls

2010-12-03 Thread Mark Chaisson
Hi, this is my first post to hdf-forum so apologies in advance for anything that is redundant that I've missed in previous posts. I have a process that is running for several hours opening and reading from many hdf files. The heap memory is constantly growing, and because all memory is returned u

[Hdf-forum] write faster than read? weird.

2010-12-03 Thread stnchris
Finally developed things to a point that I can get useful performance numbers for my application. So far, things look good. But, when I look at the performance numbers I see behavior I don't expect -- namely, that my write throughput is almost 2x greater than my read throughput. My system

Re: [Hdf-forum] Creating HDF5 file with HDFView

2010-12-03 Thread Hillman, Jonathan E
Thanks for replying... this is the path I thought would let me create a test HDF5 file with 4 datasets and importing data from a file... HDFView: 1) New -> HDF5 (Test.h5) 2) Right click on Test.h5, New -> Dataset Dataset Name: Sensor1 Datatype class: Float, Size: native, Byte ordering: nati

Re: [Hdf-forum] Creating HDF5 file with HDFView

2010-12-03 Thread Peter Cao
Hi Jonathan, Could you be more specific how you did it? Thanks --pc On 12/3/2010 11:22 AM, Hillman, Jonathan E wrote: I am trying to create a trivial HDF5 formatted file with HDFView. I created 4 datasets of single diminsion with 1 item and unlimited max size each for testing. I assumed I co

[Hdf-forum] Creating HDF5 file with HDFView

2010-12-03 Thread Hillman, Jonathan E
I am trying to create a trivial HDF5 formatted file with HDFView. I created 4 datasets of single diminsion with 1 item and unlimited max size each for testing. I assumed I could load the data using import data from file, but that doesn't seem to work. Am I attempting to do something not intend

[Hdf-forum] H5Fclose blocking on independent only h5 access in only one MPI process

2010-12-03 Thread Roger Martin
Hi, Using hdf1.8.5 and 1.8.6 pre2; openmpi 1.4.3 on linux rhel4 and rhel5 In a case where the hdf5 operations aren't using MPI but build an h5 file exclusive to individual MPI jobs/processes: The create: currentFileID = H5Fcreate(filePath.c_str(), H5F_ACC_TRUNC, H5P_DEFAULT, H5P_DEFAULT);