Hey Lance, why not use a standard C++ vector? #include <vector>
... vector<float> geoArray(X*Y); dataset.read(&geoArray[0], ...) ... This gives you a 1D array, but then you can just use a macro or inline function to implement the 2D [i,j] access, if you’d like. G. From: Hdf-forum [mailto:[email protected]] On Behalf Of Lance Steele Sent: Thursday, June 4, 2015 1:25 PM To: [email protected] Subject: [Hdf-forum] Difficulty reading VIIRS satellite data using the HDF5 C++ API Greetings everyone, I am attempting to use the HDF5 C++ API to read data from the Visible Infrared Imaging Radiometer Suite (VIIRS) satellite. VIIRS Sensor Data Records (SDR) can be downloaded freely from NOAA’s Comprehensive Large Array-Data Stewardship System (CLASS) at this URL: http://www.nsof.class.noaa.gov/saa/products/search?sub_id=0&datatype_family=VIIRS_SDR&submit.x=19&submit.y=3. The problem I am encountering is that the data is stored in large arrays. Some data are 6144 by 6400 32-bit floating point format. That requires 314,573 kilobytes of memory to load into an array. In C++, I must declare that array in the stack memory, and there is not nearly enough for that size. Here is a condensed section of c++ code: const int X = 6144; const int Y = 6400; float geoArray[X][Y] ... dataSet.read(geoArray, PredType::NATIVE_FLOAT, memSpace, dataSpace); Even when I change my computer’s stack size limit to 65,532 kilobytes or attempt to dynamically allocate the array, neither solution is not enough to prevent a segmentation fault. I am currently using h5dump to create binary files that I can then read into c++ vectors (which allows the data to be stored in the heap rather than the limited stack). Does the dataset read function have any provision or usage to access values individually rather than as a full array? How else can I access such large amounts of data using the HDF5 C++ API? Thank you and best regards, Lance
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
