With C++ you can eliminate the need for the row pointers. Here's a simple
2d heap array class. I just typed it out quickly so it might not be perfect
but it captures the gist of what you were doing.
template<typename T>
class TwoDArray
{
public:
typedef T value_type;
typedef T* pointer;
TwoDArray(size_t _rows, size_t _cols) : rows_(_rows), cols_(_cols),
data_(new value_type[_rows*_cols]) {}
~TwoDArray() { delete data_; }
pointer operator[](size_t row) { return data_ + row*cols_; }
size_t rows() const { return rows_; }
size_t cols() const { return cols_; }
pointer data() { return data_; }
private:
size_t rows_;
size_t cols_;
pointer data_;
};
void f()
{
TwoDArray<double> x(3, 5);
x[1][4] = 5.1;
H5DWrite(... x.data());
}
On Tue, May 17, 2016 at 9:09 AM, David <[email protected]> wrote:
> > My original example was about new and allocating to the heap
>
> I remembered incorrectly, sorry. HDF5 works fine with 'new'. You were
> trying to to use the heap and have multi dim indexing. The problem was that
> you made separate allocations for each row and then were writing the row
> pointers instead of the data. To write data with a single call to HD5Write
> the data must allocated with a single call to 'new'.
>
> Here's my sample code from before showing how to do what you were
> attempting, using 'new' and getting multi dim indexing. You will give
> 'data' to HDF5 for writing and use 'md_data' for the indexing behavior.
> This could be made into a class to encapsulate the behavior for easy reuse.
>
> auto data = new double[rows*cols]; // allocate all data in one block
> auto md_data = new double*[rows]; // allocate pointers for each row
> for (int r = 0; r != rows; ++r) // set row pointers
> md_data[r] = data + r*cols;
> md_data[2][5] = 1.0; // row pointer array can be used as a pseudo md array
>
> H5DWrite(... , data);
>
> David
>
> On Tue, May 17, 2016 at 8:31 AM, Steven Walton <[email protected]>
> wrote:
>
>> I guess I am just surprised that hdf5 does not accept the standard c++
>> "array" creation. New allocates to the heap. It may be pointer, but this is
>> pretty standard and they teach it in all c++ classes. New has existed for
>> quite some time. I just thought the hdf5 group would follow these standard
>> types.
>>
>> I understand that vector is a 1D array. My original example was about new
>> and allocating to the heap, with standard c++.
>>
>> On Mon, May 16, 2016 at 9:30 PM, David <[email protected]> wrote:
>>
>>> The limitation you ran into in your original example was that
>>> std::vector is only for 1 dimensional arrays and you wanted to use multi
>>> dimensional indexing. C++ does not have a built in class for heap allocated
>>> multi dim arrays. However you can simply use boost::multi_array. The
>>> standards committee wisely has been using boost as a testing ground for new
>>> library features, allowing them to mature and prove themselves before being
>>> canonized. I will not be surprised if boost::multi_array is adopted into a
>>> future standard.
>>>
>>> David
>>>
>>>
>>> On Mon, May 16, 2016 at 7:01 PM, Steven Walton <[email protected]
>>> > wrote:
>>>
>>>> I actually just created a buffer array and then passed back to a
>>>> vector. But what I'm saying is that this is an extremely common way to
>>>> store data in C++, if not the default way for most users. Vectors are
>>>> extremely common as well. Why are we still held back by using C type
>>>> arrays?
>>>>
>>>> On Mon, May 9, 2016 at 12:58 PM, David <[email protected]> wrote:
>>>>
>>>>> Hi Steve,
>>>>>
>>>>> boost::multi_array provides a clean interface for multi dimensional
>>>>> arrays in C++.
>>>>>
>>>>> You can also do something like this:
>>>>>
>>>>> auto data = new double[rows*cols]; // allocate all data in one block
>>>>> auto md_data = new double*[rows]; // allocate pointers for each row
>>>>> for (int r = 0; r != rows; ++r) // set row pointers
>>>>> md_data[r] = data + r*cols;
>>>>> md_data[2][5] = 1.0; // row pointer array can be used as a pseudo md
>>>>> array
>>>>>
>>>>>
>>>>> On Fri, May 6, 2016 at 1:29 PM, Steven Walton <
>>>>> [email protected]> wrote:
>>>>>
>>>>>> So I am noticing some interesting behavior and is wondering if there
>>>>>> is a way around this.
>>>>>> I am able so assign a rank 1 array dynamically and write this to an
>>>>>> hdf5 filetype but I do not seem to be able to do with with higher order
>>>>>> arrays. I would like to be able to write a PPx array to h5 and retain the
>>>>>> data integrity. More specifically I am trying to create a easy to use
>>>>>> vector to array library <https://github.com/stevenwalton/H5Easy>
>>>>>> that can handle multidimensional data (works with rank 1).
>>>>>>
>>>>>> Let me give some examples. I will also show the typenames of the
>>>>>> arrays.
>>>>>>
>>>>>> Works:
>>>>>> double *a = new double[numPts]; // typename: Pd
>>>>>> double a[numPts]; // typename A#pts_d
>>>>>> double a[num1][num2]; typename:Anum1_Anum2_d
>>>>>>
>>>>>> What doesn't work:
>>>>>> double **a = new double*[num1];
>>>>>> for ( size_t i = 0; i < num1; ++i )
>>>>>> a[i] = new double[num2];
>>>>>> // typename PPd
>>>>>>
>>>>>> Testing the saved arrays with h5dump (and loading and reading
>>>>>> directly) I find that if I have typename PPx (not necessarily double) I
>>>>>> get
>>>>>> garbage stored. Here is an example code and output from h5dump showing
>>>>>> the
>>>>>> behavior.
>>>>>> ------------------------------------------------------------
>>>>>> compiled with h5c++ -std=c++11
>>>>>> ------------------------------------------------------------
>>>>>> #include "H5Cpp.h"
>>>>>> using namespace H5;
>>>>>>
>>>>>> #define FILE "multi.h5"
>>>>>>
>>>>>> int main()
>>>>>> {
>>>>>> hsize_t dims[2];
>>>>>> herr_t status;
>>>>>> H5File file(FILE, H5F_ACC_TRUNC);
>>>>>> dims[0] = 4;
>>>>>> dims[1] = 6;
>>>>>>
>>>>>> double **data = new double*[dims[0]];
>>>>>> for ( size_t i = 0; i < dims[0]; ++i )
>>>>>> data[i] = new double[dims[1]];
>>>>>>
>>>>>> for ( size_t i = 0; i < dims[0]; ++i )
>>>>>> for ( size_t j = 0; j < dims[1]; ++j )
>>>>>> data[i][j] = i + j;
>>>>>>
>>>>>> DataSpace dataspace = DataSpace(2,dims);
>>>>>> DataSet dataset( file.createDataSet( "test", PredType::IEEE_F64LE,
>>>>>> dataspace ) );
>>>>>> dataset.write(data, PredType::IEEE_F64LE);
>>>>>> dataset.close();
>>>>>> dataspace.close();
>>>>>> file.close();
>>>>>>
>>>>>> return 0;
>>>>>> }
>>>>>> ------------------------------------------------------------
>>>>>> h5dump
>>>>>> ------------------------------------------------------------
>>>>>> HDF5 "multi.h5" {
>>>>>> GROUP "/" {
>>>>>> DATASET "test" {
>>>>>> DATATYPE H5T_IEEE_F64LE
>>>>>> DATASPACE SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
>>>>>> DATA {
>>>>>> (0,0): 1.86018e-316, 1.86018e-316, 1.86018e-316, 1.86019e-316,
>>>>>> 0,
>>>>>> (0,5): 3.21143e-322,
>>>>>> (1,0): 0, 1, 2, 3, 4, 5,
>>>>>> (2,0): 0, 3.21143e-322, 1, 2, 3, 4,
>>>>>> (3,0): 5, 6, 0, 3.21143e-322, 2, 3
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>> }
>>>>>> ------------------------------------------------------------------
>>>>>> As can be seen the (0,0) set is absolute garbage (except the last
>>>>>> character which is the first number of the actual array), (0,5) is out of
>>>>>> bounds, and has garbage data. (1,0) has always contained real data
>>>>>> (though
>>>>>> it should be located at (0,0)). So this seems like some addressing
>>>>>> problem.
>>>>>>
>>>>>> Is this a bug in the h5 libraries that allows me to read and write Pd
>>>>>> data as well as Ax0_...Axn_t data but not P...Pt data? Or is this for
>>>>>> some
>>>>>> reason intentional? As using new is a fairly standard way to assign
>>>>>> arrays,
>>>>>> making P...Pt type data common, I have a hard time seeing this as
>>>>>> intentional. In the mean time is anyone aware of a workaround to this?
>>>>>> The
>>>>>> data I am taking in will be dynamically allocated so I do not see a way
>>>>>> to
>>>>>> get Ax_... type data.
>>>>>>
>>>>>> Thank you,
>>>>>> Steven
>>>>>>
>>>>>> _______________________________________________
>>>>>> Hdf-forum is for HDF software users discussion.
>>>>>> [email protected]
>>>>>>
>>>>>> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>>>>>> Twitter: https://twitter.com/hdf5
>>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Hdf-forum is for HDF software users discussion.
>>>>> [email protected]
>>>>> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>>>>> Twitter: https://twitter.com/hdf5
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Hdf-forum is for HDF software users discussion.
>>>> [email protected]
>>>> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>>>> Twitter: https://twitter.com/hdf5
>>>>
>>>
>>>
>>> _______________________________________________
>>> Hdf-forum is for HDF software users discussion.
>>> [email protected]
>>> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>>> Twitter: https://twitter.com/hdf5
>>>
>>
>>
>> _______________________________________________
>> Hdf-forum is for HDF software users discussion.
>> [email protected]
>> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>> Twitter: https://twitter.com/hdf5
>>
>
>
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
