*I* have to chime in and say that I'm happy with the way HDF works now. Please don't take our single usecase as a need to spend a lot of work on this. I expected a simple yes or no to my question! Im totally fine with HDF not being the end all of every usecase.
On Thu, Apr 27, 2017 at 7:13 PM, Quincey Koziol <[email protected]> wrote: > Hi Werner, > > On Apr 26, 2017, at 12:34 AM, Werner Benger <[email protected]> wrote: > > Probably correct, Quincey might know better how far the hyperslab > implementation could be modified to allow a negative stride. Maybe it > wouldn't be much of a change, just there was no need so far to do it. If > that is the case, then at least it would mean that the "hyperslab grammar" > would indeed allow operations like transpose (and rotation) of a dataset, > just the implementation does not support it. That aside, it's probably not > efficient as it would require multiple hyperslab operations, so a point > selection would be faster, but for big data and out-of-core data > management, sometimes seemingly inefficient operations turn out to be > better. > > > I like the idea quite a lot. :-) However, it might be quite difficult to > implement. :-( The hyperslabs are “compiled” into [complex] internal data > structures as they are built up, in order to make the I/O operations as > fast as possible. It would be possible to defer that compilation until > when the actual I/O operation occurs, but that would be another layer of > data structures, etc. I believe it would be worthwhile, particularly so > that transposes are possible, but it would definitely be a fair bit of work… > > Quincey > > > On 26.04.2017 00:13, Isaac Gerg wrote: > > I am not sure if you are allowed a negative stride but even if you are, I > believe the transpose still won't work because the stride doesnt stride > dimension like numpy but strides across the memory space and wrap is not > allowed. Does this line of thinking make sense? > > On Tue, Apr 25, 2017 at 6:06 PM, Werner Benger <[email protected]> wrote: > >> Hi Isaac, >> >> it's just thought that it could be done like this: >> >> Copy operation: start = 0, stride = 1 >> >> target_index = 0 + source_index * 1 >> >> Reflection: start = max_index, stride = -1 >> >> target_index = max_index + source_index * -1 >> So a negative stride in one dimensions would go "backwards", thus do a >> reflection in this direction. In practice the stride parameter is probably >> an unsigned integer and thus running out of bounds instead of going >> backwards, as you say. >> >> Werner >> >> >> >> On 25.04.2017 23:48, Isaac Gerg wrote: >> >> Hi Werner, thanks for the reply. Let's wrestle with this a bit. So >> numpy does their indexing by strides, the stride for each dimension. hdf5 >> does their "striding" across the data in the sense of more of a selection >> than a stride. I would be inclined to say that even a negative stride >> wouldnt allow this. The problem is the stride is a simply stride and not >> one based on dimension index so the stride can't wrap so you end up with >> out of bounds issues with the datasets. Does this seem right? >> >> On Tue, Apr 25, 2017 at 5:29 PM, Werner Benger <[email protected]> >> wrote: >> >>> Hi Isaac, >>> >>> without proof, but I could imagine it's possible to implement a shear >>> operation via hyperslabs. If so, then hyperslabs could be used to implement >>> rotations: >>> >>> e.g. https://www.ocf.berkeley.edu/~fricke/projects/israel/paeth/r >>> otation_by_shearing.html >>> To do a transpose, one would need a reflection in addition to rotation >>> (e.g. http://techieme.in/matrix-rotation/ ) , and I guess doing this >>> operation of a reflection is not possible via hyperslabs, so doing a >>> transpose is out of reach. Probably it would require to allow something >>> like a negative stride value... >>> >>> Werner >>> >>> >>> >>> On 25.04.2017 22:24, Isaac Gerg wrote: >>> >>> Hi Quincy, I just was reading about point selections and was wondering >>> if that's the way we would go. Thanks for confirming! >>> >>> On Tue, Apr 25, 2017 at 4:21 PM, Quincey Koziol <[email protected]> wrote: >>> >>>> >>>> > On Apr 25, 2017, at 1:02 PM, Isaac Gerg <[email protected]> >>>> wrote: >>>> > >>>> > A coworker and I are mulling over if its possible to take a simple >>>> 3x3 matrix (stored row major) in a dataspace and write it out transposed to >>>> disk using a hyperslab. It seems that the grammer provided by start, >>>> stride, blocks, and count won't allow for this. We hope we are wrong! It >>>> is possible to do this? >>>> >>>> Hyperslabs won’t do this, but you could use a point selection >>>> instead. >>>> >>>> Quincey >>>> >>>> >>>> >>>> _______________________________________________ >>>> Hdf-forum is for HDF software users discussion. >>>> [email protected] >>>> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org >>>> Twitter: https://twitter.com/hdf5 >>> >>> >>> >>> >>> _______________________________________________ >>> Hdf-forum is for HDF software users >>> [email protected]http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org >>> Twitter: https://twitter.com/hdf5 >>> >>> -- >>> ___________________________________________________________________________ >>> Dr. Werner Benger Visualization Research >>> Center for Computation & Technology at Louisiana State University (CCT/LSU) >>> 2019 Digital Media Center, Baton Rouge, Louisiana 70803 >>> Tel.: +1 225 578 4809 <%28225%29%20578-4809> Fax.: >>> +1 225 578-5362 <%28225%29%20578-5362> >>> >>> _______________________________________________ Hdf-forum is for HDF >>> software users discussion. [email protected] >>> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org >>> Twitter: https://twitter.com/hdf5 >> >> _______________________________________________ >> Hdf-forum is for HDF software users >> [email protected]http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org >> Twitter: https://twitter.com/hdf5 >> >> -- >> ___________________________________________________________________________ >> Dr. Werner Benger Visualization Research >> Center for Computation & Technology at Louisiana State University (CCT/LSU) >> 2019 Digital Media Center, Baton Rouge, Louisiana 70803 >> Tel.: +1 225 578 4809 <%28225%29%20578-4809> Fax.: +1 >> 225 578-5362 <%28225%29%20578-5362> >> >> _______________________________________________ Hdf-forum is for HDF >> software users discussion. [email protected] >> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org >> Twitter: https://twitter.com/hdf5 > > _______________________________________________ > Hdf-forum is for HDF software users > [email protected]http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 > > -- > ___________________________________________________________________________ > Dr. Werner Benger Visualization Research > Center for Computation & Technology at Louisiana State University (CCT/LSU) > 2019 Digital Media Center, Baton Rouge, Louisiana 70803 > Tel.: +1 225 578 4809 <(225)%20578-4809> Fax.: +1 225 > 578-5362 <(225)%20578-5362> > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 > > > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 >
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
