2016-10-31 22:50 GMT+01:00 Peter Lindstrom <[email protected]>: > My bad--I saw the table of six integers in README_MORE and assumed those > were the six integers in PARAMS. I should have kept reading... > > Still, looking at the integer parameters there appear to be some > unintentional byte swapping issues. The first four bytes should be 'z', > 'f', 'p', \005, and these bytes show up in the second (32-bit) integer > parameter (91252346 = 0x0570667a = ((\005 << 24) + ('p' << 16) + ('f' << 8) > + ('z' << 0))).
I see now that the first entry (5242928 = 0x00500030) is the version of ZFP / H5Z-ZFP (0.5.0 / 0.3.0), the filter encodes the ZFP header from entry 1 and forward. Elvis > > Mark and I will work to track down the issue. Elvis, can you confirm that > you compiled zfp with -DBIT_STREAM_WORD_TYPE=uint8? See the zfp FAQ and > H5Z-ZFP/README_MORE for more information on this. > > > On 10/31/16 11:18, Miller, Mark C. wrote: > > Hi Elvis, > > The PARAMS dumped from the dataset header are *not* the same cd_values > passed in memory between caller and plugin. > > The dataset header in the file gets something different; ZFP's header > including mode and metadata + H5Z-ZFP plugin version info > > The README_MORE file in plugin on github explains this. > > Long story short, you can't deduce much from those values dumped by > h5dump/h5ls, etc. You'd have to reverse engineer how > ZFP library encodes magic, version, mode and metadata into its header. > > Mark > > > > -- > Mark C. Miller, LLNL > > From: Hdf-forum <[email protected]> on behalf of Peter > Lindstrom <[email protected]> > Reply-To: HDF Users Discussion List <[email protected]> > Date: Monday, October 31, 2016 at 11:03 AM > To: HDF Users Discussion List <[email protected]> > Subject: Re: [Hdf-forum] New HDF5 compression plugin > > Elvis, > > Your compression parameters look suspicious to me: > > PARAMS { 5242928 91252346 313532218 -1043792 -937099264 67112167 } > > I would start by debugging those. If I understand Mark's filter correctly, > the first integer should be 1, 2, 3, or 4. > > On 10/31/16 09:55, Elvis Stansvik wrote: > > Den 31 okt. 2016 5:41 em skrev "Miller, Mark C." <[email protected]>: >> >> Hi Elvis, >> >>> >>> I've successfully tried using the plugin through h5py. I did have a >>> problem with the fixed-accuracy mode however and filed an issue: >>> >>> https://github.com/LLNL/H5Z-ZFP/issues/1 >>> >>> It's very likely that I'm doing something wrong though. >> >> >> Thanks so much!! >> >> I will take a look at the issue you reported later this week. >> >> I may have to rope the researcher who developed ZFP in to help. >> >> AFAICT, I am calling ZFP with correct parameters and, your email confirms >> you see zfp_stream_set_accuracy() being called. > > Alright, no hurry. > > I've been in contact with Peter earlier with some other questions. He seems > very helpful. > > And yes, zfp_stream_set_accuracy seems to be called correctly with the > parameters I pass through h5py (same as when I do the equivalent with the > zfp command line tool), so I think something else is going on. The dataset > type class and dimensions are also correctly identified by the filter > plugin. > > Elvis > >> >> Mark >> >> >> _______________________________________________ >> Hdf-forum is for HDF software users discussion. >> [email protected] >> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org >> Twitter: https://twitter.com/hdf5 > > > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected]http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 > > > > -- > Peter Lindstrom . [email protected] . http://people.llnl.gov/pl . 925-423-5925 > > > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 > > > > -- > Peter Lindstrom . [email protected] . http://people.llnl.gov/pl . 925-423-5925 > > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
