My bad--I saw the table of six integers in README_MORE and assumed those
were the six integers in PARAMS. I should have kept reading...
Still, looking at the integer parameters there appear to be some
unintentional byte swapping issues. The first four bytes should be 'z',
'f', 'p', \005, and these bytes show up in the second (32-bit) integer
parameter (91252346 = 0x0570667a = ((\005 << 24) + ('p' << 16) + ('f' <<
8) + ('z' << 0))).
Mark and I will work to track down the issue. Elvis, can you confirm
that you compiled zfp with -DBIT_STREAM_WORD_TYPE=uint8? See the zfp FAQ
and H5Z-ZFP/README_MORE for more information on this.
On 10/31/16 11:18, Miller, Mark C. wrote:
Hi Elvis,
The PARAMS dumped from the dataset header are *not* the same cd_values
passed in memory between caller and plugin.
The dataset header in the file gets something different; ZFP's header
including mode and metadata + H5Z-ZFP plugin version info
The README_MORE file in plugin on github explains this.
Long story short, you can't deduce much from those values dumped by
h5dump/h5ls, etc. You'd have to reverse engineer how
ZFP library encodes magic, version, mode and metadata into its header.
Mark
--
Mark C. Miller, LLNL
From: Hdf-forum <[email protected]
<mailto:[email protected]>> on behalf of Peter
Lindstrom <[email protected] <mailto:[email protected]>>
Reply-To: HDF Users Discussion List <[email protected]
<mailto:[email protected]>>
Date: Monday, October 31, 2016 at 11:03 AM
To: HDF Users Discussion List <[email protected]
<mailto:[email protected]>>
Subject: Re: [Hdf-forum] New HDF5 compression plugin
Elvis,
Your compression parameters look suspicious to me:
PARAMS { 5242928 91252346 313532218 -1043792 -937099264 67112167 }
I would start by debugging those. If I understand Mark's filter
correctly, the first integer should be 1, 2, 3, or 4.
On 10/31/16 09:55, Elvis Stansvik wrote:
Den 31 okt. 2016 5:41 em skrev "Miller, Mark C."
<[email protected] <mailto:[email protected]>>:
>
> Hi Elvis,
>
>>
>> I've successfully tried using the plugin through h5py. I did
have a
>> problem with the fixed-accuracy mode however and filed an issue:
>>
>> https://github.com/LLNL/H5Z-ZFP/issues/1
>>
>> It's very likely that I'm doing something wrong though.
>
>
> Thanks so much!!
>
> I will take a look at the issue you reported later this week.
>
> I may have to rope the researcher who developed ZFP in to help.
>
> AFAICT, I am calling ZFP with correct parameters and, your
email confirms you see zfp_stream_set_accuracy() being called.
Alright, no hurry.
I've been in contact with Peter earlier with some other
questions. He seems very helpful.
And yes, zfp_stream_set_accuracy seems to be called correctly
with the parameters I pass through h5py (same as when I do the
equivalent with the zfp command line tool), so I think something
else is going on. The dataset type class and dimensions are also
correctly identified by the filter plugin.
Elvis
>
> Mark
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [email protected] <mailto:[email protected]>
>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> Twitter: https://twitter.com/hdf5
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter:https://twitter.com/hdf5
--
Peter Lindstrom [email protected] .http://people.llnl.gov/pl . 925-423-5925
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
--
Peter Lindstrom . [email protected] . http://people.llnl.gov/pl . 925-423-5925
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
