Hi Elena, Thanks for the response, my problem is not that h5dump or other hdf5 command line utilities do not work the lzf plugin filter (they work just fine) but the Java HDF5 and HDFView can't find the plugin_filter. I will check the symbols tomorrow but as far as I can tell the problem is somewhere with the java hdf5 library.
On Mon, Nov 23, 2015 at 11:41 PM, Elena Pourmal <[email protected]> wrote: > Hello, > > What kind of error are you getting? I was able to get the filter to work > on Linux (CentOS 7) with h5dump from 1.8.16 distribution I built. Please > notice that one has to specify HDF5 include files when building > liblzf_filter.so. > > Details: > > Here is the output of the nm command and h5dump when include files are not > specified: > > [epourmal@platypus lzf]$ gcc -O2 -fPIC -shared lzf/*.c lzf_filter.c -o > liblzf_filter.so > epourmal@platypus lzf]$ nm liblzf_filter.so > U H5E_CALLBACK_g > U H5E_CANTREGISTER_g > U H5E_PLINE_g > U H5Epush1 > U H5Pget_chunk > U H5Pget_filter_by_id2 > U H5Pmodify_filter > U H5Tget_size > U H5Zregister > U H5open > [epourmal@platypus lzf]$ hdf5/hdf5/bin/h5dump test_lzf.hdf5 > HDF5 "test_lzf.hdf5" { > GROUP "/" { > DATASET "dset" { > DATATYPE H5T_IEEE_F32LE > DATASPACE SIMPLE { ( 100, 100, 100 ) / ( 100, 100, 100 ) } > DATA {h5dump error: unable to print data > > } > } > } > } > [epourmal@platypus lzf]$ hdf5/hdf5/bin/h5dump -pH test_lzf.hdf5 > HDF5 "test_lzf.hdf5" { > GROUP "/" { > DATASET "dset" { > DATATYPE H5T_IEEE_F32LE > DATASPACE SIMPLE { ( 100, 100, 100 ) / ( 100, 100, 100 ) } > STORAGE_LAYOUT { > CHUNKED ( 1, 100, 100 ) > SIZE 174288 (22.951:1 COMPRESSION) > } > FILTERS { > PREPROCESSING SHUFFLE > USER_DEFINED_FILTER { > FILTER_ID 32000 > COMMENT lzf > PARAMS { 4 261 40000 } > } > ... > > Now if I specify HDF5 include files, I will get (notice two symbols > H5PLget_plugin_info/type in the nm output!) > > [epourmal@platypus lzf]$ gcc > -I//scr/epourmal/h5py-master/lzf/hdf5/hdf5/include -O2 -fPIC -shared > lzf/*.c lzf_filter.c -o liblzf_filter.so > [epourmal@platypus lzf]$ nm liblzf_filter.so > U H5E_CALLBACK_g > U H5E_CANTREGISTER_g > U H5E_PLINE_g > U H5Epush1 > 0000000000001080 T H5PLget_plugin_info > 0000000000001070 T H5PLget_plugin_type > U H5Pget_chunk > U H5Pget_filter_by_id2 > U H5Pmodify_filter > U H5Tget_size > U H5Zregister > U H5open > > And here we are: > > [epourmal@platypus lzf]$ hdf5/hdf5/bin/h5dump test_lzf.hdf5 |more > HDF5 "test_lzf.hdf5" { > GROUP "/" { > DATASET "dset" { > DATATYPE H5T_IEEE_F32LE > DATASPACE SIMPLE { ( 100, 100, 100 ) / ( 100, 100, 100 ) } > DATA { > (0,0,0): 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, > 17, > (0,0,18): 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, > (0,0,33): 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, > (0,0,48): 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, > (0,0,63): 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, > (0,0,78): 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, > (0,0,93): 93, 94, 95, 96, 97, 98, 99, > ……. > > Hope it will help! > > Elena > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > Elena Pourmal The HDF Group http://hdfgroup.org > 1800 So. Oak St., Suite 203, Champaign IL 61820 > 217.531.6112 > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > > > On Nov 23, 2015, at 8:50 AM, Daniel Tetlow <[email protected]> > wrote: > > Hi, > > I had a similar problem on Windows. I needed to link the LZF plugin > against the hdf5 library (as expected) to get it to compile, and it worked > fine with C++, but it wouldn’t work with the Java H5 libraries. Attempting > to link it against the jhdf5 2.11 library instead failed. I couldn’t figure > out what I was doing wrong, so in the end I rebuilt jhdf5 with the plugin > sources built in. > > Cheers, > > Dan > > > *From:* Hdf-forum [mailto:[email protected] > <[email protected]>] *On Behalf Of *Ümit Seren > *Sent:* 23 November 2015 14:10 > *To:* [email protected] > *Subject:* [Hdf-forum] Problem with a filter as shared library in > Java-HDF5 and HDF5View > > I have been trying to compile the lzf compressor as a shared library (not > linked against hdf5) to be used as a filter plugin for HDF5. > > Basically I was following this: > https://github.com/h5py/h5py/tree/master/lzf > > I compiled lzf as a shared library using this compile command: > > > gcc -O2 -fPIC -shared lzf/*.c lzf_filter.c -o liblzf_filter.so > > > > This will create a lzf filter plugin that I copied into the > /usr/local/hdf5/lib/plugin folder. > > The normal command line tools (h5ls, h5dump) work fine with lzf compressed > dataset, so the plugin seems to work. > > > > However when I try to read the same lzf compressed dataset using Java HDF5 or > HDFView (2.11), I get the error that the lzf filter plugin can not be found. > > > > The only workaround is to link it against the hdf5 library: > > > > gcc -O2 -fPIC -shared lzf/*.c lzf_filter.c -lhdf5 -o liblzf_filter.so > > > > Is this a known issue ? > > This was on Ubuntu 14.04 and HDF5 1.8.15patch1 > > thanks in advance > > cheers > Ümit > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 > > > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 >
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
