Hi Ahmed,
I understand that you found the problem, but I just want to mention to you about the HDF4 Mapping tool: http://www.hdfgroup.org/projects/h4map/, which might be useful to your applications. This tool will give you a "map" that describes the structure of an HDF file. Binh-Minh ________________________________ From: Hdf-forum <[email protected]> on behalf of Ahmed Eldawy <[email protected]> Sent: Wednesday, March 11, 2015 6:47 PM To: hdf-forum Subject: Re: [Hdf-forum] Read a compressed (deflate) block using Java Sorry. My bad. I had to use InflaterInputStream instead of DeflaterInputStream. What I was doing was simply recompressing the data that is already compressed. It works fine now. Best regards, Ahmed Eldawy On Wed, Mar 11, 2015 at 5:36 PM, Ahmed Eldawy <[email protected]<mailto:[email protected]>> wrote: Hi all, I'm working on a project that deals with HDF4 files provided by NASA. This is part of a big project in University of Minnesota (http://spatialhadoop.cs.umn.edu/) that uses Hadoop as a distributed environment to process large data including satellite data. I used to use HDFJava wrappers to deal with HDF files and it worked OK. However, since it depends on native libraries, I frequently have troubles distributing the code over a heterogeneous cluster of different architectures and systems. So, I decided to write my own Java-native code to open HDF files. Currently, I'm just writing the minimal code to support the files I'm working on. I followed the specifications provided on the website and I was successfully able to get the structure of the HDF file and reach the data blocks that contain the main data (temperature in my case). These blocks are compressed using deflate with level 1 as indicated by the file. However, when I try to decompress these blocks, they do not give me the expected size. I feel that there's some kind of incompatible decompression parameters that I need to fix. Here's the code I wrote to decompress the file byte[] compressedData = readRawData(in); // Reads the raw data from the block DeflaterInputStream dis = new DeflaterInputStream(new ByteArrayInputStream(compressedData), new Deflater(1, true)); // Creates a decompressor ByteArrayOutputStream baos = new ByteArrayOutputStream(); // A temporary place to store decompressed data // This loop reads the uncompressed data and caches it in memory byte[] buffer = new byte[4096]; int bufferLength; while ((bufferLength = dis.read(buffer)) > 0) { baos.write(buffer, 0, bufferLength); } dis.close(); baos.close(); // Retrieve the decompressed data uncompressedData = baos.toByteArray(); The header of the compressed data indicate that the size after decompression should be 2880000 bytes, but my code finds that it is 1613097, which is totally different. When creating the deflater, I tried to set the 'nowrap' parameter to true or false but both do not work. The same file works with HDFJava libraries so it should be a bug in my code. Thanks in advance for your help. Please let me know if you need more details about this issue. Best regards, Ahmed Eldawy
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
