Hi, Long! Would you please list the articles that compare both of them? I'm interested in reading them, too. If Big HDF FAQs [1] is not in your list, you may want to read it first. As the FAQs document says, comparing them is not straightforward so you're not alone if you're not clear about the issue.
In my opinion, I think HDF5 is for science and Hadoop is for analytics. The presentation [2] summarizes this well: big data as output (HDF5) and big data as input (Hadoop). Not many organizations do big science other than national labs with super computers. Commercial companies want to do big data analytics and pay for Hadoop solutions without owning super computers. Here, cloud computing is ideal for companies. That explains a little bit why only few commercial applications (MATLAB, IDL, Mathematica, etc.) support HDF5. Scientists can program what they want for their simulation. Fortan and Python may suffice for them. However, I think the main reason is that the marketing effort of The HDF Group, as a non-profit, has not been strong and the learning curve of HDF5 for average users is quite steep (no easy-to-use killer application like Excel, difficult to install, complex APIs and file format specification, etc.) [1] http://www.hdfgroup.org/pubs/papers/Big_HDF_FAQs.pdf [2] http://www.csm.ornl.gov/workshops/SOS17/documents/Plimpton_sos_Mar13.pdf -- Save the Earth. Save Earth data in HDF-EOS. Save Big data in HDF. On Thu, Nov 13, 2014 at 8:47 AM, long zhao <[email protected]> wrote: > Dear, > > I used HDF5 for a project. And I also read some documents about other big > data solutions like Hadoop, and also I know there are some articles compare > both of them. > But I am still not too clear that what the issue is best for HDF5 but not > for hadoop. And why there are only few commercial application based on HDF5 > but hadoop has a lot. > > Kind regards, > Long > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
