The difference between PIG and Hive seems to be pretty insignificant.
Difference between Pig and Hive is significant, specifically:
(1) Pig doesn't require underlying structure to the data, Hive does
imply structure via a metastore. This has it pros and cons. It allows
Pig to be more suitable for ETL kind tasks where the input data is still
a mish-mash and you want to convert it to be structured. On the other
hand, Hive's metastore provides a dictionary that lets you easily see
what columns exist in which tables which can be very handy.
(2) Pig is a new language, easy to learn if you know languages similar
to Perl. Hive is a sub-set of SQL with very simple variations to enable
map-reduce like computation. So, if you come from a SQL background you
will find Hive QL extremely easy to pickup (many of your SQL queries
will run as is), while if you come from a procedural programming
background (w/o SQL knowledge) then Pig will be much more suitable for
you. Furthermore, Hive is a bit easier to integrate with other systems
and tools since it speaks the language they already speak (i.e. SQL).
You're right that HBase is a completely different game, HBase is not
about being a high level language that compiles to map-reduce, HBase is
about allowing Hadoop to support lookups/transactions on key/value
pairs. HBase allows you to (1) do quick random lookups, versus scan all
of data sequentially, (2) do insert/update/delete from middle, not just
add/append.
-- amr
Ricky Ho wrote:
Jeff,
Thanks for the pointer.
It is pretty clear that Hive and PIG are the same kind and HBase is a different
kind.
The difference between PIG and Hive seems to be pretty insignificant. Layer a
tool on top of them can completely hide their difference.
I am viewing your PIG and Hive tutorial and hopefully can extract some
technical details there.
Rgds,
Ricky
-----Original Message-----
From: Jeff Hammerbacher [mailto:ham...@cloudera.com]
Sent: Wednesday, May 06, 2009 1:38 PM
To: core-user@hadoop.apache.org
Subject: Re: PIG and Hive
Here's a permalink for the thread on MarkMail:
http://markmail.org/thread/ee4hpcji74higqvk
On Wed, May 6, 2009 at 4:55 AM, Sharad Agarwal <shara...@yahoo-inc.com>wrote:
see core-user mail thread with subject "HBase, Hive, Pig and other Hadoop
based technologies"
- Sharad
Ricky Ho wrote:
Are they competing technologies of providing a higher level language for
Map/Reduce programming ?
Or are they complementary ?
Any comparison between them ?
Rgds,
Ricky
