Regarding pointers for LZO compression in Hive and Hadoop

[Abhishek Pratap Singh]

Hi,

I m looking for some useful docs on enabling LZO on hadoop cluster. I tried
few of the blogs, but somehow its not working.
Here is my requirement.

I have a hadoop 0.20.2 and Hive 0.6. I have some tables with 1.5 TB of
data, i want to compress them using LZO and enable LZO in hive as well as
in hadoop.
Let me know if you have any useful docs or pointers for the same.


Regards,
Abhishek

Re: Regarding pointers for LZO compression in Hive and Hadoop

[Prashant Kommireddi]

http://code.google.com/p/hadoop-gpl-packing/

Thanks,
Prashant

On Wed, Dec 14, 2011 at 11:32 AM, Abhishek Pratap Singh manu.i...@gmail.com
 wrote:

 Hi,

 I m looking for some useful docs on enabling LZO on hadoop cluster. I tried
 few of the blogs, but somehow its not working.
 Here is my requirement.

 I have a hadoop 0.20.2 and Hive 0.6. I have some tables with 1.5 TB of
 data, i want to compress them using LZO and enable LZO in hive as well as
 in hadoop.
 Let me know if you have any useful docs or pointers for the same.


 Regards,
 Abhishek

Re: Regarding pointers for LZO compression in Hive and Hadoop

[James Warren]

Hi Abhishek -

(Redirecting to user@hive, bcc'ing common-user)

I found this blog to be particularly useful when incorporating Hive and LZO:

http://www.mrbalky.com/2011/02/24/hive-tables-partitions-and-lzo-compression/

And if you're having issues setting up LZO with Hadoop in general, check out

https://github.com/toddlipcon/hadoop-lzo

cheers,
-James



On Wed, Dec 14, 2011 at 11:32 AM, Abhishek Pratap Singh manu.i...@gmail.com
 wrote:

 Hi,

 I m looking for some useful docs on enabling LZO on hadoop cluster. I tried
 few of the blogs, but somehow its not working.
 Here is my requirement.

 I have a hadoop 0.20.2 and Hive 0.6. I have some tables with 1.5 TB of
 data, i want to compress them using LZO and enable LZO in hive as well as
 in hadoop.
 Let me know if you have any useful docs or pointers for the same.


 Regards,
 Abhishek