Re: Partitions in pig

2012-12-18 Thread abhishek
It works for me thanks. Regards Abhi Sent from my iPhone On Dec 18, 2012, at 7:43 PM, Russell Jurney wrote: > It will work like so: > http://stackoverflow.com/questions/3515481/pig-latin-load-multiple-files-from-a-date-range-part-of-the-directory-structur > > Russell Jurney http://datasyndrom

Re: Partitions in pig

2012-12-18 Thread abhishek
Hi Russell, I will try this and get back to you. Regards Abhishek Sent from my iPhone On Dec 18, 2012, at 7:43 PM, Russell Jurney wrote: > It will work like so: > http://stackoverflow.com/questions/3515481/pig-latin-load-multiple-files-from-a-date-range-part-of-the-directory-structur > > Ru

Re: Partitions in pig

2012-12-18 Thread Russell Jurney
It will work like so: http://stackoverflow.com/questions/3515481/pig-latin-load-multiple-files-from-a-date-range-part-of-the-directory-structur Russell Jurney http://datasyndrome.com On Dec 18, 2012, at 4:27 PM, abhishek wrote: > Directory based partition in hive. > > Partition by date > > Than

Re: Partitions in pig

2012-12-18 Thread abhishek
Directory based partition in hive. Partition by date Thanks Abhi Sent from my iPhone On Dec 18, 2012, at 7:20 PM, Russell Jurney wrote: > Are you doing a directory-based partition with Hive, or are you > letting Hive's RCFile partition data for you? > > Russell Jurney http://datasyndrome.com

Re: Partitions in pig

2012-12-18 Thread Russell Jurney
Are you doing a directory-based partition with Hive, or are you letting Hive's RCFile partition data for you? Russell Jurney http://datasyndrome.com On Dec 18, 2012, at 4:12 PM, abhishek wrote: > Hi Russell, > > Thanks for the reply.How RCFile loader is related to partitions? > > I did not get

Re: Partitions in pig

2012-12-18 Thread abhishek
Hi Russell, Thanks for the reply.How RCFile loader is related to partitions? I did not get your point in this. Regards Abhi Sent from my iPhone On Dec 18, 2012, at 6:13 PM, Russell Jurney wrote: > This is what HCatalog and Pig's HCatStorage is for, to access data > from Hive from Pig. Unfort

Re: Partitions in pig

2012-12-18 Thread Cheolsoo Park
To be clear, the next CDH release is going to include HCatalog. Thanks, Cheolsoo On Tue, Dec 18, 2012 at 3:13 PM, Russell Jurney wrote: > This is what HCatalog and Pig's HCatStorage is for, to access data > from Hive from Pig. Unfortunately you are running CDH, which doesn't > support the Apach

Re: Partitions in pig

2012-12-18 Thread Russell Jurney
This is what HCatalog and Pig's HCatStorage is for, to access data from Hive from Pig. Unfortunately you are running CDH, which doesn't support the Apache HCatalog project. HDP includes Apache HCatalog: http://hortonworks.com/hdp/hdp-hcatalog-metadata-services/ More info on Apache HCatalog is avail

Partitions in pig

2012-12-18 Thread abhishek
Hi all, I have a use case which is implemented in hive with partitions. Say Customer_data/2012-12-18/ /2012-12-17/ /2012-12-16/ / / I want implement this in pig. How will partitions work