Re: Partitions in pig

It works for me thanks. Regards Abhi Sent from my iPhone On Dec 18, 2012, at 7:43 PM, Russell Jurney wrote: > It will work like so: > http://stackoverflow.com/questions/3515481/pig-latin-load-multiple-files-from-a-date-range-part-of-the-directory-structur > > Russell Jurney http://datasyndrom

Re: Partitions in pig

Hi Russell, I will try this and get back to you. Regards Abhishek Sent from my iPhone On Dec 18, 2012, at 7:43 PM, Russell Jurney wrote: > It will work like so: > http://stackoverflow.com/questions/3515481/pig-latin-load-multiple-files-from-a-date-range-part-of-the-directory-structur > > Ru

Re: Partitions in pig

It will work like so: http://stackoverflow.com/questions/3515481/pig-latin-load-multiple-files-from-a-date-range-part-of-the-directory-structur Russell Jurney http://datasyndrome.com On Dec 18, 2012, at 4:27 PM, abhishek wrote: > Directory based partition in hive. > > Partition by date > > Than

Re: Partitions in pig

Directory based partition in hive. Partition by date Thanks Abhi Sent from my iPhone On Dec 18, 2012, at 7:20 PM, Russell Jurney wrote: > Are you doing a directory-based partition with Hive, or are you > letting Hive's RCFile partition data for you? > > Russell Jurney http://datasyndrome.com

Re: Partitions in pig

Are you doing a directory-based partition with Hive, or are you letting Hive's RCFile partition data for you? Russell Jurney http://datasyndrome.com On Dec 18, 2012, at 4:12 PM, abhishek wrote: > Hi Russell, > > Thanks for the reply.How RCFile loader is related to partitions? > > I did not get

Re: Partitions in pig

Hi Russell, Thanks for the reply.How RCFile loader is related to partitions? I did not get your point in this. Regards Abhi Sent from my iPhone On Dec 18, 2012, at 6:13 PM, Russell Jurney wrote: > This is what HCatalog and Pig's HCatStorage is for, to access data > from Hive from Pig. Unfort

Re: Partitions in pig

To be clear, the next CDH release is going to include HCatalog. Thanks, Cheolsoo On Tue, Dec 18, 2012 at 3:13 PM, Russell Jurney wrote: > This is what HCatalog and Pig's HCatStorage is for, to access data > from Hive from Pig. Unfortunately you are running CDH, which doesn't > support the Apach

Re: Partitions in pig

This is what HCatalog and Pig's HCatStorage is for, to access data from Hive from Pig. Unfortunately you are running CDH, which doesn't support the Apache HCatalog project. HDP includes Apache HCatalog: http://hortonworks.com/hdp/hdp-hcatalog-metadata-services/ More info on Apache HCatalog is avail

Partitions in pig

Hi all, I have a use case which is implemented in hive with partitions. Say Customer_data/2012-12-18/ /2012-12-17/ /2012-12-16/ / / I want implement this in pig. How will partitions work

Storing dates in LzoJsonStorage

Hi, I'm working on a pig script (and some associated UDFs) to do a little cleaning on some data, stored as JSON objects, being used by other scripts down the pipe. The JSON objects contain dates, such as: "end_time" : ISODate("2012-11-07T00:29:58.728Z"), which I've noticed that I can rea

Limit number of Streaming Programs

Hi, I have around 4 million time series. ~1000 of them had a special occurrence at some point. Now, I want to draw 10 samples for each special time-series based on a similarity comparison. What I have currently implemented is a script in Python which consumes time-series one-by-one and does a com