RE: Loading data from ranges of ordered subdirs

2013-06-10 Thread Rodrick Megraw
a repeated basis and with many data sets. I will read up on it for sure. > Date: Mon, 10 Jun 2013 17:02:37 -0400 > Subject: Re: Loading data from ranges of ordered subdirs > From: pradeep...@gmail.com > To: user@pig.apache.org > > There's two possibilites that come to mind

Re: Loading data from ranges of ordered subdirs

2013-06-10 Thread Pradeep Gollakota
There's two possibilites that come to mind. 1. Write a custom LoadFunc in which you can handle these regular expressions. *Not the most ideal solution* 2. Use HCatalog. The example they have in their documentation seems to fit your use case perfectly. (http://incubator.apache.org/hcatalog/docs/r0.