Re: Pig Load function with variable tuple length

2011-05-26 Thread Dmitriy Ryaboy
The simplest thing to do might be to use the simple TextLoader, and do the parsing of the lines yourself, using either the various built-in regex extraction functions, or a custom UDF. On Thu, May 26, 2011 at 9:10 AM, sridhar basam wrote: > Hey, >       I have a file similar to syslog output. It

Pig Load function with variable tuple length

2011-05-26 Thread sridhar basam
Hey, I have a file similar to syslog output. It is 1 tuple per line, space seperated, but the tuple can have variable number of arguments if you use the standard PigStorage function to load the file. The first 4 variables are always defined and have a strict format, the rest of the line i wo