PigStorage doesn't have an escaping mechanism at the moment. You
could create a load function that extends PigStorage and adds escaping
for field delimiters.
Alan.
On Apr 23, 2010, at 7:28 PM, Toli Kuznets wrote:
Hi,
I'm trying to read in a comma-separated file with a simple command:
a = load 'myfile' using PigStorage(',');
However, some lines in my file have the , inside a quoted string, and
Pig is picking it up as 2 separate tokens:
Example:
A,b,c,d,"string-with,comma",F
I get the "string-with" and "comma" as 2 separate tokens.
Is there a way to configure the load using PigStorage command to
respect
"escaped" delimeters? Or do I need to write my loader? Or is there a
better way of doing this?
thanks
toli
