PigStorage() - need ability to turn off "Attempt to access field" warnings
---------------------------------------------------------------------------
Key: PIG-784
URL: https://issues.apache.org/jira/browse/PIG-784
Project: Pig
Issue Type: Bug
Reporter: David Ciemiewicz
I want an option to PigStorage() for LOAD which will allow me to turn off the
"Attempt to access field" warnings.
Something like:
{code}
define PigStorage PigStorage("warn_load_nonexistent_field=off");
A = load 'mydata.txt' using PigStorage()
as (col1: chararray, col2_optional: int, col3_optional: float);
{code}
or
{code}
A = load 'mydata.txt' using PigStorage("warn_load_nonexistent_field=0")
as (col1: chararray, col2_optional: int, col3_optional: float);
{code}
If I have a very large data set with optional columns that are not populated
(and have no tab separator), I'd like to just read the file as is and not
generate the warnings.
The warnings are problematic because the fill up the logging output and every
System.out.println will generate slow down the overall processing. Especially
if the data file being processed is missing one or more columns on every single
row.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
