Aggregate Warning does not work as expected on Embedding Pig in Java 0.9.1
--------------------------------------------------------------------------
Key: PIG-2425
URL: https://issues.apache.org/jira/browse/PIG-2425
Project: Pig
Issue Type: Bug
Affects Versions: 0.9.1
Reporter: Prashant Kommireddi
Fix For: 0.9.1
Attachments: aggregateWarning.patch
Property "aggregate.warning" is not being set by default when running
PigServer, embedding Pig in Java.
I was initially creating a PigServer object this way:
{code}
PigServer pigServer = new PigServer(ExecType.MAPREDUCE);
{code}
But this generated detailed logs in the log directory. To code around this on
the client-side you could do
{code}
Properties properties = PropertiesUtil.loadDefaultProperties();
properties.setProperty("aggregate.warning", "true");
PigServer pigServer = new PigServer(ExecType.MAPREDUCE, properties);
{code}
The behavior between Pig scripting and Embedded Pig should be similar. Looking
at
the main constructor of PigServer, it looks like "aggregateWarning" is set
to false if its not loaded in to Properties object.
{code}
public PigServer(PigContext context, boolean connect) throws ExecException {
this.pigContext = context;
currDAG = new Graph(false);
aggregateWarning =
"true".equalsIgnoreCase(pigContext.getProperties().getProperty("aggregate.warning"));
isMultiQuery =
"true".equalsIgnoreCase(pigContext.getProperties().getProperty("opt.multiquery","true"));
jobName = pigContext.getProperties().getProperty(
PigContext.JOB_NAME,
PigContext.JOB_NAME_PREFIX + ":DefaultJobName");
if (connect) {
pigContext.connect();
}
addJarsFromProperties();
}
{code}
I suggest adding "aggregate.warning" to Properties object of PigContext so its
picked up across all users of this property (MapReduceLauncher)
{code}
public PigServer(PigContext context, boolean connect) throws ExecException {
this.pigContext = context;
currDAG = new Graph(false);
aggregateWarning =
"true".equalsIgnoreCase(pigContext.getProperties().getProperty("aggregate.warning",
"true"));
if(aggregateWarning) {
pigContext.getProperties().setProperty("aggregate.warning",
"true");
}
isMultiQuery =
"true".equalsIgnoreCase(pigContext.getProperties().getProperty("opt.multiquery","true"));
jobName = pigContext.getProperties().getProperty(
PigContext.JOB_NAME,
PigContext.JOB_NAME_PREFIX + ":DefaultJobName");
if (connect) {
pigContext.connect();
}
addJarsFromProperties();
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira