[jira] Commented: (PIG-943) Pig crash when it cannot get counter from hadoop
[ https://issues.apache.org/jira/browse/PIG-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12752673#action_12752673 ] Daniel Dai commented on PIG-943: Since there is no unit test included, I manually tested with two failed script, both pass without error message. > Pig crash when it cannot get counter from hadoop > > > Key: PIG-943 > URL: https://issues.apache.org/jira/browse/PIG-943 > Project: Pig > Issue Type: Bug > Components: impl >Affects Versions: 0.3.0 >Reporter: Daniel Dai >Assignee: Daniel Dai > Fix For: 0.4.0 > > Attachments: PIG-943-1.patch, PIG-943-2.patch, PIG-943-3.patch, > PIG-943-4.patch, PIG-943-5.patch > > > We see following call stacks in Pig: > Case 1: > Caused by: java.lang.NullPointerException > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.computeWarningAggregate(MapReduceLauncher.java:390) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:238) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) > Case 2: > Caused by: java.lang.NullPointerException > at > org.apache.pig.tools.pigstats.PigStats.accumulateMRStats(PigStats.java:150) > at > org.apache.pig.tools.pigstats.PigStats.accumulateStats(PigStats.java:91) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:192) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) > In both cases, hadoop jobs finishes without error. The cause of both problems > is RunningJob.getCounters() returns a null, and Pig do not currently check > for that. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-943) Pig crash when it cannot get counter from hadoop
[ https://issues.apache.org/jira/browse/PIG-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751676#action_12751676 ] Hadoop QA commented on PIG-943: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12418677/PIG-943-2.patch against trunk revision 811203. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/14/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/14/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/14/console This message is automatically generated. > Pig crash when it cannot get counter from hadoop > > > Key: PIG-943 > URL: https://issues.apache.org/jira/browse/PIG-943 > Project: Pig > Issue Type: Bug > Components: impl >Affects Versions: 0.3.0 >Reporter: Daniel Dai >Assignee: Daniel Dai > Fix For: 0.4.0 > > Attachments: PIG-943-1.patch, PIG-943-2.patch > > > We see following call stacks in Pig: > Case 1: > Caused by: java.lang.NullPointerException > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.computeWarningAggregate(MapReduceLauncher.java:390) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:238) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) > Case 2: > Caused by: java.lang.NullPointerException > at > org.apache.pig.tools.pigstats.PigStats.accumulateMRStats(PigStats.java:150) > at > org.apache.pig.tools.pigstats.PigStats.accumulateStats(PigStats.java:91) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:192) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) > In both cases, hadoop jobs finishes without error. The cause of both problems > is RunningJob.getCounters() returns a null, and Pig do not currently check > for that. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-943) Pig crash when it cannot get counter from hadoop
[ https://issues.apache.org/jira/browse/PIG-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751650#action_12751650 ] Pradeep Kamath commented on PIG-943: Reviewed PIG-943-2.patch. Changes look good +1 One very minor comment, the following message: "We cannot retrieve hadoop counter for .." should change to "Unable to retrieve hadoop counter for .." > Pig crash when it cannot get counter from hadoop > > > Key: PIG-943 > URL: https://issues.apache.org/jira/browse/PIG-943 > Project: Pig > Issue Type: Bug > Components: impl >Affects Versions: 0.3.0 >Reporter: Daniel Dai >Assignee: Daniel Dai > Fix For: 0.4.0 > > Attachments: PIG-943-1.patch, PIG-943-2.patch > > > We see following call stacks in Pig: > Case 1: > Caused by: java.lang.NullPointerException > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.computeWarningAggregate(MapReduceLauncher.java:390) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:238) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) > Case 2: > Caused by: java.lang.NullPointerException > at > org.apache.pig.tools.pigstats.PigStats.accumulateMRStats(PigStats.java:150) > at > org.apache.pig.tools.pigstats.PigStats.accumulateStats(PigStats.java:91) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:192) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) > In both cases, hadoop jobs finishes without error. The cause of both problems > is RunningJob.getCounters() returns a null, and Pig do not currently check > for that. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-943) Pig crash when it cannot get counter from hadoop
[ https://issues.apache.org/jira/browse/PIG-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751645#action_12751645 ] Daniel Dai commented on PIG-943: Hi, Dmitriy, Thank you for your clarification. I look up the code, if we change -1 to a Long.NaN or null, then we need to change more code. Besides, I don't see much difference if we use null instead of -1. In both cases, we use a special value to indicate something special. > Pig crash when it cannot get counter from hadoop > > > Key: PIG-943 > URL: https://issues.apache.org/jira/browse/PIG-943 > Project: Pig > Issue Type: Bug > Components: impl >Affects Versions: 0.3.0 >Reporter: Daniel Dai >Assignee: Daniel Dai > Fix For: 0.4.0 > > Attachments: PIG-943-1.patch, PIG-943-2.patch > > > We see following call stacks in Pig: > Case 1: > Caused by: java.lang.NullPointerException > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.computeWarningAggregate(MapReduceLauncher.java:390) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:238) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) > Case 2: > Caused by: java.lang.NullPointerException > at > org.apache.pig.tools.pigstats.PigStats.accumulateMRStats(PigStats.java:150) > at > org.apache.pig.tools.pigstats.PigStats.accumulateStats(PigStats.java:91) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:192) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) > In both cases, hadoop jobs finishes without error. The cause of both problems > is RunningJob.getCounters() returns a null, and Pig do not currently check > for that. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-943) Pig crash when it cannot get counter from hadoop
[ https://issues.apache.org/jira/browse/PIG-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751631#action_12751631 ] Dmitriy V. Ryaboy commented on PIG-943: --- Hi Daniel, My apologies, I worded my comment poorly. I wasn't minus-oneing the patch, I was saying that the use of -1 as a magic value is a bit hacky. I think inserting Long.NaN or null and checking for it on the other end, instead of checking for -1, is cleaner. > Pig crash when it cannot get counter from hadoop > > > Key: PIG-943 > URL: https://issues.apache.org/jira/browse/PIG-943 > Project: Pig > Issue Type: Bug > Components: impl >Affects Versions: 0.3.0 >Reporter: Daniel Dai >Assignee: Daniel Dai > Fix For: 0.4.0 > > Attachments: PIG-943-1.patch > > > We see following call stacks in Pig: > Case 1: > Caused by: java.lang.NullPointerException > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.computeWarningAggregate(MapReduceLauncher.java:390) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:238) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) > Case 2: > Caused by: java.lang.NullPointerException > at > org.apache.pig.tools.pigstats.PigStats.accumulateMRStats(PigStats.java:150) > at > org.apache.pig.tools.pigstats.PigStats.accumulateStats(PigStats.java:91) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:192) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) > In both cases, hadoop jobs finishes without error. The cause of both problems > is RunningJob.getCounters() returns a null, and Pig do not currently check > for that. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-943) Pig crash when it cannot get counter from hadoop
[ https://issues.apache.org/jira/browse/PIG-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751575#action_12751575 ] Hadoop QA commented on PIG-943: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12418545/PIG-943-1.patch against trunk revision 811203. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/13/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/13/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/13/console This message is automatically generated. > Pig crash when it cannot get counter from hadoop > > > Key: PIG-943 > URL: https://issues.apache.org/jira/browse/PIG-943 > Project: Pig > Issue Type: Bug > Components: impl >Affects Versions: 0.3.0 >Reporter: Daniel Dai >Assignee: Daniel Dai > Fix For: 0.4.0 > > Attachments: PIG-943-1.patch > > > We see following call stacks in Pig: > Case 1: > Caused by: java.lang.NullPointerException > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.computeWarningAggregate(MapReduceLauncher.java:390) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:238) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) > Case 2: > Caused by: java.lang.NullPointerException > at > org.apache.pig.tools.pigstats.PigStats.accumulateMRStats(PigStats.java:150) > at > org.apache.pig.tools.pigstats.PigStats.accumulateStats(PigStats.java:91) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:192) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) > In both cases, hadoop jobs finishes without error. The cause of both problems > is RunningJob.getCounters() returns a null, and Pig do not currently check > for that. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-943) Pig crash when it cannot get counter from hadoop
[
https://issues.apache.org/jira/browse/PIG-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751560#action_12751560
]
Pradeep Kamath commented on PIG-943:
I am wondering if we should report that we were unable to determine the number
of records/bytes written -
{code}
if (stats.getRecordsWritten()!=-1)
log.info("Records written : " + stats.getRecordsWritten());
if (stats.getBytesWritten()!=-1)
log.info("Bytes written : " + stats.getBytesWritten());
}
{code}
could change to
{code}
if(stats.getRecordsWritten()==-1) {
log.info("Records written : Unable to determine number of records
written");
} else {
log.info("Records written : stats.getRecordsWritten()");
}
if(stats.getBytesWritten()==-1) {
log.info("Bytes written : Unable to determine number of bytes written");
} else {
log.info("Bytes written : stats.getBytesWritten()");
}
{code}
If we are using -1 to indicate we did not get counter information but the
warning has occured some number of times (maybe in one of the
jobs where we did get counter information), the following function's output
message should change
:
{code}
public static void logAggregate(Map aggMap, MessageType
messageType, Log log) {
for(Enum e: aggMap.keySet()) {
Long count = aggMap.get(e);
if(count != null && count > 0) {
String message = "Encountered " + messageType + " " +
e.toString() + " " + count + " time(s).";
logMessage(message, messageType, log);
}
}
}
{code}
to
{code}
public static void logAggregate(Map aggMap, MessageType
messageType, Log log) {
for(Enum e: aggMap.keySet()) {
Long count = aggMap.get(e);
if(count != null && (count == -1 || count > 0)) {
String message = "Encountered " + messageType + " " +
e.toString() + " " + (count == -1 ? "unknown number of ": count.toString()) + "
time(s).";
logMessage(message, messageType, log);
}
}
}
{code}
This way we still warn the user about the warning message which got aggregated
but just not report the number of occurences.
To enable the above change the code in MapReduceLauncher.java should also
change to ensure we report 3 types of counts:
0 -> the warning did not occur
-1 ->the warning occured in some of the jobs but in one or more jobs we get a
null counter and hence we cannot report an accurate count
>0 -> we can accurately report the count of occurences of the warning.
> Pig crash when it cannot get counter from hadoop
>
>
> Key: PIG-943
> URL: https://issues.apache.org/jira/browse/PIG-943
> Project: Pig
> Issue Type: Bug
> Components: impl
>Affects Versions: 0.3.0
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 0.4.0
>
> Attachments: PIG-943-1.patch
>
>
> We see following call stacks in Pig:
> Case 1:
> Caused by: java.lang.NullPointerException
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.computeWarningAggregate(MapReduceLauncher.java:390)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:238)
> at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265)
> Case 2:
> Caused by: java.lang.NullPointerException
> at
> org.apache.pig.tools.pigstats.PigStats.accumulateMRStats(PigStats.java:150)
> at
> org.apache.pig.tools.pigstats.PigStats.accumulateStats(PigStats.java:91)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:192)
> at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265)
> In both cases, hadoop jobs finishes without error. The caus
[jira] Commented: (PIG-943) Pig crash when it cannot get counter from hadoop
[ https://issues.apache.org/jira/browse/PIG-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751210#action_12751210 ] Daniel Dai commented on PIG-943: Hi, Dmitriy, All this patch does is checking for nulls. The extra work in the patch is when counters object is null, we treat all counter as unknown, so we do not report confusing results to user. Which part do you think "hacky"? Also parseLong is there for a long time and currently the only problem we observe is RunningJob.getCounters is null. If this is your concern I can change it to use stringToLong. > Pig crash when it cannot get counter from hadoop > > > Key: PIG-943 > URL: https://issues.apache.org/jira/browse/PIG-943 > Project: Pig > Issue Type: Bug > Components: impl >Affects Versions: 0.3.0 >Reporter: Daniel Dai > Fix For: 0.4.0 > > Attachments: PIG-943-1.patch > > > We see following call stacks in Pig: > Case 1: > Caused by: java.lang.NullPointerException > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.computeWarningAggregate(MapReduceLauncher.java:390) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:238) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) > Case 2: > Caused by: java.lang.NullPointerException > at > org.apache.pig.tools.pigstats.PigStats.accumulateMRStats(PigStats.java:150) > at > org.apache.pig.tools.pigstats.PigStats.accumulateStats(PigStats.java:91) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:192) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) > In both cases, hadoop jobs finishes without error. The cause of both problems > is RunningJob.getCounters() returns a null, and Pig do not currently check > for that. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-943) Pig crash when it cannot get counter from hadoop
[ https://issues.apache.org/jira/browse/PIG-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751200#action_12751200 ] Dmitriy V. Ryaboy commented on PIG-943: --- Daniel, -1 is a bit of a hack. Maybe cleaner to insert and check for nulls? Also, perhaps using stringToLong from CastUtils is warranted as it's a bit safer than parseLong(). -D > Pig crash when it cannot get counter from hadoop > > > Key: PIG-943 > URL: https://issues.apache.org/jira/browse/PIG-943 > Project: Pig > Issue Type: Bug > Components: impl >Affects Versions: 0.3.0 >Reporter: Daniel Dai > Fix For: 0.4.0 > > Attachments: PIG-943-1.patch > > > We see following call stacks in Pig: > Case 1: > Caused by: java.lang.NullPointerException > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.computeWarningAggregate(MapReduceLauncher.java:390) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:238) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) > Case 2: > Caused by: java.lang.NullPointerException > at > org.apache.pig.tools.pigstats.PigStats.accumulateMRStats(PigStats.java:150) > at > org.apache.pig.tools.pigstats.PigStats.accumulateStats(PigStats.java:91) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:192) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) > In both cases, hadoop jobs finishes without error. The cause of both problems > is RunningJob.getCounters() returns a null, and Pig do not currently check > for that. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
