[jira] [Commented] (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
[ https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233836#comment-13233836 ] Hudson commented on MAPREDUCE-1740: --- Integrated in Hadoop-Mapreduce-trunk #1025 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1025/]) MAPREDUCE-1740. NPE in getMatchingLevelForNodes when node locations are variable depth (ahmed via tucu) (Revision 1303076) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1303076 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestJobInProgress.java > NPE in getMatchingLevelForNodes when node locations are variable depth > -- > > Key: MAPREDUCE-1740 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 0.20.3 >Reporter: Todd Lipcon >Assignee: Ahmed Radwan > Attachments: MAPREDUCE-1740.patch, MAPREDUCE-1740_branch-1.0.patch, > MAPREDUCE-1740_trunk.patch, mapreduce-1740.txt > > > In getMatchingLevelForNodes, we assume that both nodes have the same "depth" > (ie number of path components). If the user provides a topology script that > assigns one node a path like /foo/bar/baz and another node a path like > /foo/blah, this function will throw an NPE. > I'm not sure if there are other places where we assume that all node > locations have a constant number of paths. If so we should check the output > of the topology script aggressively to be sure this is the case. Otherwise I > think we simply need to add && n2 != null to the while loop -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
[ https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233802#comment-13233802 ] Hudson commented on MAPREDUCE-1740: --- Integrated in Hadoop-Hdfs-trunk-Commit #1981 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1981/]) MAPREDUCE-1740. NPE in getMatchingLevelForNodes when node locations are variable depth (ahmed via tucu) (Revision 1303076) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1303076 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestJobInProgress.java > NPE in getMatchingLevelForNodes when node locations are variable depth > -- > > Key: MAPREDUCE-1740 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 0.20.3 >Reporter: Todd Lipcon >Assignee: Ahmed Radwan > Attachments: MAPREDUCE-1740.patch, MAPREDUCE-1740_branch-1.0.patch, > MAPREDUCE-1740_trunk.patch, mapreduce-1740.txt > > > In getMatchingLevelForNodes, we assume that both nodes have the same "depth" > (ie number of path components). If the user provides a topology script that > assigns one node a path like /foo/bar/baz and another node a path like > /foo/blah, this function will throw an NPE. > I'm not sure if there are other places where we assume that all node > locations have a constant number of paths. If so we should check the output > of the topology script aggressively to be sure this is the case. Otherwise I > think we simply need to add && n2 != null to the while loop -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
[ https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233786#comment-13233786 ] Hudson commented on MAPREDUCE-1740: --- Integrated in Hadoop-Common-trunk-Commit #1907 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1907/]) MAPREDUCE-1740. NPE in getMatchingLevelForNodes when node locations are variable depth (ahmed via tucu) (Revision 1303076) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1303076 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestJobInProgress.java > NPE in getMatchingLevelForNodes when node locations are variable depth > -- > > Key: MAPREDUCE-1740 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 0.20.3 >Reporter: Todd Lipcon >Assignee: Ahmed Radwan > Attachments: MAPREDUCE-1740.patch, MAPREDUCE-1740_branch-1.0.patch, > MAPREDUCE-1740_trunk.patch, mapreduce-1740.txt > > > In getMatchingLevelForNodes, we assume that both nodes have the same "depth" > (ie number of path components). If the user provides a topology script that > assigns one node a path like /foo/bar/baz and another node a path like > /foo/blah, this function will throw an NPE. > I'm not sure if there are other places where we assume that all node > locations have a constant number of paths. If so we should check the output > of the topology script aggressively to be sure this is the case. Otherwise I > think we simply need to add && n2 != null to the while loop -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
[ https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233658#comment-13233658 ] Arun C Murthy commented on MAPREDUCE-1740: -- This is nice to fix. However, we should look at HDFS too. Doing it in isolation in MR isn't sufficient. > NPE in getMatchingLevelForNodes when node locations are variable depth > -- > > Key: MAPREDUCE-1740 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 0.20.3 >Reporter: Todd Lipcon >Assignee: Ahmed Radwan > Attachments: MAPREDUCE-1740.patch, MAPREDUCE-1740_branch-1.0.patch, > MAPREDUCE-1740_trunk.patch, mapreduce-1740.txt > > > In getMatchingLevelForNodes, we assume that both nodes have the same "depth" > (ie number of path components). If the user provides a topology script that > assigns one node a path like /foo/bar/baz and another node a path like > /foo/blah, this function will throw an NPE. > I'm not sure if there are other places where we assume that all node > locations have a constant number of paths. If so we should check the output > of the topology script aggressively to be sure this is the case. Otherwise I > think we simply need to add && n2 != null to the while loop -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
[ https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233645#comment-13233645 ] Alejandro Abdelnur commented on MAPREDUCE-1740: --- +1 > NPE in getMatchingLevelForNodes when node locations are variable depth > -- > > Key: MAPREDUCE-1740 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 0.20.3 >Reporter: Todd Lipcon > Attachments: MAPREDUCE-1740.patch, MAPREDUCE-1740_branch-1.0.patch, > MAPREDUCE-1740_trunk.patch, mapreduce-1740.txt > > > In getMatchingLevelForNodes, we assume that both nodes have the same "depth" > (ie number of path components). If the user provides a topology script that > assigns one node a path like /foo/bar/baz and another node a path like > /foo/blah, this function will throw an NPE. > I'm not sure if there are other places where we assume that all node > locations have a constant number of paths. If so we should check the output > of the topology script aggressively to be sure this is the case. Otherwise I > think we simply need to add && n2 != null to the while loop -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
[ https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13232837#comment-13232837 ] Ahmed Radwan commented on MAPREDUCE-1740: - The TestWritableJobConf failure doesn't look related to this patch. > NPE in getMatchingLevelForNodes when node locations are variable depth > -- > > Key: MAPREDUCE-1740 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 0.20.3 >Reporter: Todd Lipcon > Attachments: MAPREDUCE-1740.patch, MAPREDUCE-1740_branch-1.0.patch, > MAPREDUCE-1740_trunk.patch, mapreduce-1740.txt > > > In getMatchingLevelForNodes, we assume that both nodes have the same "depth" > (ie number of path components). If the user provides a topology script that > assigns one node a path like /foo/bar/baz and another node a path like > /foo/blah, this function will throw an NPE. > I'm not sure if there are other places where we assume that all node > locations have a constant number of paths. If so we should check the output > of the topology script aggressively to be sure this is the case. Otherwise I > think we simply need to add && n2 != null to the while loop -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
[ https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230501#comment-13230501 ] Hadoop QA commented on MAPREDUCE-1740: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12518506/MAPREDUCE-1740_trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.mapred.TestWritableJobConf +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2059//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2059//console This message is automatically generated. > NPE in getMatchingLevelForNodes when node locations are variable depth > -- > > Key: MAPREDUCE-1740 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 0.20.3 >Reporter: Todd Lipcon > Attachments: MAPREDUCE-1740.patch, MAPREDUCE-1740_branch-1.0.patch, > MAPREDUCE-1740_trunk.patch, mapreduce-1740.txt > > > In getMatchingLevelForNodes, we assume that both nodes have the same "depth" > (ie number of path components). If the user provides a topology script that > assigns one node a path like /foo/bar/baz and another node a path like > /foo/blah, this function will throw an NPE. > I'm not sure if there are other places where we assume that all node > locations have a constant number of paths. If so we should check the output > of the topology script aggressively to be sure this is the case. Otherwise I > think we simply need to add && n2 != null to the while loop -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
[ https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13229566#comment-13229566 ] Todd Lipcon commented on MAPREDUCE-1740: Hi Ahmed. Can you attach a patch for trunk? We need to fix this in trunk as well, right? We can commit the 1.0 patch at the same time. > NPE in getMatchingLevelForNodes when node locations are variable depth > -- > > Key: MAPREDUCE-1740 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 0.20.3 >Reporter: Todd Lipcon > Attachments: MAPREDUCE-1740.patch, MAPREDUCE-1740_branch-1.0.patch, > mapreduce-1740.txt > > > In getMatchingLevelForNodes, we assume that both nodes have the same "depth" > (ie number of path components). If the user provides a topology script that > assigns one node a path like /foo/bar/baz and another node a path like > /foo/blah, this function will throw an NPE. > I'm not sure if there are other places where we assume that all node > locations have a constant number of paths. If so we should check the output > of the topology script aggressively to be sure this is the case. Otherwise I > think we simply need to add && n2 != null to the while loop -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
[ https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225584#comment-13225584 ] Hadoop QA commented on MAPREDUCE-1740: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517623/MAPREDUCE-1740_branch-1.0.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2019//console This message is automatically generated. > NPE in getMatchingLevelForNodes when node locations are variable depth > -- > > Key: MAPREDUCE-1740 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 0.20.3 >Reporter: Todd Lipcon > Attachments: MAPREDUCE-1740.patch, MAPREDUCE-1740_branch-1.0.patch, > mapreduce-1740.txt > > > In getMatchingLevelForNodes, we assume that both nodes have the same "depth" > (ie number of path components). If the user provides a topology script that > assigns one node a path like /foo/bar/baz and another node a path like > /foo/blah, this function will throw an NPE. > I'm not sure if there are other places where we assume that all node > locations have a constant number of paths. If so we should check the output > of the topology script aggressively to be sure this is the case. Otherwise I > think we simply need to add && n2 != null to the while loop -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
[ https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224093#comment-13224093 ] Steve Loughran commented on MAPREDUCE-1740: --- Todd -it makes it possible to view the entire map of host -> switch. If the tests can avoid that, its better because then they will all work on 1.x too. > NPE in getMatchingLevelForNodes when node locations are variable depth > -- > > Key: MAPREDUCE-1740 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 0.20.3 >Reporter: Todd Lipcon > Attachments: MAPREDUCE-1740.patch, mapreduce-1740.txt > > > In getMatchingLevelForNodes, we assume that both nodes have the same "depth" > (ie number of path components). If the user provides a topology script that > assigns one node a path like /foo/bar/baz and another node a path like > /foo/blah, this function will throw an NPE. > I'm not sure if there are other places where we assume that all node > locations have a constant number of paths. If so we should check the output > of the topology script aggressively to be sure this is the case. Otherwise I > think we simply need to add && n2 != null to the while loop -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
[ https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223859#comment-13223859 ] Todd Lipcon commented on MAPREDUCE-1740: Steve: not sure I follow what you mean about making use of getSwitchMap -- which assert are you suggesting this be used for? > NPE in getMatchingLevelForNodes when node locations are variable depth > -- > > Key: MAPREDUCE-1740 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 0.20.3 >Reporter: Todd Lipcon > Attachments: MAPREDUCE-1740.patch, mapreduce-1740.txt > > > In getMatchingLevelForNodes, we assume that both nodes have the same "depth" > (ie number of path components). If the user provides a topology script that > assigns one node a path like /foo/bar/baz and another node a path like > /foo/blah, this function will throw an NPE. > I'm not sure if there are other places where we assume that all node > locations have a constant number of paths. If so we should check the output > of the topology script aggressively to be sure this is the case. Otherwise I > think we simply need to add && n2 != null to the while loop -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
[
https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223131#comment-13223131
]
Steve Loughran commented on MAPREDUCE-1740:
---
Looks OK on a quick review; will need to test a bit more.
(trunk only) {{AbstractDNSToSwitchMapping()}} has a {{getSwitchMap()}} method
which can return the map of host->switch (or null in the base class), which
could be used for more assertions. This class isn't in branch1, so these
asserts will have to be left out there.
It also has {{dumpTopology()}} method to convert the map to a string, which can
be used for diagnostics in logs or assertions.
style issues
# please follow hadoop project spacing rules esp. in if () conditions
# use LOG over {{System.out}} even in tests
# some of the static fields used in the tests could be made final
> NPE in getMatchingLevelForNodes when node locations are variable depth
> --
>
> Key: MAPREDUCE-1740
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
>Affects Versions: 0.20.3
>Reporter: Todd Lipcon
> Attachments: MAPREDUCE-1740.patch, mapreduce-1740.txt
>
>
> In getMatchingLevelForNodes, we assume that both nodes have the same "depth"
> (ie number of path components). If the user provides a topology script that
> assigns one node a path like /foo/bar/baz and another node a path like
> /foo/blah, this function will throw an NPE.
> I'm not sure if there are other places where we assume that all node
> locations have a constant number of paths. If so we should check the output
> of the topology script aggressively to be sure this is the case. Otherwise I
> think we simply need to add && n2 != null to the while loop
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
[ https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223025#comment-13223025 ] Hadoop QA commented on MAPREDUCE-1740: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517192/MAPREDUCE-1740.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2010//console This message is automatically generated. > NPE in getMatchingLevelForNodes when node locations are variable depth > -- > > Key: MAPREDUCE-1740 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 0.20.3 >Reporter: Todd Lipcon > Attachments: MAPREDUCE-1740.patch, mapreduce-1740.txt > > > In getMatchingLevelForNodes, we assume that both nodes have the same "depth" > (ie number of path components). If the user provides a topology script that > assigns one node a path like /foo/bar/baz and another node a path like > /foo/blah, this function will throw an NPE. > I'm not sure if there are other places where we assume that all node > locations have a constant number of paths. If so we should check the output > of the topology script aggressively to be sure this is the case. Otherwise I > think we simply need to add && n2 != null to the while loop -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
[ https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13003813#comment-13003813 ] Hadoop QA commented on MAPREDUCE-1740: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12454248/mapreduce-1740.txt against trunk revision 1079072. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause tar ant target to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: -1 contrib tests. The patch failed contrib unit tests. -1 system test framework. The patch failed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/128//testReport/ Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/128//console This message is automatically generated. > NPE in getMatchingLevelForNodes when node locations are variable depth > -- > > Key: MAPREDUCE-1740 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 0.20.3, 0.21.0, 0.22.0 >Reporter: Todd Lipcon > Attachments: mapreduce-1740.txt > > > In getMatchingLevelForNodes, we assume that both nodes have the same "depth" > (ie number of path components). If the user provides a topology script that > assigns one node a path like /foo/bar/baz and another node a path like > /foo/blah, this function will throw an NPE. > I'm not sure if there are other places where we assume that all node > locations have a constant number of paths. If so we should check the output > of the topology script aggressively to be sure this is the case. Otherwise I > think we simply need to add && n2 != null to the while loop -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
[ https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13000142#comment-13000142 ] Hadoop QA commented on MAPREDUCE-1740: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12454248/mapreduce-1740.txt against trunk revision 1075216. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause tar ant target to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: -1 contrib tests. The patch failed contrib unit tests. -1 system test framework. The patch failed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/82//testReport/ Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/82//console This message is automatically generated. > NPE in getMatchingLevelForNodes when node locations are variable depth > -- > > Key: MAPREDUCE-1740 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 0.20.3, 0.21.0, 0.22.0 >Reporter: Todd Lipcon > Attachments: mapreduce-1740.txt > > > In getMatchingLevelForNodes, we assume that both nodes have the same "depth" > (ie number of path components). If the user provides a topology script that > assigns one node a path like /foo/bar/baz and another node a path like > /foo/blah, this function will throw an NPE. > I'm not sure if there are other places where we assume that all node > locations have a constant number of paths. If so we should check the output > of the topology script aggressively to be sure this is the case. Otherwise I > think we simply need to add && n2 != null to the while loop -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-1740) NPE in getMatchingLevelForNodes when node locations are variable depth
[ https://issues.apache.org/jira/browse/MAPREDUCE-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907808#action_12907808 ] Todd Lipcon commented on MAPREDUCE-1740: Attached patch fixes the NPE, though I haven't had a chance to test on an actual misconfigured cluster. Is this the right approach or should we just do a stricter job of verifying the results of the topology mapping script to ensure that all hosts are the same level? Does anyone have a legitimate use case for locality of varying depths? > NPE in getMatchingLevelForNodes when node locations are variable depth > -- > > Key: MAPREDUCE-1740 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1740 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 0.20.3, 0.21.0, 0.22.0 >Reporter: Todd Lipcon > Attachments: mapreduce-1740.txt > > > In getMatchingLevelForNodes, we assume that both nodes have the same "depth" > (ie number of path components). If the user provides a topology script that > assigns one node a path like /foo/bar/baz and another node a path like > /foo/blah, this function will throw an NPE. > I'm not sure if there are other places where we assume that all node > locations have a constant number of paths. If so we should check the output > of the topology script aggressively to be sure this is the case. Otherwise I > think we simply need to add && n2 != null to the while loop -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
