[jira] [Commented] (HDFS-14960) TesteBalancerWithNodeGroup should not succeed with DFSNetworkTopology
[
https://issues.apache.org/jira/browse/HDFS-14960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16983831#comment-16983831
]
Ayush Saxena commented on HDFS-14960:
-
Ideally we should change the test only, Improve the test in a way, that it
fails if it isn't using {{NetworkTopologyWithNodeGroup}} and refrain from
making changes in the non-test code for this.
> TesteBalancerWithNodeGroup should not succeed with DFSNetworkTopology
> -
>
> Key: HDFS-14960
> URL: https://issues.apache.org/jira/browse/HDFS-14960
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
>Affects Versions: 3.1.3
>Reporter: Jim Brennan
>Priority: Minor
>
> As reported in HDFS-14958, TestBalancerWithNodeGroup was succeeding even
> though it was using DFSNetworkTopology instead of
> NetworkTopologyWithNodeGroup.
> [~inigoiri] rightly suggested that this indicates the test is not very good -
> it should fail when run without NetworkTopologyWithNodeGroup.
> We should improve this test.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14960) TesteBalancerWithNodeGroup should not succeed with DFSNetworkTopology
[ https://issues.apache.org/jira/browse/HDFS-14960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16983809#comment-16983809 ] Íñigo Goiri commented on HDFS-14960: Let's add the check and make the test more specific so it would fail anyway without the check. > TesteBalancerWithNodeGroup should not succeed with DFSNetworkTopology > - > > Key: HDFS-14960 > URL: https://issues.apache.org/jira/browse/HDFS-14960 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Affects Versions: 3.1.3 >Reporter: Jim Brennan >Priority: Minor > > As reported in HDFS-14958, TestBalancerWithNodeGroup was succeeding even > though it was using DFSNetworkTopology instead of > NetworkTopologyWithNodeGroup. > [~inigoiri] rightly suggested that this indicates the test is not very good - > it should fail when run without NetworkTopologyWithNodeGroup. > We should improve this test. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14960) TesteBalancerWithNodeGroup should not succeed with DFSNetworkTopology
[
https://issues.apache.org/jira/browse/HDFS-14960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16983711#comment-16983711
]
hemanthboyina commented on HDFS-14960:
--
got the intention of this Jira [~Jim_Brennan] . we may need to improve the
tests.
{quote}
NetworkTopology's clusterMap should be instance of NetworkTopologyWithNodeGroup
{quote}
if this check was present , +HDFS-14958+ wouldn't have been occured .
> TesteBalancerWithNodeGroup should not succeed with DFSNetworkTopology
> -
>
> Key: HDFS-14960
> URL: https://issues.apache.org/jira/browse/HDFS-14960
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
>Affects Versions: 3.1.3
>Reporter: Jim Brennan
>Priority: Minor
>
> As reported in HDFS-14958, TestBalancerWithNodeGroup was succeeding even
> though it was using DFSNetworkTopology instead of
> NetworkTopologyWithNodeGroup.
> [~inigoiri] rightly suggested that this indicates the test is not very good -
> it should fail when run without NetworkTopologyWithNodeGroup.
> We should improve this test.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14960) TesteBalancerWithNodeGroup should not succeed with DFSNetworkTopology
[ https://issues.apache.org/jira/browse/HDFS-14960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16983598#comment-16983598 ] Jim Brennan commented on HDFS-14960: [~hemanthboyina] that does seem like a reasonable check to me, and likely would have caught the problem reported in HDFS-14958. I think the intent of this Jira is to improve the test so that it includes some test cases that are unique to NetworkTopologyWithNodeGroup. The fact that it was succeeding when it wasn't using the right class suggests that it could be improved. > TesteBalancerWithNodeGroup should not succeed with DFSNetworkTopology > - > > Key: HDFS-14960 > URL: https://issues.apache.org/jira/browse/HDFS-14960 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Affects Versions: 3.1.3 >Reporter: Jim Brennan >Priority: Minor > > As reported in HDFS-14958, TestBalancerWithNodeGroup was succeeding even > though it was using DFSNetworkTopology instead of > NetworkTopologyWithNodeGroup. > [~inigoiri] rightly suggested that this indicates the test is not very good - > it should fail when run without NetworkTopologyWithNodeGroup. > We should improve this test. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14960) TesteBalancerWithNodeGroup should not succeed with DFSNetworkTopology
[
https://issues.apache.org/jira/browse/HDFS-14960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16983200#comment-16983200
]
hemanthboyina commented on HDFS-14960:
--
[~elgoiri] [~Jim_Brennan]
I think there should be a validation in BlockPlacementPolicyWithNodeGroup.java
{code:java}
@Override
public void initialize(Configuration conf, FSClusterStats stats,
NetworkTopology clusterMap,
Host2NodesMap host2datanodeMap) {
super.initialize(conf, stats, clusterMap, host2datanodeMap);
} {code}
NetworkTopology's clusterMap should be instance of NetworkTopologyWithNodeGroup
, if it is not then we should throw the exception .
please correct me if am wrong .
> TesteBalancerWithNodeGroup should not succeed with DFSNetworkTopology
> -
>
> Key: HDFS-14960
> URL: https://issues.apache.org/jira/browse/HDFS-14960
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
>Affects Versions: 3.1.3
>Reporter: Jim Brennan
>Priority: Minor
>
> As reported in HDFS-14958, TestBalancerWithNodeGroup was succeeding even
> though it was using DFSNetworkTopology instead of
> NetworkTopologyWithNodeGroup.
> [~inigoiri] rightly suggested that this indicates the test is not very good -
> it should fail when run without NetworkTopologyWithNodeGroup.
> We should improve this test.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
