[
https://issues.apache.org/jira/browse/HDDS-699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16755329#comment-16755329
]
Tsz Wo Nicholas Sze commented on HDDS-699:
------------------------------------------
> ... For the NetworkTopology performance, at the beginning I thought most of
> the access are reads after the network topology is built, so a read write
> reentrant single netlock approach may not cause much performance penalty. ...
In a large cluster, datanodes keep up and down so that NetworkTopology keeps
changing. There are a large amount of NetworkTopology queries.
NetworkTopology becomes a scalability bottleneck.
Consider that Ozone is to support small objects. We can foresee that the
problem will be more worse compared to HDFS.
> If it will really cause big performance issue, then we'd better do some
> improvement. ...
When we see the problem in production clusters, it is hard to do the
improvement since it is very risky to change such critical code at that time.
Also, there are API incompatibility -- it is impossible to change NodeImpl
from mutable to immutable. So, it is now or never. :)
> Detect Ozone Network topology
> -----------------------------
>
> Key: HDDS-699
> URL: https://issues.apache.org/jira/browse/HDDS-699
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Reporter: Xiaoyu Yao
> Assignee: Sammi Chen
> Priority: Major
> Attachments: HDDS-699.00.patch, HDDS-699.01.patch
>
>
> Traditionally this has been implemented in Hadoop via script or customizable
> java class. One thing we want to add here is the flexible multi-level support
> instead of fixed levels like DC/Rack/NG/Node.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
