[ 
https://issues.apache.org/jira/browse/HDFS-8782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinayakumar B resolved HDFS-8782.
---------------------------------
    Resolution: Duplicate

Resolved as dup of HDFS-8578.
Feel free to re-open if it doesnt serve the purpose.

> Upgrade to block ID-based DN storage layout delays DN registration
> ------------------------------------------------------------------
>
>                 Key: HDFS-8782
>                 URL: https://issues.apache.org/jira/browse/HDFS-8782
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Haohui Mai
>            Priority: Critical
>
> We have seen multiple incidents at production sites that there are long 
> delays for DNs to register to the NN when upgrading to post 2.6 release.
> Further investigation shows that the DN is blocked when upgrading the storage 
> layout introduced in HDFS-6482. The new storage layout requires making up to 
> 64k directories in the underlying file system. Unfortunately the current 
> implementation calls {{mkdirs()}} sequentially and upgrades each volume in 
> sequential order.
> As a result, upgrading a DN with a lot of disks or with blocks that have 
> random block ID takes a long time (usually in hours), and the DN won't 
> register to the NN unless it finishes upgrading all the storage directory. 
> The excessive delays confuse operations and break the assumption of rolling 
> upgrades.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to