[ 
https://issues.apache.org/jira/browse/HDFS-7421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14289795#comment-14289795
 ] 

Aaron T. Myers commented on HDFS-7421:
--------------------------------------

Hey Kihwal, yes indeed, this seems like a dupe. I'll go ahead and close this 
one. Thanks for pointing that out, and thanks for filing/fixing the issue in 
HDFS-6425.

> Move processing of postponed over-replicated blocks to a background task
> ------------------------------------------------------------------------
>
>                 Key: HDFS-7421
>                 URL: https://issues.apache.org/jira/browse/HDFS-7421
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: ha, namenode
>    Affects Versions: 2.6.0
>            Reporter: Aaron T. Myers
>            Assignee: Aaron T. Myers
>
> In an HA environment, we postpone sending block invalidates to DNs until all 
> DNs holding a given block have done at least one block report to the NN after 
> it became active. When that first block report after becoming active does 
> occur, we attempt to reprocess all postponed misreplicated blocks inline with 
> the block report RPC. In the case where there are many postponed 
> misreplicated blocks, this can cause block report RPCs to take an 
> inordinately long time to complete, sometimes on the order of minutes, which 
> has the potential to tie up RPC handlers, block incoming RPCs, etc. There's 
> no need to hurriedly process all postponed misreplicated blocks so that we 
> can quickly send invalidate commands back to DNs, so let's move this 
> processing outside of the RPC handler context and into a background thread.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to