[
https://issues.apache.org/jira/browse/HDFS-16283?focusedWorklogId=787493&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-787493
]
ASF GitHub Bot logged work on HDFS-16283:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 04/Jul/22 06:16
Start Date: 04/Jul/22 06:16
Worklog Time Spent: 10m
Work Description: ZanderXu commented on PR #4524:
URL: https://github.com/apache/hadoop/pull/4524#issuecomment-1173395426
Thank @Hexiaoqiao for your solution. In the beginning, we try to carry the
writing paths to RBF to fix this issue. After running for a while, I found some
cases also need to be fixed:
- Long run client case. There may be many files being written at the same
time
- Multiple destination case. RBF always forwards the renew lease rpc to all
destination name service
Also, the number of renewLease requests between client and rbf will also
increases, depending on the number of files being written at the same time.
Issue Time Tracking
-------------------
Worklog Id: (was: 787493)
Time Spent: 4h 20m (was: 4h 10m)
> RBF: improve renewLease() to call only a specific NameNode rather than make
> fan-out calls
> -----------------------------------------------------------------------------------------
>
> Key: HDFS-16283
> URL: https://issues.apache.org/jira/browse/HDFS-16283
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: rbf
> Reporter: Aihua Xu
> Assignee: Aihua Xu
> Priority: Major
> Labels: pull-request-available
> Attachments: RBF_ improve renewLease() to call only a specific
> NameNode rather than make fan-out calls.pdf
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> Currently renewLease() against a router will make fan-out to all the
> NameNodes. Since renewLease() call is so frequent and if one of the NameNodes
> are slow, then eventually the router queues are blocked by all renewLease()
> and cause router degradation.
> We will make a change in the client side to keep track of NameNode Id in
> additional to current fileId so routers understand which NameNodes the client
> is renewing lease against.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
