Hi community,
Router uses Zookeeper to its MountTable, which is the key dataset to make
the component function. Over time more and more entries are added into the
Mount table. The current Router in memory cache from ZK often takes a lot
of time to sync. (10-15 mins to download 1Million entries in
Fengnan Li created HDFS-16487:
-
Summary: RBF: getListing uses raw mount table points
Key: HDFS-16487
URL: https://issues.apache.org/jira/browse/HDFS-16487
Project: Hadoop HDFS
Issue Type: Bug
Fengnan Li created HDFS-16486:
-
Summary: RBF: Don't override listing if there is a physical path
from subcluster
Key: HDFS-16486
URL: https://issues.apache.org/jira/browse/HDFS-16486
Project: Hadoop
Fengnan Li created HDFS-16483:
-
Summary: RBF: DataNode talk to Router requesting block info in
WebHDFS
Key: HDFS-16483
URL: https://issues.apache.org/jira/browse/HDFS-16483
Project: Hadoop HDFS
Fengnan Li created HDFS-16436:
-
Summary: RBF: CheckSafeMode before Read Operation
Key: HDFS-16436
URL: https://issues.apache.org/jira/browse/HDFS-16436
Project: Hadoop HDFS
Issue Type
[
https://issues.apache.org/jira/browse/HDFS-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fengnan Li resolved HDFS-16188.
---
Resolution: Fixed
> RBF: Router to support resolving monitored namenodes with
[
https://issues.apache.org/jira/browse/HDFS-16157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fengnan Li resolved HDFS-16157.
---
Resolution: Resolved
> Support configuring DNS record to get list of journal no
[
https://issues.apache.org/jira/browse/HDFS-15878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fengnan Li reopened HDFS-15878:
---
Reopen to change the closing status.
> RBF: Flaky test
> TestRouterWebHDFSContract
[
https://issues.apache.org/jira/browse/HDFS-15878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fengnan Li resolved HDFS-15878.
---
Resolution: Not A Problem
> RBF: Flaky test
> TestRouterWebHDFSContract
[
https://issues.apache.org/jira/browse/HDFS-15878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fengnan Li resolved HDFS-15878.
---
Resolution: Resolved
> RBF: Flaky test
> TestRouterWebHDFSContractCreate>AbstractContractC
Fengnan Li created HDFS-16005:
-
Summary: RBF: AccessControlException is counted as proxy failure
Key: HDFS-16005
URL: https://issues.apache.org/jira/browse/HDFS-16005
Project: Hadoop HDFS
Issue
Hi community,
Has someone ever tried to implement sampling logging for ipc Server? We would
like to gain more observability for all of the traffics to the Namenode.
Thanks,
Fengnan
Fengnan Li created HDFS-15833:
-
Summary: Make ObserverReadProxyProvider able to talk to DNS of
Observers
Key: HDFS-15833
URL: https://issues.apache.org/jira/browse/HDFS-15833
Project: Hadoop HDFS
Fengnan Li created HDFS-15832:
-
Summary: Using DNS to access Zookeeper cluster
Key: HDFS-15832
URL: https://issues.apache.org/jira/browse/HDFS-15832
Project: Hadoop HDFS
Issue Type: Improvement
Fengnan Li created HDFS-15831:
-
Summary: Adopt more DNS resolving for HDFS
Key: HDFS-15831
URL: https://issues.apache.org/jira/browse/HDFS-15831
Project: Hadoop HDFS
Issue Type: Improvement
Fengnan Li created HDFS-15757:
-
Summary: RBF: Improving Router Connection Management
Key: HDFS-15757
URL: https://issues.apache.org/jira/browse/HDFS-15757
Project: Hadoop HDFS
Issue Type
Fengnan Li created HDFS-15754:
-
Summary: Create packet metrics for DataNode
Key: HDFS-15754
URL: https://issues.apache.org/jira/browse/HDFS-15754
Project: Hadoop HDFS
Issue Type: Improvement
10:58 AM
To: Fengnan Li
Cc: Hdfs-dev
Subject: Re: [E] Cost Based FairCallQueue latency issue
We're internally running the patch I submitted on HDFS-14403 which was
subsequently modified by other ppl in the community, so it's possible the
community flavor may behave differently.
notice that call
taking long?
Fengnan
From: Daryn Sharp
Date: Thursday, November 5, 2020 at 10:58 AM
To: Fengnan Li
Cc: Hdfs-dev
Subject: Re: [E] Cost Based FairCallQueue latency issue
We're internally running the patch I submitted on HDFS-14403 which was
subsequently modifi
, 2020 at 9:19 AM
To: Fengnan Li
Cc: Hdfs-dev
Subject: Re: [E] Cost Based FairCallQueue latency issue
I submitted the original 2.8 cost-based FCQ patch (thanks to community members
for porting to other branches). We've been running with it since early 2019 on
all clusters. Mul
: Chen Liang
Date: Wednesday, November 4, 2020 at 12:08 PM
To: Fengnan Li
Cc: Hdfs-dev
Subject: Re: Cost Based FairCallQueue latency issue
Hi Fengnan,
We had been testing cost based faire call queue internally. We also saw latency
increase, and we are trying to debug into this issue as
Hi all,
Has someone deployed Cost Based Fair Call Queue in their production cluster? We
ran into some RPC queue latency degradation with ~30k-40k rps. I tried to debug
but didn’t find anything suspicious. It is worth mentioning there is no memory
issue coming with the extra heap usage for st
Fengnan Li created HDFS-15634:
-
Summary: Invalidate block on decommissioning DataNode after
replication
Key: HDFS-15634
URL: https://issues.apache.org/jira/browse/HDFS-15634
Project: Hadoop HDFS
Hi all,
Right now after a Datanode is decommissioned the process will stay alive, but
do blocks on this Datanode stay as they are? Or they will be deleted when
blockmanager is dealing with the excess blocks.
Thanks,
Fengnan
Fengnan Li created HDFS-15599:
-
Summary: RBF: Add API to expose resolved destinations (namespace)
in Router
Key: HDFS-15599
URL: https://issues.apache.org/jira/browse/HDFS-15599
Project: Hadoop HDFS
Fengnan Li created HDFS-15554:
-
Summary: RBF: force router check file existence before
adding/updating mount points
Key: HDFS-15554
URL: https://issues.apache.org/jira/browse/HDFS-15554
Project: Hadoop
Well deserved! Congratulations!
On 7/24/20, 8:15 AM, "Chao Sun" wrote:
Congratulations Ayush!
On Fri, Jul 24, 2020 at 6:43 AM Rahul Gupta wrote:
> Congratulations
>
> On Thu, Jul 23, 2020 at 8:02 PM HarshaKiran Reddy Boreddy <
> bharsh...@gmail.com> wrote:
>
>
[
https://issues.apache.org/jira/browse/HDFS-15447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fengnan Li resolved HDFS-15447.
---
Resolution: Resolved
> RBF: Add top owners metrics for delegation tok
Fengnan Li created HDFS-15447:
-
Summary: RBF: Add top owners metrics for delegation tokens
Key: HDFS-15447
URL: https://issues.apache.org/jira/browse/HDFS-15447
Project: Hadoop HDFS
Issue Type
Fengnan Li created HDFS-15383:
-
Summary: RBF: Disable watch in ZKDelegationSecretManager for
performance
Key: HDFS-15383
URL: https://issues.apache.org/jira/browse/HDFS-15383
Project: Hadoop HDFS
Fengnan Li created HDFS-15196:
-
Summary: RouterRpcServer getListing cannot list large dirs
correctly
Key: HDFS-15196
URL: https://issues.apache.org/jira/browse/HDFS-15196
Project: Hadoop HDFS
Fengnan Li created HDFS-14914:
-
Summary: Observer should throw StandbyException in Safemode
Key: HDFS-14914
URL: https://issues.apache.org/jira/browse/HDFS-14914
Project: Hadoop HDFS
Issue Type
Fengnan Li created HDFS-14647:
-
Summary: NPE during secure namenode startup
Key: HDFS-14647
URL: https://issues.apache.org/jira/browse/HDFS-14647
Project: Hadoop HDFS
Issue Type: Bug
Fengnan Li created HDFS-14449:
-
Summary: Expose total number of dt in jmx for KMS/Namenode
Key: HDFS-14449
URL: https://issues.apache.org/jira/browse/HDFS-14449
Project: Hadoop HDFS
Issue Type
[
https://issues.apache.org/jira/browse/HDFS-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fengnan Li resolved HDFS-1.
---
Resolution: Duplicate
> RBF: Add safemode to Router
Fengnan Li created HDFS-1:
-
Summary: Add safemode to Router UI
Key: HDFS-1
URL: https://issues.apache.org/jira/browse/HDFS-1
Project: Hadoop HDFS
Issue Type: Sub-task
Fengnan Li created HDFS-14427:
-
Summary: Optimize some testing set up logic in MiniRouterDFSCluster
Key: HDFS-14427
URL: https://issues.apache.org/jira/browse/HDFS-14427
Project: Hadoop HDFS
Fengnan Li created HDFS-14426:
-
Summary: RBF: Add delegation token total count as one of the
federation metrics
Key: HDFS-14426
URL: https://issues.apache.org/jira/browse/HDFS-14426
Project: Hadoop HDFS
Fengnan Li created HDFS-14405:
-
Summary: RBF: Client should be able to renew dt immediately after
it fetched the dt
Key: HDFS-14405
URL: https://issues.apache.org/jira/browse/HDFS-14405
Project: Hadoop
Fengnan Li created HDFS-14327:
-
Summary: Support security for DNS resolving
Key: HDFS-14327
URL: https://issues.apache.org/jira/browse/HDFS-14327
Project: Hadoop HDFS
Issue Type: Improvement
Fengnan Li created HDFS-14310:
-
Summary: Improve documents for using DNS to resolve namenodes and
routers
Key: HDFS-14310
URL: https://issues.apache.org/jira/browse/HDFS-14310
Project: Hadoop HDFS
Fengnan Li created HDFS-14239:
-
Summary: Fix the comment for getClass in Configuration
Key: HDFS-14239
URL: https://issues.apache.org/jira/browse/HDFS-14239
Project: Hadoop HDFS
Issue Type
Fengnan Li created HDFS-14118:
-
Summary: RBF: Use DNS to help resolve routers
Key: HDFS-14118
URL: https://issues.apache.org/jira/browse/HDFS-14118
Project: Hadoop HDFS
Issue Type: Sub-task
43 matches
Mail list logo
