[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Summary: KeyProviderCache does not get closed when DFSCLient shutdown
(was: KeyProviderCache does not get
Lei Yang created HDFS-16518:
---
Summary: KeyProviderCache does not get closed in
ShutdownHookManager
Key: HDFS-16518
URL: https://issues.apache.org/jira/browse/HDFS-16518
Project: Hadoop HDFS
Issue
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
The cache has ttl and can close KeyProvider when cache entry is expired but we
also want to
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
The cache has ttl and can close KeyProvider when cache entry is expired but
when DFSClient
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
The cache has ttl and can close KeyProvider when cache entry is expired but
when DFSClient
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
The cache has ttl and can close KeyProvider when cache entry is expired but
when DFSClient
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
The cache has ttl and can close KeyProvider when cache entry is expired but
when DFSClient
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Summary: KeyProviderCache does not get closed when DFSClient is closed
(was: KeyProviderCache does not
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
The cache has ttl and can close KeyProvider when cache entry is expired but we
also want to
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Summary: Cached KeyProvider in KeyProviderCache does not get closed when
DFSClient is closed (was:
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
The cache has ttl and can close KeyProvider when cache entry is expired but
when DFSClient
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
The cache has ttl and can close KeyProvider when cache entry is expired but
when DFSClient
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
The cache has ttl and can close KeyProvider when cache entry is expired but
when DFSClient
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
The cache has ttl and can close KeyProvider when cache entry is expired but
when DFSClient
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
KeyProvider implements Closable interface but some custom implementation of
KeyProvider also
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Fix Version/s: 2.10.0
> KeyProviderCache close cached KeyProvider with Hadoop ShutdownHookManager
>
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
{code:java}
Class KeyProviderCache
...
public KeyProviderCache(long expiryMs) {
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
KeyProvider implements Closable interface but some custom implementation of
KeyProvider also
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
We need to make sure the underlying KeyProvider used by multiple DFSClient
instances is
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Summary: Cached KeyProvider in KeyProviderCache should be closed with
ShutdownHookManager (was: Cached
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
KeyProvider implements Closable interface but some custom impl of KeyProvider
needs explicit
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
KeyProvider implements Closable interface but some custom impl of KeyProvider
also needs
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
KeyProvider implements Closable interface but some custom implementation of
KeyProvider also
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Summary: KeyProviderCache close cached KeyProvider with Hadoop
ShutdownHookManager (was: Cached
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
KeyProvider implements Closable interface but some custom implementation of
KeyProvider also
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
KeyProvider implements Closable interface but some custom implementation of
KeyProvider also
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
KeyProvider implements Closable interface but some custom implementation of
KeyProvider also
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
KeyProvider implements Closable interface but some custom impl of KeyProvider
needs explicit
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
KeyProvider implements Closable interface but some custom impl of KeyProvider
also needs
[
https://issues.apache.org/jira/browse/HDFS-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16518:
Description:
KeyProvider implements Closable interface but some custom implementation of
KeyProvider also
Lei Yang created HDFS-16828:
---
Summary: Fsck doesn't count orphaned missing blocks.
Key: HDFS-16828
URL: https://issues.apache.org/jira/browse/HDFS-16828
Project: Hadoop HDFS
Issue Type: Bug
[
https://issues.apache.org/jira/browse/HDFS-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16828:
Description:
Fsck use corrputedFIles.size instead of blockManager.getMissinngBlockCount() to
get missing
[
https://issues.apache.org/jira/browse/HDFS-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16828:
Description:
Fsck use corrputedFIles.size instead of blockManager.getMissinngBlockCount() to
get missing
[
https://issues.apache.org/jira/browse/HDFS-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16828:
Attachment: image-2022-10-28-15-18-24-944.png
> Fsck doesn't count orphaned missing blocks.
>
[
https://issues.apache.org/jira/browse/HDFS-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16828:
Affects Version/s: 2.10.0
> Fsck doesn't count orphaned missing blocks.
>
[
https://issues.apache.org/jira/browse/HDFS-16836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634479#comment-17634479
]
Lei Yang edited comment on HDFS-16836 at 11/15/22 6:32 PM:
---
Thanks
[
https://issues.apache.org/jira/browse/HDFS-16836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634479#comment-17634479
]
Lei Yang commented on HDFS-16836:
-
Thanks [~hexiaoqiao] for taking look. docheckpoint() can fail for
Lei Yang created HDFS-16836:
---
Summary: StandbyCheckpointer can still trigger rollback fs image
after RU is finalized
Key: HDFS-16836
URL: https://issues.apache.org/jira/browse/HDFS-16836
Project: Hadoop
[
https://issues.apache.org/jira/browse/HDFS-16836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16836:
Description:
StandbyCheckpointer trigger rollback fsimage when RU is started.
When ru is started, a flag
[
https://issues.apache.org/jira/browse/HDFS-16836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16836:
Description:
StandbyCheckpointer trigger rollback fsimage when RU is started.
When ru is started, a flag
[
https://issues.apache.org/jira/browse/HDFS-16836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16836:
Summary: StandbyCheckpointer can still trigger rollback fs image after RU
is finalized, leaving checkpoint
[
https://issues.apache.org/jira/browse/HDFS-16836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-16836:
Summary: StandbyCheckpointer can still trigger rollback fs image after RU
is finalized (was:
[
https://issues.apache.org/jira/browse/HDFS-16836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634022#comment-17634022
]
Lei Yang commented on HDFS-16836:
-
[~hexiaoqiao] PR: [https://github.com/apache/hadoop/pull/5135]
I
[
https://issues.apache.org/jira/browse/HDFS-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-17290:
Description:
Clients are backoff when rpcs cannot be enqueued. However there are different
scenarios when
[
https://issues.apache.org/jira/browse/HDFS-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-17290:
Affects Version/s: 3.4.0
(was: 3.3.6)
> HDFS: add client rpc backoff metrics
[
https://issues.apache.org/jira/browse/HDFS-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-17290:
Description:
Clients are backoff when rpcs cannot be enqueued. However there are different
scenarios when
[
https://issues.apache.org/jira/browse/HDFS-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798382#comment-17798382
]
Lei Yang edited comment on HDFS-17290 at 12/18/23 11:48 PM:
[~goiri]
[
https://issues.apache.org/jira/browse/HDFS-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-17290:
Summary: HDFS: add client rpc backoff metrics due to disconnection from
lowest priority queue (was: HDFS:
[
https://issues.apache.org/jira/browse/HDFS-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798382#comment-17798382
]
Lei Yang commented on HDFS-17290:
-
[~goiri] [~simbadzina] Can you please review this and let me know if
[
https://issues.apache.org/jira/browse/HDFS-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-17290:
Description:
Clients are backoff when rpcs cannot be enqueued. However there are different
scenarios when
Lei Yang created HDFS-17290:
---
Summary: HDFS: add client rpc backoff metrics due to throttling
from lowest priority queue
Key: HDFS-17290
URL: https://issues.apache.org/jira/browse/HDFS-17290
Project:
[
https://issues.apache.org/jira/browse/HDFS-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-17290:
Fix Version/s: 3.4.0
> HDFS: add client rpc backoff metrics due to disconnection from lowest
> priority
[
https://issues.apache.org/jira/browse/HDFS-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17804483#comment-17804483
]
Lei Yang commented on HDFS-17290:
-
[~simbadzina] Thanks for reviewing and merging the PR.
> HDFS: add
[
https://issues.apache.org/jira/browse/HDFS-17341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-17341:
Description:
Some service users today in namenode like ETL, metrics collection, ad-hoc users
that are
[
https://issues.apache.org/jira/browse/HDFS-17341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-17341:
Description:
Some service users today in namenode like ETL, metrics collection accounts for
many traffic
Lei Yang created HDFS-17341:
---
Summary: Support dedicated user queues in Namenode FairCallQueue
Key: HDFS-17341
URL: https://issues.apache.org/jira/browse/HDFS-17341
Project: Hadoop HDFS
Issue
[
https://issues.apache.org/jira/browse/HDFS-17341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-17341:
Affects Version/s: 2.10.0
3.4.0
> Support dedicated user queues in Namenode
[
https://issues.apache.org/jira/browse/HDFS-17341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-17341:
Description:
Some service users today in namenode like ETL, metrics collection, ad-hoc users
that are
[
https://issues.apache.org/jira/browse/HDFS-17341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-17341:
Description:
Some service users today in namenode like ETL, metrics collection, ad-hoc users
that are
[
https://issues.apache.org/jira/browse/HDFS-17341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-17341:
Description:
Some service users today in namenode like ETL, metrics collection, ad-hoc users
that are
[
https://issues.apache.org/jira/browse/HDFS-17341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17808392#comment-17808392
]
Lei Yang commented on HDFS-17341:
-
[~hexiaoqiao] Thanks for your comment.
{quote}One concerns, we
[
https://issues.apache.org/jira/browse/HDFS-17341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Yang updated HDFS-17341:
Description:
Some service users today in namenode like ETL, metrics collection, ad-hoc users
that are
62 matches
Mail list logo
