[jira] [Updated] (HDFS-16283) RBF: improve renewLease() to call only a specific NameNode rather than make fan-out calls

[ https://issues.apache.org/jira/browse/HDFS-16283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HDFS-16283: Attachment: RBF_ improve renewLease() to call only a specific NameNode rather than make fan-out calls.pdf

[jira] [Commented] (HDFS-16283) RBF: improve renewLease() to call only a specific NameNode rather than make fan-out calls

[ https://issues.apache.org/jira/browse/HDFS-16283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435067#comment-17435067 ] Aihua Xu commented on HDFS-16283: - [~jingzhao] and [~inigoiri] Can you help review the change? > RBF:

[jira] [Created] (HDFS-16283) RBF: improve renewLease() to call only a specific NameNode rather than make fan-out calls

Aihua Xu created HDFS-16283: --- Summary: RBF: improve renewLease() to call only a specific NameNode rather than make fan-out calls Key: HDFS-16283 URL: https://issues.apache.org/jira/browse/HDFS-16283

[jira] [Commented] (HDFS-16200) Improve NameNode failover

[ https://issues.apache.org/jira/browse/HDFS-16200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415781#comment-17415781 ] Aihua Xu commented on HDFS-16200: - [~hexiaoqiao] Thanks for checking. Regarding improving topology

[jira] [Created] (HDFS-16200) Improve NameNode failover

Aihua Xu created HDFS-16200: --- Summary: Improve NameNode failover Key: HDFS-16200 URL: https://issues.apache.org/jira/browse/HDFS-16200 Project: Hadoop HDFS Issue Type: Task Components:

[jira] [Created] (HDFS-16159) Support resolvePath() in DistributedFileSystem for federated HDFS

Aihua Xu created HDFS-16159: --- Summary: Support resolvePath() in DistributedFileSystem for federated HDFS Key: HDFS-16159 URL: https://issues.apache.org/jira/browse/HDFS-16159 Project: Hadoop HDFS

[jira] [Commented] (HDFS-15905) Improve Router performance with router redirection

[ https://issues.apache.org/jira/browse/HDFS-15905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17304294#comment-17304294 ] Aihua Xu commented on HDFS-15905: - [~elgoiri], [~jingzhao], [~fengnanli] Can you provide any

[jira] [Created] (HDFS-15905) Improve Router performance with router redirection

Aihua Xu created HDFS-15905: --- Summary: Improve Router performance with router redirection Key: HDFS-15905 URL: https://issues.apache.org/jira/browse/HDFS-15905 Project: Hadoop HDFS Issue Type: New

[jira] [Updated] (HDFS-15800) DataNode to handle NameNode IP changes

[ https://issues.apache.org/jira/browse/HDFS-15800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HDFS-15800: Assignee: Aihua Xu Status: Patch Available (was: Open) Simple change to update remoteId.address

[jira] [Updated] (HDFS-15800) DataNode to handle NameNode IP changes

[ https://issues.apache.org/jira/browse/HDFS-15800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HDFS-15800: Attachment: HDFS-15800.patch > DataNode to handle NameNode IP changes >

[jira] [Created] (HDFS-15800) DataNode to handle NameNode IP changes

Aihua Xu created HDFS-15800: --- Summary: DataNode to handle NameNode IP changes Key: HDFS-15800 URL: https://issues.apache.org/jira/browse/HDFS-15800 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Updated] (HDFS-15727) RpcQueueTimeAvgTime of the NameNode increases after it becomes StandBy

[ https://issues.apache.org/jira/browse/HDFS-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HDFS-15727: Description: RpcQueueTimeAvgTime of the NameNode on port 8020 (the client RPC calls) increases after it

[jira] [Commented] (HDFS-15727) RpcQueueTimeAvgTime of the NameNode increases after it becomes StandBy

[ https://issues.apache.org/jira/browse/HDFS-15727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248080#comment-17248080 ] Aihua Xu commented on HDFS-15727: - [~kihwal] Seems it doesn't cause issues but wrong metrics. BTW: this

[jira] [Created] (HDFS-15727) RpcQueueTimeAvgTime of the NameNode increases after it becomes StandBy

Aihua Xu created HDFS-15727: --- Summary: RpcQueueTimeAvgTime of the NameNode increases after it becomes StandBy Key: HDFS-15727 URL: https://issues.apache.org/jira/browse/HDFS-15727 Project: Hadoop HDFS

[jira] [Commented] (HDFS-15562) StandbyCheckpointer will do checkpoint repeatedly while connecting observer/active namenode failed

[ https://issues.apache.org/jira/browse/HDFS-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17233178#comment-17233178 ] Aihua Xu commented on HDFS-15562: - Thanks [~shv] for your comment. When I get time, I will focus on not

[jira] [Commented] (HDFS-15467) ObserverReadProxyProvider should skip logging first failover from each proxy

[ https://issues.apache.org/jira/browse/HDFS-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17229783#comment-17229783 ] Aihua Xu commented on HDFS-15467: - [~csun] It doesn't fail but print the info indicating failover message

[jira] [Comment Edited] (HDFS-15467) ObserverReadProxyProvider should skip logging first failover from each proxy

[ https://issues.apache.org/jira/browse/HDFS-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228899#comment-17228899 ] Aihua Xu edited comment on HDFS-15467 at 11/10/20, 1:33 AM: [~csun] In

[jira] [Comment Edited] (HDFS-15467) ObserverReadProxyProvider should skip logging first failover from each proxy

[ https://issues.apache.org/jira/browse/HDFS-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228899#comment-17228899 ] Aihua Xu edited comment on HDFS-15467 at 11/10/20, 1:33 AM: [~csun] In

[jira] [Comment Edited] (HDFS-15467) ObserverReadProxyProvider should skip logging first failover from each proxy

[ https://issues.apache.org/jira/browse/HDFS-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228899#comment-17228899 ] Aihua Xu edited comment on HDFS-15467 at 11/10/20, 1:32 AM: [~csun] In

[jira] [Commented] (HDFS-15467) ObserverReadProxyProvider should skip logging first failover from each proxy

[ https://issues.apache.org/jira/browse/HDFS-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17228899#comment-17228899 ] Aihua Xu commented on HDFS-15467: - [~csun] In ObserverReadProxyProvider, the FailoverProxyProvider

[jira] [Commented] (HDFS-15597) ContentSummary.getSpaceConsumed does not consider replication

[ https://issues.apache.org/jira/browse/HDFS-15597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17227882#comment-17227882 ] Aihua Xu commented on HDFS-15597: - Thanks for reviewing [~weichiu] and [~ayushtkn]. Let me take a look to

[jira] [Commented] (HDFS-15467) ObserverReadProxyProvider should skip logging first failover from each proxy

[ https://issues.apache.org/jira/browse/HDFS-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17225120#comment-17225120 ] Aihua Xu commented on HDFS-15467: - I will take a look. > ObserverReadProxyProvider should skip logging

[jira] [Assigned] (HDFS-15467) ObserverReadProxyProvider should skip logging first failover from each proxy

[ https://issues.apache.org/jira/browse/HDFS-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu reassigned HDFS-15467: --- Assignee: Aihua Xu > ObserverReadProxyProvider should skip logging first failover from each proxy >

[jira] [Resolved] (HDFS-15664) Prevent Observer NameNode from becoming StandBy NameNode

[ https://issues.apache.org/jira/browse/HDFS-15664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu resolved HDFS-15664. - Resolution: Duplicate > Prevent Observer NameNode from becoming StandBy NameNode >

[jira] [Commented] (HDFS-15664) Prevent Observer NameNode from becoming StandBy NameNode

[ https://issues.apache.org/jira/browse/HDFS-15664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17224985#comment-17224985 ] Aihua Xu commented on HDFS-15664: - Didn't know it was fixed in another place. Thanks [~csun] > Prevent

[jira] [Commented] (HDFS-15664) Prevent Observer NameNode from becoming StandBy NameNode

[ https://issues.apache.org/jira/browse/HDFS-15664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17224828#comment-17224828 ] Aihua Xu commented on HDFS-15664: - + [~sunchao] Comments on this? > Prevent Observer NameNode from

[jira] [Created] (HDFS-15664) Prevent Observer NameNode from becoming StandBy NameNode

Aihua Xu created HDFS-15664: --- Summary: Prevent Observer NameNode from becoming StandBy NameNode Key: HDFS-15664 URL: https://issues.apache.org/jira/browse/HDFS-15664 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-15562) StandbyCheckpointer will do checkpoint repeatedly while connecting observer/active namenode failed

[ https://issues.apache.org/jira/browse/HDFS-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223287#comment-17223287 ] Aihua Xu commented on HDFS-15562: - [~weichiu], [~csun] Can you help review the change? Thanks. >

[jira] [Updated] (HDFS-15562) StandbyCheckpointer will do checkpoint repeatedly while connecting observer/active namenode failed

[ https://issues.apache.org/jira/browse/HDFS-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HDFS-15562: Status: Patch Available (was: Open) patch-1: Standby NameNode will do the checkpoint and upload the image

[jira] [Updated] (HDFS-15562) StandbyCheckpointer will do checkpoint repeatedly while connecting observer/active namenode failed

[ https://issues.apache.org/jira/browse/HDFS-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HDFS-15562: Attachment: HDFS-15562.patch > StandbyCheckpointer will do checkpoint repeatedly while connecting >

[jira] [Updated] (HDFS-14327) Using FQDN instead of IP to access servers with DNS resolving

[ https://issues.apache.org/jira/browse/HDFS-14327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HDFS-14327: Description: *strong text*With [HDFS-14118|https://issues.apache.org/jira/browse/HDFS-14118], clients can

[jira] [Updated] (HDFS-14327) Using FQDN instead of IP to access servers with DNS resolving

[ https://issues.apache.org/jira/browse/HDFS-14327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HDFS-14327: Description: With [HDFS-14118|https://issues.apache.org/jira/browse/HDFS-14118], clients can get the IP

[jira] [Commented] (HDFS-15601) Batch listing: gracefully fallback to use non-batched listing when NameNode doesn't support the feature

[ https://issues.apache.org/jira/browse/HDFS-15601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17211161#comment-17211161 ] Aihua Xu commented on HDFS-15601: - Seems I won't have time to work on this. Assign back. > Batch

[jira] [Assigned] (HDFS-15601) Batch listing: gracefully fallback to use non-batched listing when NameNode doesn't support the feature

[ https://issues.apache.org/jira/browse/HDFS-15601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu reassigned HDFS-15601: --- Assignee: (was: Aihua Xu) > Batch listing: gracefully fallback to use non-batched listing when

[jira] [Assigned] (HDFS-15601) Batch listing: gracefully fallback to use non-batched listing when NameNode doesn't support the feature

[ https://issues.apache.org/jira/browse/HDFS-15601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu reassigned HDFS-15601: --- Assignee: Aihua Xu > Batch listing: gracefully fallback to use non-batched listing when NameNode >

[jira] [Commented] (HDFS-15601) Batch listing: gracefully fallback to use non-batched listing when NameNode doesn't support the feature

[ https://issues.apache.org/jira/browse/HDFS-15601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17207801#comment-17207801 ] Aihua Xu commented on HDFS-15601: - I will take a look. > Batch listing: gracefully fallback to use

[jira] [Commented] (HDFS-15562) StandbyCheckpointer will do checkpoint repeatedly while connecting observer/active namenode failed

[ https://issues.apache.org/jira/browse/HDFS-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206838#comment-17206838 ] Aihua Xu commented on HDFS-15562: - [~aswqazxsd] I will take a look. If you can provide more details like

[jira] [Assigned] (HDFS-15562) StandbyCheckpointer will do checkpoint repeatedly while connecting observer/active namenode failed

[ https://issues.apache.org/jira/browse/HDFS-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu reassigned HDFS-15562: --- Assignee: Aihua Xu > StandbyCheckpointer will do checkpoint repeatedly while connecting >

[jira] [Commented] (HDFS-15597) ContentSummary.getSpaceConsumed does not consider replication

[ https://issues.apache.org/jira/browse/HDFS-15597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206583#comment-17206583 ] Aihua Xu commented on HDFS-15597: - [~weichiu] Can you review the simple fix? Thanks. >

[jira] [Updated] (HDFS-15597) ContentSummary.getSpaceConsumed does not consider replication

[ https://issues.apache.org/jira/browse/HDFS-15597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HDFS-15597: Status: Patch Available (was: Open) Patch-1: update getContentSummary function to consider replication

[jira] [Updated] (HDFS-15597) ContentSummary.getSpaceConsumed does not consider replication

[ https://issues.apache.org/jira/browse/HDFS-15597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HDFS-15597: Attachment: HDFS-15597.patch > ContentSummary.getSpaceConsumed does not consider replication >

[jira] [Commented] (HDFS-15597) ContentSummary.getSpaceConsumed does not consider replication

[ https://issues.apache.org/jira/browse/HDFS-15597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206492#comment-17206492 ] Aihua Xu commented on HDFS-15597: - Let me take a look. > ContentSummary.getSpaceConsumed does not

[jira] [Assigned] (HDFS-15597) ContentSummary.getSpaceConsumed does not consider replication

[ https://issues.apache.org/jira/browse/HDFS-15597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu reassigned HDFS-15597: --- Assignee: Aihua Xu > ContentSummary.getSpaceConsumed does not consider replication >

[jira] [Assigned] (HDFS-5389) A Namenode that keeps only a part of the namespace in memory

[ https://issues.apache.org/jira/browse/HDFS-5389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu reassigned HDFS-5389: -- Assignee: Aihua Xu (was: Haohui Mai) > A Namenode that keeps only a part of the namespace in memory >

[jira] [Commented] (HDFS-5389) A Namenode that keeps only a part of the namespace in memory

[ https://issues.apache.org/jira/browse/HDFS-5389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206394#comment-17206394 ] Aihua Xu commented on HDFS-5389: Seems no activity on this but sounds a great area to help scale NN and

[jira] [Commented] (HDFS-14450) Erasure Coding: decommissioning datanodes cause replicate a large number of duplicate EC internal blocks

[ https://issues.apache.org/jira/browse/HDFS-14450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824561#comment-16824561 ] Aihua Xu commented on HDFS-14450: - [~wuweiwei] Is this issue always reproducible? > Erasure Coding:

[jira] [Created] (HDFS-14364) DataNode crashes without any workload

Aihua Xu created HDFS-14364: --- Summary: DataNode crashes without any workload Key: HDFS-14364 URL: https://issues.apache.org/jira/browse/HDFS-14364 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-14363) Namenode crashes without any workload

Aihua Xu created HDFS-14363: --- Summary: Namenode crashes without any workload Key: HDFS-14363 URL: https://issues.apache.org/jira/browse/HDFS-14363 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Updated] (HDFS-14362) HDFS cluster with EC enabled crashes without any workload

[ https://issues.apache.org/jira/browse/HDFS-14362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HDFS-14362: Summary: HDFS cluster with EC enabled crashes without any workload (was: HDFS cluster crashes without any

[jira] [Created] (HDFS-14362) HDFS cluster crashes without any workload

Aihua Xu created HDFS-14362: --- Summary: HDFS cluster crashes without any workload Key: HDFS-14362 URL: https://issues.apache.org/jira/browse/HDFS-14362 Project: Hadoop HDFS Issue Type: Bug

[jira] [Assigned] (HDFS-10586) Erasure Code misfunctions when 3 DataNode down

[ https://issues.apache.org/jira/browse/HDFS-10586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu reassigned HDFS-10586: --- Assignee: Aihua Xu > Erasure Code misfunctions when 3 DataNode down >

[jira] [Commented] (HDFS-10586) Erasure Code misfunctions when 3 DataNode down

[ https://issues.apache.org/jira/browse/HDFS-10586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739908#comment-16739908 ] Aihua Xu commented on HDFS-10586: - I tested out this briefly and it doesn't seem to be an issue any more.

[jira] [Commented] (HDFS-14190) Copying folders containing = - characters between hdfs (using webhdfs) does not work in distcp

[ https://issues.apache.org/jira/browse/HDFS-14190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737572#comment-16737572 ] Aihua Xu commented on HDFS-14190: - I will take a look. > Copying folders containing = - characters

[jira] [Assigned] (HDFS-14190) Copying folders containing = - characters between hdfs (using webhdfs) does not work in distcp

[ https://issues.apache.org/jira/browse/HDFS-14190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu reassigned HDFS-14190: --- Assignee: Aihua Xu > Copying folders containing = - characters between hdfs (using webhdfs) does >

[jira] [Commented] (HDFS-10815) The state of the EC file is erroneously recognized when you restart the NameNode.

[ https://issues.apache.org/jira/browse/HDFS-10815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736229#comment-16736229 ] Aihua Xu commented on HDFS-10815: - Along with HDFS-10775, I will try out for the scenario and close out

[jira] [Assigned] (HDFS-10815) The state of the EC file is erroneously recognized when you restart the NameNode.

[ https://issues.apache.org/jira/browse/HDFS-10815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu reassigned HDFS-10815: --- Assignee: Aihua Xu > The state of the EC file is erroneously recognized when you restart the >

[jira] [Commented] (HDFS-10775) Under-Replicated Blocks can not be recovered

[ https://issues.apache.org/jira/browse/HDFS-10775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736221#comment-16736221 ] Aihua Xu commented on HDFS-10775: - Not sure if it's still an issue since it's a very old one. I will take

[jira] [Assigned] (HDFS-10775) Under-Replicated Blocks can not be recovered

[ https://issues.apache.org/jira/browse/HDFS-10775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu reassigned HDFS-10775: --- Assignee: Aihua Xu > Under-Replicated Blocks can not be recovered >

[jira] [Commented] (HDFS-13293) RBF: The RouterRPCServer should transfer CallerContext and client ip to NamenodeRpcServer

[ https://issues.apache.org/jira/browse/HDFS-13293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733707#comment-16733707 ] Aihua Xu commented on HDFS-13293: - [~ferhui] How is the progress on this jira? We are also interested in

[jira] [Commented] (HDFS-12953) XORRawDecoder.doDecode throws NullPointerException

[ https://issues.apache.org/jira/browse/HDFS-12953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733614#comment-16733614 ] Aihua Xu commented on HDFS-12953: - [~xiaochen] Are you working on this? I can investigate the issue

[jira] [Commented] (HDFS-14098) use fsck tools for EC blockId will throw NullPointerException

[ https://issues.apache.org/jira/browse/HDFS-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733607#comment-16733607 ] Aihua Xu commented on HDFS-14098: - [~luoge123] Are you working on the fix? We are also seeing the same

[jira] [Commented] (HDFS-10586) Erasure Code misfunctions when 3 DataNode down

[ https://issues.apache.org/jira/browse/HDFS-10586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662600#comment-16662600 ] Aihua Xu commented on HDFS-10586: - [~gaoshbj] This is a very old jira. I'm wondering if you have any

[jira] [Created] (HDFS-10385) LocalFileSystem rename() function should return false when destination file exists

Aihua Xu created HDFS-10385: --- Summary: LocalFileSystem rename() function should return false when destination file exists Key: HDFS-10385 URL: https://issues.apache.org/jira/browse/HDFS-10385 Project: