[jira] [Commented] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292929#comment-16292929 ] Kihwal Lee commented on HDFS-12070: --- bq. the PD needs to ... tell the namenode to exclude the failed

[jira] [Commented] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292829#comment-16292829 ] Kihwal Lee commented on HDFS-12070: --- I have recently encountered two occurrences of this, all involving

[jira] [Updated] (HDFS-12907) Allow read-only access to reserved raw for non-superusers

[ https://issues.apache.org/jira/browse/HDFS-12907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12907: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.4 3.0.1

[jira] [Resolved] (HDFS-4411) Asserts are disabled in unit tests

[ https://issues.apache.org/jira/browse/HDFS-4411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee resolved HDFS-4411. -- Resolution: Invalid > Asserts are disabled in unit tests > -- > >

[jira] [Commented] (HDFS-12909) SSLConnectionConfigurator should be created only if security is enabled

[ https://issues.apache.org/jira/browse/HDFS-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16283672#comment-16283672 ] Kihwal Lee commented on HDFS-12909: --- It has more to do with the SSL setting than the security setting.

[jira] [Commented] (HDFS-12888) NameNode web UI shows stale config values after cli refresh

[ https://issues.apache.org/jira/browse/HDFS-12888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278728#comment-16278728 ] Kihwal Lee commented on HDFS-12888: --- Each service has multiple active instances of conf. I don't think

[jira] [Updated] (HDFS-11754) Make FsServerDefaults cache configurable.

[ https://issues.apache.org/jira/browse/HDFS-11754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11754: -- Fix Version/s: 2.8.4 2.9.1 2.10.0 3.1.0

[jira] [Commented] (HDFS-11754) Make FsServerDefaults cache configurable.

[ https://issues.apache.org/jira/browse/HDFS-11754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271556#comment-16271556 ] Kihwal Lee commented on HDFS-11754: --- Thanks for working on this, [~erofeev]. Thanks for reviews,

[jira] [Commented] (HDFS-11754) Make FsServerDefaults cache configurable.

[ https://issues.apache.org/jira/browse/HDFS-11754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271545#comment-16271545 ] Kihwal Lee commented on HDFS-11754: --- +1 looks good. > Make FsServerDefaults cache configurable. >

[jira] [Updated] (HDFS-11754) Make FsServerDefaults cache configurable.

[ https://issues.apache.org/jira/browse/HDFS-11754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11754: -- Target Version/s: 2.8.4 (was: 2.10.0) > Make FsServerDefaults cache configurable. >

[jira] [Commented] (HDFS-12866) Recursive delete of a large directory or snapshot makes namenode unresponsive

[ https://issues.apache.org/jira/browse/HDFS-12866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270782#comment-16270782 ] Kihwal Lee commented on HDFS-12866: --- bq. 1.1. hold the fsn write lock, disconnect the target to be

[jira] [Updated] (HDFS-12754) Lease renewal can hit a deadlock

[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12754: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Lease renewal can hit a deadlock

[jira] [Commented] (HDFS-12754) Lease renewal can hit a deadlock

[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267781#comment-16267781 ] Kihwal Lee commented on HDFS-12754: --- I've committed this to trunk, branch-3.0, branch-3.0.0, branch-2,

[jira] [Updated] (HDFS-12754) Lease renewal can hit a deadlock

[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12754: -- Fix Version/s: 2.8.4 2.9.1 > Lease renewal can hit a deadlock >

[jira] [Comment Edited] (HDFS-12754) Lease renewal can hit a deadlock

[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267734#comment-16267734 ] Kihwal Lee edited comment on HDFS-12754 at 11/27/17 10:55 PM: -- I ran

[jira] [Updated] (HDFS-12754) Lease renewal can hit a deadlock

[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12754: -- Fix Version/s: 2.10.0 > Lease renewal can hit a deadlock > - > >

[jira] [Commented] (HDFS-12754) Lease renewal can hit a deadlock

[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267734#comment-16267734 ] Kihwal Lee commented on HDFS-12754: --- I ran {{TestLeaseRecovery2}} *without* the patch and two cases

[jira] [Updated] (HDFS-12754) Lease renewal can hit a deadlock

[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12754: -- Fix Version/s: 3.0.0 > Lease renewal can hit a deadlock > - > >

[jira] [Commented] (HDFS-12754) Lease renewal can hit a deadlock

[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16263201#comment-16263201 ] Kihwal Lee commented on HDFS-12754: --- [~andrew.wang], I think it is good to add this to 3.0.0. If you

[jira] [Updated] (HDFS-12754) Lease renewal can hit a deadlock

[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12754: -- Fix Version/s: 3.1.0 > Lease renewal can hit a deadlock > - > >

[jira] [Commented] (HDFS-12754) Lease renewal can hit a deadlock

[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16263121#comment-16263121 ] Kihwal Lee commented on HDFS-12754: --- +1 looks good. > Lease renewal can hit a deadlock >

[jira] [Commented] (HDFS-12754) Lease renewal can hit a deadlock

[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16260767#comment-16260767 ] Kihwal Lee commented on HDFS-12754: --- Kicked the build manually. > Lease renewal can hit a deadlock >

[jira] [Updated] (HDFS-12771) Add genstamp and block size to metasave Corrupt blocks list

[ https://issues.apache.org/jira/browse/HDFS-12771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12771: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.1.0 3.0.0

[jira] [Commented] (HDFS-12771) Add genstamp and block size to metasave Corrupt blocks list

[ https://issues.apache.org/jira/browse/HDFS-12771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16238289#comment-16238289 ] Kihwal Lee commented on HDFS-12771: --- +1 > Add genstamp and block size to metasave Corrupt blocks list

[jira] [Commented] (HDFS-12754) Lease renewal can hit a deadlock

[ https://issues.apache.org/jira/browse/HDFS-12754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235844#comment-16235844 ] Kihwal Lee commented on HDFS-12754: --- Please take a look at HDFS-12757. > Lease renewal can hit a

[jira] [Comment Edited] (HDFS-12757) DeadLock Happened Between DFSOutputStream and LeaseRenewer when LeaseRenewer#renew SocketTimeException

[ https://issues.apache.org/jira/browse/HDFS-12757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235842#comment-16235842 ] Kihwal Lee edited comment on HDFS-12757 at 11/2/17 2:50 PM: I think it is a

[jira] [Commented] (HDFS-12757) DeadLock Happened Between DFSOutputStream and LeaseRenewer when LeaseRenewer#renew SocketTimeException

[ https://issues.apache.org/jira/browse/HDFS-12757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235842#comment-16235842 ] Kihwal Lee commented on HDFS-12757: --- I think it is a dupe of HDFS-12754. With the patch applied the

[jira] [Commented] (HDFS-12749) DN may not send block report to NN after NN restart

[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227402#comment-16227402 ] Kihwal Lee commented on HDFS-12749: --- The rpc calls are timing out because the NN is not able to serve

[jira] [Commented] (HDFS-12738) Namenode logs many "SSL renegotiate denied" warnings after enable HTTPS for HDFS

[ https://issues.apache.org/jira/browse/HDFS-12738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225248#comment-16225248 ] Kihwal Lee commented on HDFS-12738: --- How frequent is it relative to the request rate? > Namenode logs

[jira] [Comment Edited] (HDFS-12737) Thousands of sockets lingering in TIME_WAIT state due to frequent file open operations

[ https://issues.apache.org/jira/browse/HDFS-12737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222989#comment-16222989 ] Kihwal Lee edited comment on HDFS-12737 at 10/27/17 10:49 PM: -- You quoted

[jira] [Commented] (HDFS-12737) Thousands of sockets lingering in TIME_WAIT state due to frequent file open operations

[ https://issues.apache.org/jira/browse/HDFS-12737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222989#comment-16222989 ] Kihwal Lee commented on HDFS-12737: --- You quoted {{ClientDatanodeProtocol}}, so I guess they are the

[jira] [Commented] (HDFS-12737) Thousands of sockets lingering in TIME_WAIT state due to frequent file open operations

[ https://issues.apache.org/jira/browse/HDFS-12737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222981#comment-16222981 ] Kihwal Lee commented on HDFS-12737: --- If the same user (i.e. the same UGI instance) talks to the same

[jira] [Commented] (HDFS-12711) deadly hdfs test

[ https://issues.apache.org/jira/browse/HDFS-12711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16218892#comment-16218892 ] Kihwal Lee commented on HDFS-12711: --- bq. -1 on adding Skynet logic to the hdfs tests. +2 to make it +1.

[jira] [Commented] (HDFS-12638) NameNode exits due to ReplicationMonitor thread received Runtime exception in ReplicationWork#chooseTargets

[ https://issues.apache.org/jira/browse/HDFS-12638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200361#comment-16200361 ] Kihwal Lee commented on HDFS-12638: --- We have also seen blocks with "null" bc staying in the replication

[jira] [Commented] (HDFS-12614) FSPermissionChecker#getINodeAttrs() throws NPE when INodeAttributesProvider configured

[ https://issues.apache.org/jira/browse/HDFS-12614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196983#comment-16196983 ] Kihwal Lee commented on HDFS-12614: --- We've seen this too. [~daryn] and [~agresch] analyzed it. I hope

[jira] [Commented] (HDFS-10391) Always enable NameNode service RPC port

[ https://issues.apache.org/jira/browse/HDFS-10391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194920#comment-16194920 ] Kihwal Lee commented on HDFS-10391: --- As Daryn explained, using service port may not be enough to allow

[jira] [Commented] (HDFS-7964) Add support for async edit logging

[ https://issues.apache.org/jira/browse/HDFS-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193784#comment-16193784 ] Kihwal Lee commented on HDFS-7964: -- bq. Should we consider turning this on by default in newer versions of

[jira] [Commented] (HDFS-7964) Add support for async edit logging

[ https://issues.apache.org/jira/browse/HDFS-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185888#comment-16185888 ] Kihwal Lee commented on HDFS-7964: -- bq. How many handler threads are used for the 285k QPS? 100. More

[jira] [Commented] (HDFS-12084) Scheduled Count will not decrement when file is deleted before all IBR's received

[ https://issues.apache.org/jira/browse/HDFS-12084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179080#comment-16179080 ] Kihwal Lee commented on HDFS-12084: --- I was looking at the 2.8 patch. - Explicit calls to abandonBlock()

[jira] [Comment Edited] (HDFS-12371) "BlockVerificationFailures" and "BlocksVerified" show up as 0 in Datanode JMX

[ https://issues.apache.org/jira/browse/HDFS-12371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174799#comment-16174799 ] Kihwal Lee edited comment on HDFS-12371 at 9/21/17 2:03 PM: I've committed

[jira] [Updated] (HDFS-12371) "BlockVerificationFailures" and "BlocksVerified" show up as 0 in Datanode JMX

[ https://issues.apache.org/jira/browse/HDFS-12371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12371: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.1.0 2.8.3

[jira] [Commented] (HDFS-12371) "BlockVerificationFailures" and "BlocksVerified" show up as 0 in Datanode JMX

[ https://issues.apache.org/jira/browse/HDFS-12371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174762#comment-16174762 ] Kihwal Lee commented on HDFS-12371: --- +1 lgtm > "BlockVerificationFailures" and "BlocksVerified" show up

[jira] [Commented] (HDFS-12395) Support erasure coding policy operations in namenode edit log

[ https://issues.apache.org/jira/browse/HDFS-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170628#comment-16170628 ] Kihwal Lee commented on HDFS-12395: --- bq. since we don't need to support upgrade from alpha->beta In

[jira] [Commented] (HDFS-12395) Support erasure coding policy operations in namenode edit log

[ https://issues.apache.org/jira/browse/HDFS-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170598#comment-16170598 ] Kihwal Lee commented on HDFS-12395: --- You need to increment the NN layout version. > Support erasure

[jira] [Commented] (HDFS-12371) "BlockVerificationFailures" and "BlocksVerified" show up as 0 in Datanode JMX

[ https://issues.apache.org/jira/browse/HDFS-12371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170581#comment-16170581 ] Kihwal Lee commented on HDFS-12371: --- Test failures are not related. They either pass when ran locally or

[jira] [Updated] (HDFS-12371) "BlockVerificationFailures" and "BlocksVerified" show up as 0 in Datanode JMX

[ https://issues.apache.org/jira/browse/HDFS-12371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12371: -- Status: Patch Available (was: Open) > "BlockVerificationFailures" and "BlocksVerified" show up as 0 in

[jira] [Commented] (HDFS-12084) Scheduled Count will not decrement when file is deleted before all IBR's received

[ https://issues.apache.org/jira/browse/HDFS-12084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170082#comment-16170082 ] Kihwal Lee commented on HDFS-12084: --- Sorry, I will get to it today. > Scheduled Count will not

[jira] [Updated] (HDFS-12424) Datatable sorting on the Datanode Information page in the Namenode UI is broken

[ https://issues.apache.org/jira/browse/HDFS-12424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12424: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.1.0 2.8.3

[jira] [Assigned] (HDFS-12441) Suppress UnresolvedPathException in namenode log

[ https://issues.apache.org/jira/browse/HDFS-12441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reassigned HDFS-12441: - Assignee: Kihwal Lee > Suppress UnresolvedPathException in namenode log >

[jira] [Commented] (HDFS-12420) Disable Namenode format for prod clusters when data already exists

[ https://issues.apache.org/jira/browse/HDFS-12420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168194#comment-16168194 ] Kihwal Lee commented on HDFS-12420: --- Adding an extra layer of protection from mistakes is fine, but we

[jira] [Commented] (HDFS-12424) Datatable sorting on the Datanode Information page in the Namenode UI is broken

[ https://issues.apache.org/jira/browse/HDFS-12424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167932#comment-16167932 ] Kihwal Lee commented on HDFS-12424: --- +1 > Datatable sorting on the Datanode Information page in the

[jira] [Comment Edited] (HDFS-12424) Datatable sorting on the Datanode Information page in the Namenode UI is broken

[ https://issues.apache.org/jira/browse/HDFS-12424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164910#comment-16164910 ] Kihwal Lee edited comment on HDFS-12424 at 9/13/17 4:27 PM: Reattaching the

[jira] [Updated] (HDFS-12424) Datatable sorting on the Datanode Information page in the Namenode UI is broken

[ https://issues.apache.org/jira/browse/HDFS-12424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12424: -- Attachment: HDFS-12424.1.patch Reattaching the patch to kick the trunk precommit build. Unfortunately

[jira] [Updated] (HDFS-12441) Suppress UnresolvedPathException in namenode log

[ https://issues.apache.org/jira/browse/HDFS-12441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12441: -- Status: Patch Available (was: Open) > Suppress UnresolvedPathException in namenode log >

[jira] [Updated] (HDFS-12441) Suppress UnresolvedPathException in namenode log

[ https://issues.apache.org/jira/browse/HDFS-12441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12441: -- Attachment: HDFS-12441.patch > Suppress UnresolvedPathException in namenode log >

[jira] [Created] (HDFS-12441) Suppress UnresolvedPathException in namenode log

Kihwal Lee created HDFS-12441: - Summary: Suppress UnresolvedPathException in namenode log Key: HDFS-12441 URL: https://issues.apache.org/jira/browse/HDFS-12441 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-12424) Datatable sorting on the Datanode Information page in the Namenode UI is broken

[ https://issues.apache.org/jira/browse/HDFS-12424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12424: -- Target Version/s: 2.8.3 > Datatable sorting on the Datanode Information page in the Namenode UI is >

[jira] [Commented] (HDFS-12386) Add fsserver defaults call to WebhdfsFileSystem.

[ https://issues.apache.org/jira/browse/HDFS-12386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154025#comment-16154025 ] Kihwal Lee commented on HDFS-12386: --- What happens when a new WebHdfsFileSystem instance tries to talk to

[jira] [Commented] (HDFS-12372) Document the impact of HDFS-11069 (Tighten the authorization of datanode RPC)

[ https://issues.apache.org/jira/browse/HDFS-12372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147612#comment-16147612 ] Kihwal Lee commented on HDFS-12372: --- As you can see from the code, issuing command as a hdfs admin user

[jira] [Commented] (HDFS-12372) Document the impact of HDFS-11069 (Tighten the authorization of datanode RPC)

[ https://issues.apache.org/jira/browse/HDFS-12372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147594#comment-16147594 ] Kihwal Lee commented on HDFS-12372: --- You should not run datanode as a hdfs superuser. Many examples

[jira] [Commented] (HDFS-11069) Tighten the authorization of datanode RPC

[ https://issues.apache.org/jira/browse/HDFS-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147586#comment-16147586 ] Kihwal Lee commented on HDFS-11069: --- [~xkrogen]. Fixed. Once it was a convention to not include

[jira] [Updated] (HDFS-11069) Tighten the authorization of datanode RPC

[ https://issues.apache.org/jira/browse/HDFS-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11069: -- Fix Version/s: 2.8.0 2.9.0 > Tighten the authorization of datanode RPC >

[jira] [Commented] (HDFS-12371) "BlockVerificationFailures" and "BlocksVerified" show up as 0 in Datanode JMX

[ https://issues.apache.org/jira/browse/HDFS-12371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145482#comment-16145482 ] Kihwal Lee commented on HDFS-12371: --- {{BlockPoolSliceScanner}} used to call {{incrBlocksVerified()}}.

[jira] [Moved] (HDFS-12371) "BlockVerificationFailures" and "BlocksVerified" show up as 0 in Datanode JMX

[ https://issues.apache.org/jira/browse/HDFS-12371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee moved HADOOP-14806 to HDFS-12371: Affects Version/s: (was: 2.7.1) 2.7.1

[jira] [Commented] (HDFS-12136) BlockSender performance regression due to volume scanner edge case

[ https://issues.apache.org/jira/browse/HDFS-12136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16142118#comment-16142118 ] Kihwal Lee commented on HDFS-12136: --- I think the performance impact is less severe after HDFS-12157, so

[jira] [Commented] (HDFS-12084) Scheduled Count will not decrement when file is deleted before all IBR's received

[ https://issues.apache.org/jira/browse/HDFS-12084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16142110#comment-16142110 ] Kihwal Lee commented on HDFS-12084: --- I don't know if it is related, but I see reserve RBW space gets

[jira] [Updated] (HDFS-12299) Race Between update pipeline and DN Re-Registration

[ https://issues.apache.org/jira/browse/HDFS-12299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12299: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.2 3.0.0-beta1

[jira] [Commented] (HDFS-12299) Race Between update pipeline and DN Re-Registration

[ https://issues.apache.org/jira/browse/HDFS-12299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16142051#comment-16142051 ] Kihwal Lee commented on HDFS-12299: --- +1 LGTM > Race Between update pipeline and DN Re-Registration >

[jira] [Resolved] (HDFS-6964) NN fails to fix under replication leading to data loss

[ https://issues.apache.org/jira/browse/HDFS-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee resolved HDFS-6964. -- Resolution: Duplicate > NN fails to fix under replication leading to data loss >

[jira] [Commented] (HDFS-6964) NN fails to fix under replication leading to data loss

[ https://issues.apache.org/jira/browse/HDFS-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140063#comment-16140063 ] Kihwal Lee commented on HDFS-6964: -- I think this is a dupe of HDFS-11960. > NN fails to fix under

[jira] [Commented] (HDFS-12348) disable removing blocks to trash while rolling upgrade

[ https://issues.apache.org/jira/browse/HDFS-12348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140054#comment-16140054 ] Kihwal Lee commented on HDFS-12348: --- I have two general comments on the issue. 1) The blocks in the

[jira] [Commented] (HDFS-7967) Reduce the performance impact of the balancer

[ https://issues.apache.org/jira/browse/HDFS-7967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137268#comment-16137268 ] Kihwal Lee commented on HDFS-7967: -- [~djp], the progress is stalled because the replica triplet structure

[jira] [Updated] (HDFS-12278) LeaseManager operations are inefficient in 2.8.

[ https://issues.apache.org/jira/browse/HDFS-12278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12278: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.2 3.0.0-beta1

[jira] [Commented] (HDFS-12278) LeaseManager operations are inefficient in 2.8.

[ https://issues.apache.org/jira/browse/HDFS-12278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120700#comment-16120700 ] Kihwal Lee commented on HDFS-12278: --- +1 looks good. > LeaseManager operations are inefficient in 2.8. >

[jira] [Updated] (HDFS-12157) Do fsyncDirectory(..) outside of FSDataset lock

[ https://issues.apache.org/jira/browse/HDFS-12157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12157: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.7.5 2.8.2

[jira] [Commented] (HDFS-12157) Do fsyncDirectory(..) outside of FSDataset lock

[ https://issues.apache.org/jira/browse/HDFS-12157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119932#comment-16119932 ] Kihwal Lee commented on HDFS-12157: --- +1 LGTM. > Do fsyncDirectory(..) outside of FSDataset lock >

[jira] [Commented] (HDFS-12272) chooseRemoteRack() semantics broken in trunk

[ https://issues.apache.org/jira/browse/HDFS-12272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117168#comment-16117168 ] Kihwal Lee commented on HDFS-12272: --- My apologies. I was looking at a wrong code base. >

[jira] [Commented] (HDFS-11530) Use HDFS specific network topology to choose datanode in BlockPlacementPolicyDefault

[ https://issues.apache.org/jira/browse/HDFS-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117163#comment-16117163 ] Kihwal Lee commented on HDFS-11530: --- Sorry, never mind. I was looking at a wrong branch. > Use HDFS

[jira] [Resolved] (HDFS-12272) chooseRemoteRack() semantics broken in trunk

[ https://issues.apache.org/jira/browse/HDFS-12272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee resolved HDFS-12272. --- Resolution: Invalid > chooseRemoteRack() semantics broken in trunk >

[jira] [Created] (HDFS-12272) chooseRemoteRack() semantics broken in trunk

Kihwal Lee created HDFS-12272: - Summary: chooseRemoteRack() semantics broken in trunk Key: HDFS-12272 URL: https://issues.apache.org/jira/browse/HDFS-12272 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-11530) Use HDFS specific network topology to choose datanode in BlockPlacementPolicyDefault

[ https://issues.apache.org/jira/browse/HDFS-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117145#comment-16117145 ] Kihwal Lee commented on HDFS-11530: --- This breaks the existing {{chooseRandom()}} behavior. It used to

[jira] [Assigned] (HDFS-12270) Allow more spreading of replicas during block placement

[ https://issues.apache.org/jira/browse/HDFS-12270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reassigned HDFS-12270: - Assignee: Kihwal Lee > Allow more spreading of replicas during block placement >

[jira] [Created] (HDFS-12270) Allow more spreading of replicas during block placement

Kihwal Lee created HDFS-12270: - Summary: Allow more spreading of replicas during block placement Key: HDFS-12270 URL: https://issues.apache.org/jira/browse/HDFS-12270 Project: Hadoop HDFS Issue

[jira] [Comment Edited] (HDFS-12232) ConfiguredFailoverProxyProvider caches DNS results indefinitely

[ https://issues.apache.org/jira/browse/HDFS-12232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108988#comment-16108988 ] Kihwal Lee edited comment on HDFS-12232 at 8/1/17 2:28 PM: --- I believe it can be

[jira] [Commented] (HDFS-12232) ConfiguredFailoverProxyProvider caches DNS results indefinitely

[ https://issues.apache.org/jira/browse/HDFS-12232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108988#comment-16108988 ] Kihwal Lee commented on HDFS-12232: --- I believe it can be worked around by enabling ipc client level

[jira] [Updated] (HDFS-12171) Reduce IIP object allocations for inode lookup

[ https://issues.apache.org/jira/browse/HDFS-12171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12171: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.3 3.0.0-beta1

[jira] [Commented] (HDFS-12171) Reduce IIP object allocations for inode lookup

[ https://issues.apache.org/jira/browse/HDFS-12171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100247#comment-16100247 ] Kihwal Lee commented on HDFS-12171: --- +1 Looks like a straightforward optimization. > Reduce IIP object

[jira] [Updated] (HDFS-12143) Improve performance of getting and removing inode features

[ https://issues.apache.org/jira/browse/HDFS-12143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12143: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.3 3.0.0-beta1

[jira] [Commented] (HDFS-12143) Improve performance of getting and removing inode features

[ https://issues.apache.org/jira/browse/HDFS-12143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100179#comment-16100179 ] Kihwal Lee commented on HDFS-12143: --- [~shahrs87], please file a separate jira if you feel the issue is

[jira] [Updated] (HDFS-11742) Improve balancer usability after HDFS-8818

[ https://issues.apache.org/jira/browse/HDFS-11742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11742: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.3

[jira] [Commented] (HDFS-11742) Improve balancer usability after HDFS-8818

[ https://issues.apache.org/jira/browse/HDFS-11742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096282#comment-16096282 ] Kihwal Lee commented on HDFS-11742: --- Thanks [~shv] for testing and review. By default, there will be

[jira] [Commented] (HDFS-11742) Improve balancer usability after HDFS-8818

[ https://issues.apache.org/jira/browse/HDFS-11742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095263#comment-16095263 ] Kihwal Lee commented on HDFS-11742: --- bq. do you think this is a blocker for 2.7.4 - it's marked as such?

[jira] [Commented] (HDFS-12136) BlockSender performance regression due to volume scanner edge case

[ https://issues.apache.org/jira/browse/HDFS-12136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088124#comment-16088124 ] Kihwal Lee commented on HDFS-12136: --- [~jojochuang], we started seeing significant performance regression

[jira] [Comment Edited] (HDFS-12136) BlockSender performance regression due to volume scanner edge case

[ https://issues.apache.org/jira/browse/HDFS-12136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086282#comment-16086282 ] Kihwal Lee edited comment on HDFS-12136 at 7/14/17 9:21 PM: Normal serving has

[jira] [Updated] (HDFS-12140) Remove BPOfferService lock contention to get block pool id

[ https://issues.apache.org/jira/browse/HDFS-12140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12140: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.3 3.0.0-beta1

[jira] [Commented] (HDFS-12140) Remove BPOfferService lock contention to get block pool id

[ https://issues.apache.org/jira/browse/HDFS-12140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088053#comment-16088053 ] Kihwal Lee commented on HDFS-12140: --- +1. For a BPOS, the blockpool ID won't change once registers. >

[jira] [Comment Edited] (HDFS-12137) DN dataset lock should be fair

[ https://issues.apache.org/jira/browse/HDFS-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088043#comment-16088043 ] Kihwal Lee edited comment on HDFS-12137 at 7/14/17 8:56 PM: Thanks for the

[jira] [Updated] (HDFS-12137) DN dataset lock should be fair

[ https://issues.apache.org/jira/browse/HDFS-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12137: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-beta1 2.9.0

[jira] [Commented] (HDFS-12137) DN dataset lock should be fair

[ https://issues.apache.org/jira/browse/HDFS-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088022#comment-16088022 ] Kihwal Lee commented on HDFS-12137: --- +1 the patch looks good. The test failure is not related. > DN

[jira] [Commented] (HDFS-12137) DN dataset lock should be fair

[ https://issues.apache.org/jira/browse/HDFS-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087734#comment-16087734 ] Kihwal Lee commented on HDFS-12137: --- The trunk precommit failed to download the patch and failed. I

<    1   2   3   4   5   6   7   8   9   10   >