[jira] [Commented] (HDFS-11648) Lazy construct the IIP pathname

2017-04-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15966351#comment-15966351 ] Kihwal Lee commented on HDFS-11648: --- +1 simple, yet effective. > Lazy construct the IIP

[jira] [Commented] (HDFS-11626) Deprecate oiv_legacy tool

2017-04-07 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15960871#comment-15960871 ] Kihwal Lee commented on HDFS-11626: --- bq. oiv_legacy only works for fsimage at or before

[jira] [Updated] (HDFS-11609) Some blocks can be permanently lost if nodes are decommissioned while dead

2017-04-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11609: -- Attachment: HDFS-11609.branch-2.patch Attaching branch-2 version of the patch, as the trunk version was

[jira] [Updated] (HDFS-11609) Some blocks can be permanently lost if nodes are decommissioned while dead

2017-04-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11609: -- Status: Patch Available (was: Open) > Some blocks can be permanently lost if nodes are decommissioned w

[jira] [Updated] (HDFS-11609) Some blocks can be permanently lost if nodes are decommissioned while dead

2017-04-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11609: -- Attachment: HDFS-11609.trunk.patch > Some blocks can be permanently lost if nodes are decommissioned whi

[jira] [Commented] (HDFS-11609) Some blocks can be permanently lost if nodes are decommissioned while dead

2017-04-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15954245#comment-15954245 ] Kihwal Lee commented on HDFS-11609: --- bq. I have a patch with 1) and a unit test, but wan

[jira] [Commented] (HDFS-11609) Some blocks can be permanently lost if nodes are decommissioned while dead

2017-03-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15951587#comment-15951587 ] Kihwal Lee commented on HDFS-11609: --- h3. Inability to correctly guess the previous repli

[jira] [Assigned] (HDFS-11609) Some blocks can be permanently lost if nodes are decommissioned while dead

2017-03-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reassigned HDFS-11609: - Assignee: Kihwal Lee > Some blocks can be permanently lost if nodes are decommissioned while dead

[jira] [Created] (HDFS-11609) Some blocks can be permanently lost if nodes are decommissioned while dead

2017-03-31 Thread Kihwal Lee (JIRA)
Kihwal Lee created HDFS-11609: - Summary: Some blocks can be permanently lost if nodes are decommissioned while dead Key: HDFS-11609 URL: https://issues.apache.org/jira/browse/HDFS-11609 Project: Hadoop HD

[jira] [Updated] (HDFS-11592) Closing a file has a wasteful preconditions

2017-03-30 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11592: -- Target Version/s: 2.8.1 > Closing a file has a wasteful preconditions >

[jira] [Updated] (HDFS-10601) Improve log message to include hostname when the NameNode is in safemode

2017-03-16 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10601: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha3 2.8.0

[jira] [Commented] (HDFS-10601) Improve log message to include hostname when the NameNode is in safemode

2017-03-16 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15928313#comment-15928313 ] Kihwal Lee commented on HDFS-10601: --- +1 looks good. > Improve log message to include ho

[jira] [Commented] (HDFS-11417) Add datanode admin command to get the storage info.

2017-02-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15868417#comment-15868417 ] Kihwal Lee commented on HDFS-11417: --- Sorry, it is actually you who are working on both i

[jira] [Commented] (HDFS-11417) Add datanode admin command to get the storage info.

2017-02-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15868413#comment-15868413 ] Kihwal Lee commented on HDFS-11417: --- More info was added to the datanoed UI. HDFS-11375

[jira] [Commented] (HDFS-11379) DFSInputStream may infinite loop requesting block locations

2017-02-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15863850#comment-15863850 ] Kihwal Lee commented on HDFS-11379: --- The impact of this bug can vary depending on how wi

[jira] [Commented] (HDFS-11379) DFSInputStream may infinite loop requesting block locations

2017-02-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15863840#comment-15863840 ] Kihwal Lee commented on HDFS-11379: --- Thanks, I just cherry-picked it to branch-2.8.0. >

[jira] [Updated] (HDFS-11379) DFSInputStream may infinite loop requesting block locations

2017-02-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11379: -- Fix Version/s: (was: 2.8.1) 2.8.0 > DFSInputStream may infinite loop requesting b

[jira] [Updated] (HDFS-11379) DFSInputStream may infinite loop requesting block locations

2017-02-10 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11379: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.1 3.0.0-alpha3

[jira] [Commented] (HDFS-11379) DFSInputStream may infinite loop requesting block locations

2017-02-10 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15861596#comment-15861596 ] Kihwal Lee commented on HDFS-11379: --- Nicely done. Although the retry in {{readWithStrate

[jira] [Commented] (HDFS-11379) DFSInputStream may infinite loop requesting block locations

2017-02-09 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15859820#comment-15859820 ] Kihwal Lee commented on HDFS-11379: --- I will review it today. > DFSInputStream may infin

[jira] [Commented] (HDFS-11192) OOM during Quota Initialization lead to Namenode hang

2017-01-20 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832211#comment-15832211 ] Kihwal Lee commented on HDFS-11192: --- Looking at the java source, {{Thread.start0()}} is

[jira] [Commented] (HDFS-11192) OOM during Quota Initialization lead to Namenode hang

2017-01-20 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831904#comment-15831904 ] Kihwal Lee commented on HDFS-11192: --- We can create the quota init ForkJoinPool with a Un

[jira] [Commented] (HDFS-9574) Reduce client failures during datanode restart

2017-01-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15828340#comment-15828340 ] Kihwal Lee commented on HDFS-9574: -- [~arpitagarwal], please elaborate the reason for marki

[jira] [Commented] (HDFS-9409) DataNode shutdown does not guarantee full shutdown of all threads due to race condition.

2017-01-11 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819152#comment-15819152 ] Kihwal Lee commented on HDFS-9409: -- {{BlockScanner}} is the same way. It removes a thread

[jira] [Commented] (HDFS-7967) Reduce the performance impact of the balancer

2017-01-11 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818861#comment-15818861 ] Kihwal Lee commented on HDFS-7967: -- The 2.8 patch doesn't seem right. > Reduce the perfo

[jira] [Commented] (HDFS-3618) SSH fencing option may incorrectly succeed if nc (netcat) command not present

2017-01-10 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816250#comment-15816250 ] Kihwal Lee commented on HDFS-3618: -- The "-z" option is supported by the plain old BSD impl

[jira] [Commented] (HDFS-7967) Reduce the performance impact of the balancer

2017-01-10 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15815227#comment-15815227 ] Kihwal Lee commented on HDFS-7967: -- I have reviewed the patch and am fine with it. We have

[jira] [Commented] (HDFS-9083) Replication violates block placement policy.

2017-01-05 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802662#comment-15802662 ] Kihwal Lee commented on HDFS-9083: -- [~djp], that's correct. > Replication violates block

[jira] [Commented] (HDFS-7967) Reduce the performance impact of the balancer

2017-01-05 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802511#comment-15802511 ] Kihwal Lee commented on HDFS-7967: -- The 2.8 patch works fine with jdk8. The unit tests ran

[jira] [Commented] (HDFS-11160) VolumeScanner reports write-in-progress replicas as corrupt incorrectly

2017-01-05 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15801541#comment-15801541 ] Kihwal Lee commented on HDFS-11160: --- [~jojochuang], please make sure it is in 2.8. > Vo

[jira] [Commented] (HDFS-7784) load fsimage in parallel

2017-01-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15795396#comment-15795396 ] Kihwal Lee commented on HDFS-7784: -- [~xiegang112], when you have a chance to test the perf

[jira] [Commented] (HDFS-11094) Send back HAState along with NamespaceInfo during a versionRequest as an optional parameter

2016-12-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15751772#comment-15751772 ] Kihwal Lee commented on HDFS-11094: --- I am fine with the latest patch. > Send back HASta

[jira] [Resolved] (HDFS-11207) Unnecessary incompatible change of NNHAStatusHeartbeat.state in DatanodeProtocolProtos

2016-12-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee resolved HDFS-11207. --- Resolution: Done > Unnecessary incompatible change of NNHAStatusHeartbeat.state in > DatanodeProtocol

[jira] [Commented] (HDFS-11207) Unnecessary incompatible change of NNHAStatusHeartbeat.state in DatanodeProtocolProtos

2016-12-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15733496#comment-15733496 ] Kihwal Lee commented on HDFS-11207: --- Reverted HDFS-5079 and HDFS-11094. > Unnecessary i

[jira] [Reopened] (HDFS-11094) Send back HAState along with NamespaceInfo during a versionRequest as an optional parameter

2016-12-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reopened HDFS-11094: --- I've reverted this, as HDFS-5079 was reverted. Please rework the patch to reflect that. > Send back HASt

[jira] [Updated] (HDFS-5079) Cleaning up NNHAStatusHeartbeat.State DatanodeProtocolProtos.

2016-12-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-5079: - Release Note: This change affects wire-compatibility of the NameNode/DataNode heartbeat protocol. Only pre

[jira] [Commented] (HDFS-5079) Cleaning up NNHAStatusHeartbeat.State DatanodeProtocolProtos.

2016-12-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15733485#comment-15733485 ] Kihwal Lee commented on HDFS-5079: -- Reverted and updated the release note. > Cleaning up

[jira] [Updated] (HDFS-5079) Cleaning up NNHAStatusHeartbeat.State DatanodeProtocolProtos.

2016-12-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-5079: - Fix Version/s: 3.0.0-alpha1 > Cleaning up NNHAStatusHeartbeat.State DatanodeProtocolProtos. > -

[jira] [Updated] (HDFS-5079) Cleaning up NNHAStatusHeartbeat.State DatanodeProtocolProtos.

2016-12-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-5079: - Fix Version/s: (was: 3.0.0-alpha1) > Cleaning up NNHAStatusHeartbeat.State DatanodeProtocolProtos. > --

[jira] [Commented] (HDFS-5079) Cleaning up NNHAStatusHeartbeat.State DatanodeProtocolProtos.

2016-12-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15732634#comment-15732634 ] Kihwal Lee commented on HDFS-5079: -- This doesn't seem to much value while breaking compati

[jira] [Commented] (HDFS-11207) Unnecessary incompatible change of NNHAStatusHeartbeat.state in DatanodeProtocolProtos

2016-12-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15732628#comment-15732628 ] Kihwal Lee commented on HDFS-11207: --- I am going to revert HDFS-5079 later today if ther

[jira] [Comment Edited] (HDFS-11160) VolumeScanner reports write-in-progress replicas as corrupt incorrectly

2016-12-07 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15729073#comment-15729073 ] Kihwal Lee edited comment on HDFS-11160 at 12/7/16 3:53 PM: +1

[jira] [Commented] (HDFS-11160) VolumeScanner reports write-in-progress replicas as corrupt incorrectly

2016-12-07 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15729073#comment-15729073 ] Kihwal Lee commented on HDFS-11160: --- The patch looks good. About {noformat} // TODO: w

[jira] [Commented] (HDFS-11207) Unnecessary incompatible change of NNHAStatusHeartbeat.state in DatanodeProtocolProtos

2016-12-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15726418#comment-15726418 ] Kihwal Lee commented on HDFS-11207: --- I think we should revert HDFS-5079. It breaks compa

[jira] [Updated] (HDFS-11198) NN UI should link DN web address using hostnames

2016-12-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11198: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2 2.8.0

[jira] [Commented] (HDFS-11198) NN UI should link DN web address using hostnames

2016-12-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15725570#comment-15725570 ] Kihwal Lee commented on HDFS-11198: --- +1 > NN UI should link DN web address using hostna

[jira] [Commented] (HDFS-11198) NN UI should link DN web address using hostnames

2016-12-05 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15722401#comment-15722401 ] Kihwal Lee commented on HDFS-11198: --- The build master is down at this moment. > NN UI s

[jira] [Commented] (HDFS-11094) Send back HAState along with NamespaceInfo during a versionRequest as an optional parameter

2016-12-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15716687#comment-15716687 ] Kihwal Lee commented on HDFS-11094: --- bq. For the unit test, can we set a very large hear

[jira] [Commented] (HDFS-11146) Excess replicas will not be deleted until all storages's FBR received after failover

2016-12-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15715608#comment-15715608 ] Kihwal Lee commented on HDFS-11146: --- Sorry, I am busy and won't be able to review it pro

[jira] [Updated] (HDFS-11198) NN UI should link DN web address using hostnames

2016-12-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11198: -- Assignee: Weiwei Yang > NN UI should link DN web address using hostnames > -

[jira] [Commented] (HDFS-11198) NN UI should link DN web address using hostnames

2016-12-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15715214#comment-15715214 ] Kihwal Lee commented on HDFS-11198: --- sure go ahead. > NN UI should link DN web address

[jira] [Updated] (HDFS-8674) Improve performance of postponed block scans

2016-12-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-8674: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2 2.8.0

[jira] [Updated] (HDFS-11198) NN UI should link DN web address using hostnames

2016-12-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11198: -- Description: The new NN UI shows links to DN web pages, but since the link is from the info address ret

[jira] [Commented] (HDFS-8674) Improve performance of postponed block scans

2016-12-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15712633#comment-15712633 ] Kihwal Lee commented on HDFS-8674: -- +1 The patch looks good. > Improve performance of pos

[jira] [Created] (HDFS-11198) NN UI should link DN web address using hostnames

2016-12-01 Thread Kihwal Lee (JIRA)
Kihwal Lee created HDFS-11198: - Summary: NN UI should link DN web address using hostnames Key: HDFS-11198 URL: https://issues.apache.org/jira/browse/HDFS-11198 Project: Hadoop HDFS Issue Type: Bu

[jira] [Commented] (HDFS-8674) Improve performance of postponed block scans

2016-12-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15712103#comment-15712103 ] Kihwal Lee commented on HDFS-8674: -- {{LightWeightLinkedSet}} does not support {{remove()}}

[jira] [Commented] (HDFS-11131) TestThrottledAsyncChecker#testContextIsPassed is flaky

2016-12-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15712070#comment-15712070 ] Kihwal Lee commented on HDFS-11131: --- {{testCancellation}} also fails. Failure doesn't se

[jira] [Commented] (HDFS-11179) LightWeightHashSet can't remove blocks correctly which have a large number blockId

2016-12-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15712026#comment-15712026 ] Kihwal Lee commented on HDFS-11179: --- We have been running with HDFS-8674 internally for

[jira] [Commented] (HDFS-11192) OOM during Quota Initialization lead to Namenode hang

2016-11-30 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15710722#comment-15710722 ] Kihwal Lee commented on HDFS-11192: --- Was it hitting its ulimit on # of threads/processes

[jira] [Commented] (HDFS-8674) Improve performance of postponed block scans

2016-11-30 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15709744#comment-15709744 ] Kihwal Lee commented on HDFS-8674: -- Looks like {{HDFS-8674.2.patch}} is for branch-2. Can

[jira] [Updated] (HDFS-8674) Improve performance of postponed block scans

2016-11-30 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-8674: - Target Version/s: 2.8.0, 3.0.0-alpha2 (was: 2.8.0, 2.7.4, 3.0.0-alpha2) > Improve performance of postponed

[jira] [Commented] (HDFS-11180) Intermittent deadlock in NameNode when failover happens.

2016-11-30 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15709732#comment-15709732 ] Kihwal Lee commented on HDFS-11180: --- +1 for the patch. I am not sure whether it is wort

[jira] [Commented] (HDFS-11180) Intermittent deadlock in NameNode when failover happens.

2016-11-28 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15702354#comment-15702354 ] Kihwal Lee commented on HDFS-11180: --- bq. NameNode holds a lock of FSEditLog and requires

[jira] [Commented] (HDFS-10966) Enhance Dispatcher logic on deciding when to give up a source DataNode

2016-11-21 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15685014#comment-15685014 ] Kihwal Lee commented on HDFS-10966: --- Committed to branch-2.7. Thanks for working on this

[jira] [Updated] (HDFS-10966) Enhance Dispatcher logic on deciding when to give up a source DataNode

2016-11-21 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10966: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2 2.7.4

[jira] [Commented] (HDFS-10966) Enhance Dispatcher logic on deciding when to give up a source DataNode

2016-11-21 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15685011#comment-15685011 ] Kihwal Lee commented on HDFS-10966: --- The 2.7 patch looks good. > Enhance Dispatcher log

[jira] [Commented] (HDFS-10966) Enhance Dispatcher logic on deciding when to give up a source DataNode

2016-11-21 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15684079#comment-15684079 ] Kihwal Lee commented on HDFS-10966: --- I've committed this to trunk through branch-2.8. Pl

[jira] [Commented] (HDFS-10966) Enhance Dispatcher logic on deciding when to give up a source DataNode

2016-11-21 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15683978#comment-15683978 ] Kihwal Lee commented on HDFS-10966: --- +1 > Enhance Dispatcher logic on deciding when to

[jira] [Commented] (HDFS-8824) Do not use small blocks for balancing the cluster

2016-11-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677540#comment-15677540 ] Kihwal Lee commented on HDFS-8824: -- While this will initially increase the efficiency of b

[jira] [Commented] (HDFS-11146) Excess replicas will not be deleted until all storages's FBR received after failover

2016-11-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15673862#comment-15673862 ] Kihwal Lee commented on HDFS-11146: --- NN could tell datanodes to send FBR in a heartbeat

[jira] [Commented] (HDFS-11153) RPC Client detect address changed should reconnect immediately

2016-11-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15673846#comment-15673846 ] Kihwal Lee commented on HDFS-11153: --- You can enable this through config. Here is an exam

[jira] [Commented] (HDFS-10966) Enhance Dispatcher logic on deciding when to give up a source DataNode

2016-11-16 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671089#comment-15671089 ] Kihwal Lee commented on HDFS-10966: --- the patch looks reasonable in general. - It is repl

[jira] [Comment Edited] (HDFS-8870) Lease is leaked on write failure

2016-11-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668304#comment-15668304 ] Kihwal Lee edited comment on HDFS-8870 at 11/15/16 9:02 PM: Com

[jira] [Commented] (HDFS-8870) Lease is leaked on write failure

2016-11-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668304#comment-15668304 ] Kihwal Lee commented on HDFS-8870: -- Committed this to trunk, branch-2 and branch-2.8. >

[jira] [Updated] (HDFS-8870) Lease is leaked on write failure

2016-11-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-8870: - Fix Version/s: 3.0.0-alpha2 2.8.0 > Lease is leaked on write failure > -

[jira] [Updated] (HDFS-8870) Lease is leaked on write failure

2016-11-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-8870: - Assignee: Kuhu Shukla (was: Daryn Sharp) > Lease is leaked on write failure >

[jira] [Commented] (HDFS-8870) Lease is leaked on write failure

2016-11-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668242#comment-15668242 ] Kihwal Lee commented on HDFS-8870: -- +1 The patch looks good. > Lease is leaked on write f

[jira] [Commented] (HDFS-11056) Concurrent append and read operations lead to checksum error

2016-11-11 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15657402#comment-15657402 ] Kihwal Lee commented on HDFS-11056: --- +1 for the 2.7 patch. It looks to be a correct port

[jira] [Commented] (HDFS-11114) Support for running async disk checks in DataNode

2016-11-09 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15651285#comment-15651285 ] Kihwal Lee commented on HDFS-4: --- This failed recently. https://builds.apache.org/jo

[jira] [Commented] (HDFS-11115) Remove bytes2Array and string2Bytes

2016-11-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15648021#comment-15648021 ] Kihwal Lee commented on HDFS-5: --- You are undoing this optimization. {code} // Usin

[jira] [Commented] (HDFS-11114) Support for running async disk checks in DataNode

2016-11-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15647937#comment-15647937 ] Kihwal Lee commented on HDFS-4: --- It broke branch-2. [~steve_l] also noticed it. > S

[jira] [Commented] (HDFS-11056) Concurrent append and read operations lead to checksum error

2016-11-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15633249#comment-15633249 ] Kihwal Lee commented on HDFS-11056: --- I was looking at the patch since yesterday. It look

[jira] [Commented] (HDFS-11090) Leave safemode immediately if all blocks have reported in

2016-11-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630287#comment-15630287 ] Kihwal Lee commented on HDFS-11090: --- I agree with [~jingzhao]. If people don't want safe

[jira] [Commented] (HDFS-11078) NPE in LazyPersistFileScrubber

2016-11-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625742#comment-15625742 ] Kihwal Lee commented on HDFS-11078: --- Since this can happen legitimately and does not req

[jira] [Updated] (HDFS-10997) Reduce number of path resolving methods

2016-11-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10997: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (w

[jira] [Commented] (HDFS-10997) Reduce number of path resolving methods

2016-11-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625365#comment-15625365 ] Kihwal Lee commented on HDFS-10997: --- The branch-2 patch looks good. I will commit it sho

[jira] [Updated] (HDFS-11069) Tighten the authorization of datanode RPC

2016-10-27 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11069: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2 2.7.4

[jira] [Commented] (HDFS-11069) Tighten the authorization of datanode RPC

2016-10-27 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15612896#comment-15612896 ] Kihwal Lee commented on HDFS-11069: --- Committed to trunk, branch-2, branch-2.8 and branch

[jira] [Commented] (HDFS-11069) Tighten the authorization of datanode RPC

2016-10-27 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15612863#comment-15612863 ] Kihwal Lee commented on HDFS-11069: --- Thanks for the review, Daryn. I've verified it wor

[jira] [Reopened] (HDFS-10455) Logging the username when deny the setOwner operation

2016-10-27 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reopened HDFS-10455: --- Reverted the commits. > Logging the username when deny the setOwner operation > -

[jira] [Updated] (HDFS-10455) Logging the username when deny the setOwner operation

2016-10-27 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10455: -- Fix Version/s: (was: 3.0.0-alpha2) (was: 2.8.0) > Logging the username when d

[jira] [Commented] (HDFS-11069) Tighten the authorization of datanode RPC

2016-10-27 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15612831#comment-15612831 ] Kihwal Lee commented on HDFS-11069: --- {{TestPermission}} is broken by HDFS-10455. The ot

[jira] [Commented] (HDFS-10455) Logging the username when deny the setOwner operation

2016-10-27 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15612820#comment-15612820 ] Kihwal Lee commented on HDFS-10455: --- The precommit was from May. You should have re-run

[jira] [Commented] (HDFS-10455) Logging the username when deny the setOwner operation

2016-10-27 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15612810#comment-15612810 ] Kihwal Lee commented on HDFS-10455: --- It broke {{TestPermission}} {noformat} Running org

[jira] [Commented] (HDFS-11063) Set NameNode RPC server handler thread name with more descriptive information about the RPC call.

2016-10-27 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15612485#comment-15612485 ] Kihwal Lee commented on HDFS-11063: --- Once side-effect is that the namenode log can get b

[jira] [Updated] (HDFS-11069) Tighten the authorization of datanode RPC

2016-10-27 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11069: -- Attachment: HDFS-11069.patch > Tighten the authorization of datanode RPC > -

[jira] [Updated] (HDFS-11069) Tighten the authorization of datanode RPC

2016-10-27 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11069: -- Status: Patch Available (was: Open) > Tighten the authorization of datanode RPC > -

[jira] [Updated] (HDFS-11069) Tighten the authorization of datanode RPC

2016-10-27 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-11069: -- Description: The current implementation of {{checkSuperuserPrivilege()}} allows the datanode user from

[jira] [Commented] (HDFS-11069) Tighten the authorization of datanode RPC

2016-10-27 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15612304#comment-15612304 ] Kihwal Lee commented on HDFS-11069: --- {code} private void checkSuperuserPrivilege() { ...

[jira] [Assigned] (HDFS-11069) Tighten the authorization of datanode RPC

2016-10-27 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reassigned HDFS-11069: - Assignee: Kihwal Lee > Tighten the authorization of datanode RPC > --

<    3   4   5   6   7   8   9   10   11   12   >