[jira] [Commented] (HDFS-10662) Optimize UTF8 string/byte conversions

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407839#comment-15407839 ] Kihwal Lee commented on HDFS-10662: --- Committed it to trunk, but for branch-2, we have to wait for

[jira] [Commented] (HDFS-10662) Optimize UTF8 string/byte conversions

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407798#comment-15407798 ] Kihwal Lee commented on HDFS-10662: --- +1 on the latest patch. > Optimize UTF8 string/byte conversions >

[jira] [Commented] (HDFS-10662) Optimize UTF8 string/byte conversions

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407785#comment-15407785 ] Kihwal Lee commented on HDFS-10662: --- TestEditLog failure is being fixed in HDFS-10169/HDFS-10722. >

[jira] [Updated] (HDFS-10569) A bug causes OutOfIndex error in BlockListAsLongs

2016-08-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10569: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Updated] (HDFS-10569) A bug causes OutOfIndex error in BlockListAsLongs

2016-08-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10569: -- Priority: Minor (was: Major) > A bug causes OutOfIndex error in BlockListAsLongs >

[jira] [Commented] (HDFS-10569) A bug causes OutOfIndex error in BlockListAsLongs

2016-08-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406554#comment-15406554 ] Kihwal Lee commented on HDFS-10569: --- The failed test is passing for me with the patch. HDFS-10572

[jira] [Commented] (HDFS-10674) Optimize creating a full path from an inode

2016-08-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406340#comment-15406340 ] Kihwal Lee commented on HDFS-10674: --- +1 the patch looks good. > Optimize creating a full path from an

[jira] [Updated] (HDFS-10674) Optimize creating a full path from an inode

2016-08-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10674: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10715) NPE when applying AvailableSpaceBlockPlacementPolicy

2016-08-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406272#comment-15406272 ] Kihwal Lee commented on HDFS-10715: --- [~zhuguangbin86], I've added you as a contributor and assigned this

[jira] [Updated] (HDFS-10715) NPE when applying AvailableSpaceBlockPlacementPolicy

2016-08-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10715: -- Assignee: Guangbin Zhu > NPE when applying AvailableSpaceBlockPlacementPolicy >

[jira] [Updated] (HDFS-742) A down DataNode makes Balancer to hang on repeatingly asking NameNode its partial block list

2016-08-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-742: Resolution: Fixed Fix Version/s: 3.0.0-alpha2 2.9.0 2.8.0

[jira] [Commented] (HDFS-742) A down DataNode makes Balancer to hang on repeatingly asking NameNode its partial block list

2016-08-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406233#comment-15406233 ] Kihwal Lee commented on HDFS-742: - The test failure is due to HDFS-9781. > A down DataNode makes Balancer

[jira] [Commented] (HDFS-10619) Cache path in InodesInPath

2016-08-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406222#comment-15406222 ] Kihwal Lee commented on HDFS-10619: --- HDFS-10653 got committed. I will run this through precommit one

[jira] [Commented] (HDFS-10656) Optimize conversion of byte arrays back to path string

2016-08-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406204#comment-15406204 ] Kihwal Lee commented on HDFS-10656: --- It looks like existing test cases (e.g. TestPathComponents) are

[jira] [Updated] (HDFS-10656) Optimize conversion of byte arrays back to path string

2016-08-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10656: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Resolved] (HDFS-6687) nn.getNamesystem() may return NPE from JspHelper

2016-08-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee resolved HDFS-6687. -- Resolution: Done > nn.getNamesystem() may return NPE from JspHelper >

[jira] [Commented] (HDFS-6687) nn.getNamesystem() may return NPE from JspHelper

2016-08-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404904#comment-15404904 ] Kihwal Lee commented on HDFS-6687: -- The new namenode UI won't hit JSPs like it used to. I don't think this

[jira] [Updated] (HDFS-742) A down DataNode makes Balancer to hang on repeatingly asking NameNode its partial block list

2016-08-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-742: Attachment: HDFS-742.v1.trunk.patch This somehow didn't get committed. Rebasing to trunk. > A down DataNode

[jira] [Commented] (HDFS-10662) Optimize UTF8 string/byte conversions

2016-08-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404773#comment-15404773 ] Kihwal Lee commented on HDFS-10662: --- Unfortunately, this just went in and the patch no longer applies.

[jira] [Updated] (HDFS-5805) TestCheckpoint.testCheckpoint fails intermittently on branch2

2016-08-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-5805: - Resolution: Fixed Fix Version/s: 3.0.0-alpha2 2.9.0 2.8.0

[jira] [Commented] (HDFS-10662) Optimize UTF8 string/byte conversions

2016-08-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404761#comment-15404761 ] Kihwal Lee commented on HDFS-10662: --- +1 lgtm. > Optimize UTF8 string/byte conversions >

[jira] [Commented] (HDFS-5805) TestCheckpoint.testCheckpoint fails intermittently on branch2

2016-08-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404756#comment-15404756 ] Kihwal Lee commented on HDFS-5805: -- Committed to trunk, branch-2 and branch-2.8. Thanks for fixing this,

[jira] [Commented] (HDFS-5805) TestCheckpoint.testCheckpoint fails intermittently on branch2

2016-08-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404720#comment-15404720 ] Kihwal Lee commented on HDFS-5805: -- +1 lgtm > TestCheckpoint.testCheckpoint fails intermittently on

[jira] [Updated] (HDFS-5805) TestCheckpoint.testCheckpoint fails intermittently on branch2

2016-08-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-5805: - Target Version/s: 2.8.0 > TestCheckpoint.testCheckpoint fails intermittently on branch2 >

[jira] [Commented] (HDFS-10601) Improve log message to include hostname when the NameNode is in safemode

2016-08-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404500#comment-15404500 ] Kihwal Lee commented on HDFS-10601: --- If the namenode http ip address is set to 0.0.0.0 on the namenode,

[jira] [Commented] (HDFS-10301) BlockReport retransmissions may lead to storages falsely being declared zombie if storage report processing happens out of order

2016-08-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404168#comment-15404168 ] Kihwal Lee commented on HDFS-10301: --- [~shv], thanks for the revert, but I think you missed branch-2.7.

[jira] [Commented] (HDFS-10714) Issue in handling checksum errors in write pipeline when fault DN is LAST_IN_PIPELINE

2016-08-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404103#comment-15404103 ] Kihwal Lee commented on HDFS-10714: --- Guessing who is faulty is complicated. Whatever works for a case

[jira] [Updated] (HDFS-10715) NPE when applying AvailableSpaceBlockPlacementPolicy

2016-08-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10715: -- Status: Patch Available (was: Open) > NPE when applying AvailableSpaceBlockPlacementPolicy >

[jira] [Commented] (HDFS-8224) Any IOException in DataTransfer#run() will run diskError thread even if it is not disk error

2016-08-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15403981#comment-15403981 ] Kihwal Lee commented on HDFS-8224: -- bq. Incidentally, we just saw the same stack trace

[jira] [Updated] (HDFS-10617) PendingReconstructionBlocks.size() should be synchronized

2016-07-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10617: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.9.0 Status: Resolved

[jira] [Commented] (HDFS-10617) PendingReconstructionBlocks.size() should be synchronized

2016-07-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15376227#comment-15376227 ] Kihwal Lee commented on HDFS-10617: --- +1 lgtm > PendingReconstructionBlocks.size() should be

[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode

2016-06-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348716#comment-15348716 ] Kihwal Lee commented on HDFS-7959: -- {{TestHttpServerLifecycle}} works fine. Ran multiple times. {noformat}

[jira] [Issue Comment Deleted] (HDFS-7959) WebHdfs logging is missing on Datanode

2016-06-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-7959: - Comment: was deleted (was: {{TestHttpServerLifecycle}} works fine. Ran multiple times. {noformat}

[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode

2016-06-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348570#comment-15348570 ] Kihwal Lee commented on HDFS-7959: -- {{TestHttpServerLifecycle}} works fine. Ran multiple times. {noformat}

[jira] [Updated] (HDFS-7959) WebHdfs logging is missing on Datanode

2016-06-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-7959: - Attachment: HDFS-7959.3.trunk.patch HDFS-7959.3.branch-2.patch > WebHdfs logging is

[jira] [Updated] (HDFS-10555) Unable to loadFSEdits due to a failure in readCachePoolInfo

2016-06-23 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10555: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.9.0 Status: Resolved

[jira] [Commented] (HDFS-10555) Unable to loadFSEdits due to a failure in readCachePoolInfo

2016-06-23 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346495#comment-15346495 ] Kihwal Lee commented on HDFS-10555: --- picked to branch-2. > Unable to loadFSEdits due to a failure in

[jira] [Updated] (HDFS-10328) Add per-cache-pool default replication num configuration

2016-06-23 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10328: -- Fix Version/s: (was: 2.3.0) 2.9.0 > Add per-cache-pool default replication num

[jira] [Commented] (HDFS-10563) Block reports could be silently dropped by NN

2016-06-23 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346489#comment-15346489 ] Kihwal Lee commented on HDFS-10563: --- bq. does not handle the case when the queue is full and offer

[jira] [Commented] (HDFS-10569) A bug causes OutOfIndex error in BlockListAsLongs

2016-06-23 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346443#comment-15346443 ] Kihwal Lee commented on HDFS-10569: --- It looks like {{index2BlockId()}} was used to get the right index

[jira] [Commented] (HDFS-10440) Improve DataNode web UI

2016-06-23 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346429#comment-15346429 ] Kihwal Lee commented on HDFS-10440: --- It looks good to me. +1 pending Vinay's review since he asked for

[jira] [Updated] (HDFS-7959) WebHdfs logging is missing on Datanode

2016-06-21 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-7959: - Attachment: HDFS-7959.2.trunk.patch HDFS-7959.2.branch-2.patch The new patches address the

[jira] [Commented] (HDFS-6440) Support more than 2 NameNodes

2016-06-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336938#comment-15336938 ] Kihwal Lee commented on HDFS-6440: -- HDFS-10536 got filed. I just looked at the edit log tailor change

[jira] [Comment Edited] (HDFS-10440) Improve DataNode web UI

2016-06-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336351#comment-15336351 ] Kihwal Lee edited comment on HDFS-10440 at 6/17/16 4:33 PM: {quote} Address

[jira] [Updated] (HDFS-10493) Add links to datanode web UI in namenode datanodes page

2016-06-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10493: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved

[jira] [Commented] (HDFS-10493) Add links to datanode web UI in namenode datanodes page

2016-06-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336357#comment-15336357 ] Kihwal Lee commented on HDFS-10493: --- +1 The patch Looks good > Add links to datanode web UI in namenode

[jira] [Commented] (HDFS-10440) Improve DataNode web UI

2016-06-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336351#comment-15336351 ] Kihwal Lee commented on HDFS-10440: --- bq. Address information is redundant, but better than breaking

[jira] [Commented] (HDFS-7597) DNs should not open new NN connections when webhdfs clients seek

2016-06-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331938#comment-15331938 ] Kihwal Lee commented on HDFS-7597: -- bq. Should we trigger a new jenkins run and commit this patch? Yes,

[jira] [Commented] (HDFS-10519) Add a configuration option to enable in-progress edit log tailing

2016-06-14 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15329706#comment-15329706 ] Kihwal Lee commented on HDFS-10519: --- How does it actually work? Can it replay the same edit log segment

[jira] [Commented] (HDFS-8046) Allow better control of getContentSummary

2016-06-14 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15329542#comment-15329542 ] Kihwal Lee commented on HDFS-8046: -- [~xiaochen], thanks for letting me know. I've cherry-picked the fix to

[jira] [Updated] (HDFS-8581) ContentSummary on / skips further counts on yielding lock

2016-06-14 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-8581: - Fix Version/s: (was: 2.8.0) 2.6.5 2.7.3 > ContentSummary on /

[jira] [Commented] (HDFS-8581) ContentSummary on / skips further counts on yielding lock

2016-06-14 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15329502#comment-15329502 ] Kihwal Lee commented on HDFS-8581: -- Cherry-picked to branch-2.7 and branch-2.6. Thanks again for reporting

[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode

2016-06-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328371#comment-15328371 ] Kihwal Lee commented on HDFS-7959: -- The test failure is not related to the patch. The whitespace warning

[jira] [Updated] (HDFS-7959) WebHdfs logging is missing on Datanode

2016-06-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-7959: - Attachment: HDFS-7959.1.trunk.patch HDFS-7959.1.branch-2.patch The new patches calls the

[jira] [Commented] (HDFS-10440) Improve DataNode web UI

2016-06-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327899#comment-15327899 ] Kihwal Lee commented on HDFS-10440: --- [~vinayrpet], I agree showing the last heartbeat and block report

[jira] [Commented] (HDFS-10490) Client may never recovery replica after a timeout during sending packet

2016-06-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327836#comment-15327836 ] Kihwal Lee commented on HDFS-10490: --- {{BlockReceiver}} flushes after writing each packet locally, so the

[jira] [Updated] (HDFS-7959) WebHdfs logging is missing on Datanode

2016-06-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-7959: - Target Version/s: 2.8.0 > WebHdfs logging is missing on Datanode > --

[jira] [Updated] (HDFS-7959) WebHdfs logging is missing on Datanode

2016-06-13 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-7959: - Attachment: HDFS-7959.trunk.patch HDFS-7959.branch-2.patch Sorry for neglecting it for so

[jira] [Commented] (HDFS-10220) A large number of expired leases can make namenode unresponsive and cause failover

2016-06-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321062#comment-15321062 ] Kihwal Lee commented on HDFS-10220: --- The patch looks fine to me. [~raviprak], how about you? > A large

[jira] [Commented] (HDFS-10493) Add links to datanode web UI in namenode datanodes page

2016-06-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321051#comment-15321051 ] Kihwal Lee commented on HDFS-10493: --- It looks a bit funny because "name" contains the xferAddr, but the

[jira] [Commented] (HDFS-10440) Improve DataNode web UI

2016-06-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316646#comment-15316646 ] Kihwal Lee commented on HDFS-10440: --- The patch looks good, waiting for the precommit to come back. I

[jira] [Updated] (HDFS-10440) Improve DataNode web UI

2016-06-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10440: -- Target Version/s: 2.8.0 Hadoop Flags: Incompatible change > Improve DataNode web UI >

[jira] [Commented] (HDFS-10490) Client may never recovery replica after a timeout during sending packet

2016-06-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316538#comment-15316538 ] Kihwal Lee commented on HDFS-10490: --- There was a similar bug, HDFS-10178, which has been fixed in

[jira] [Commented] (HDFS-10477) Stop decommission a rack of DataNodes caused NameNode fail over to standby

2016-06-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314253#comment-15314253 ] Kihwal Lee commented on HDFS-10477: --- If the lock is simply released and re-acquired right away per

[jira] [Updated] (HDFS-10480) Add an admin command to list currently open files

2016-06-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10480: -- Description: Currently there is no easy way to obtain the list of active leases or files being written.

[jira] [Comment Edited] (HDFS-10480) Add an admin command to list currently open files

2016-06-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312546#comment-15312546 ] Kihwal Lee edited comment on HDFS-10480 at 6/2/16 4:14 PM: --- While debugging

[jira] [Comment Edited] (HDFS-10480) Add an admin command to list currently open files

2016-06-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312546#comment-15312546 ] Kihwal Lee edited comment on HDFS-10480 at 6/2/16 4:11 PM: --- While debugging

[jira] [Commented] (HDFS-10480) Add an admin command to list currently open files

2016-06-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312546#comment-15312546 ] Kihwal Lee commented on HDFS-10480: --- While debugging issues, I had to dump a huge fsimage to get the

[jira] [Created] (HDFS-10480) Add an admin command to list currently open files

2016-06-02 Thread Kihwal Lee (JIRA)
Kihwal Lee created HDFS-10480: - Summary: Add an admin command to list currently open files Key: HDFS-10480 URL: https://issues.apache.org/jira/browse/HDFS-10480 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-10477) Stop decommission a rack of DataNodes caused NameNode fail over to standby

2016-06-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312438#comment-15312438 ] Kihwal Lee commented on HDFS-10477: --- It will be better if the locking is done per storage instead of per

[jira] [Comment Edited] (HDFS-10440) Improve DataNode web UI

2016-05-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15307905#comment-15307905 ] Kihwal Lee edited comment on HDFS-10440 at 5/31/16 3:27 PM: Thanks for the

[jira] [Updated] (HDFS-10440) Improve DataNode web UI

2016-05-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10440: -- Assignee: Weiwei Yang (was: WEIWEI YANG) > Improve DataNode web UI > --- > >

[jira] [Updated] (HDFS-10440) Improve DataNode web UI

2016-05-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10440: -- Assignee: WEIWEI YANG > Improve DataNode web UI > --- > > Key:

[jira] [Commented] (HDFS-10440) Improve DataNode web UI

2016-05-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15307905#comment-15307905 ] Kihwal Lee commented on HDFS-10440: --- Thanks for the patch. I think it is showing the available

[jira] [Commented] (HDFS-10438) When NameNode HA is configured to use the lifeline RPC server, it should log the address of that server.

2016-05-20 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15293639#comment-15293639 ] Kihwal Lee commented on HDFS-10438: --- Post-precommit +1 as the final nail on the coffin. > When NameNode

[jira] [Commented] (HDFS-10440) Add more information to DataNode web UI

2016-05-20 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15293610#comment-15293610 ] Kihwal Lee commented on HDFS-10440: --- bq. Running state (active, decommissioning, decommissioned or lost

[jira] [Resolved] (HDFS-10417) Improve error message from checkBlockLocalPathAccess

2016-05-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee resolved HDFS-10417. --- Resolution: Fixed > Improve error message from checkBlockLocalPathAccess >

[jira] [Updated] (HDFS-10417) Improve error message from checkBlockLocalPathAccess

2016-05-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10417: -- Status: In Progress (was: Patch Available) > Improve error message from checkBlockLocalPathAccess >

[jira] [Commented] (HDFS-10417) Improve error message from checkBlockLocalPathAccess

2016-05-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15292121#comment-15292121 ] Kihwal Lee commented on HDFS-10417: --- Committed this to trunk, branch-2 and branch-2.8. Thanks for the

[jira] [Commented] (HDFS-10417) Improve error message from checkBlockLocalPathAccess

2016-05-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15292118#comment-15292118 ] Kihwal Lee commented on HDFS-10417: --- +1 for the latest patch. > Improve error message from

[jira] [Commented] (HDFS-10246) Standby NameNode dfshealth.jsp Response very slow

2016-05-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15291131#comment-15291131 ] Kihwal Lee commented on HDFS-10246: --- HDFS-7097 and HDFS-6763 also mitigate spikes on standby name nodes.

[jira] [Updated] (HDFS-10417) Improve error message from checkBlockLocalPathAccess

2016-05-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10417: -- Status: Patch Available (was: Reopened) > Improve error message from checkBlockLocalPathAccess >

[jira] [Comment Edited] (HDFS-10417) Improve error message from checkBlockLocalPathAccess

2016-05-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287488#comment-15287488 ] Kihwal Lee edited comment on HDFS-10417 at 5/17/16 8:24 PM: Sorry, I reverted

[jira] [Commented] (HDFS-10417) Improve error message from checkBlockLocalPathAccess

2016-05-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287488#comment-15287488 ] Kihwal Lee commented on HDFS-10417: --- Sorry, I reverted it. It breaks a test case which compares the

[jira] [Issue Comment Deleted] (HDFS-10417) Improve error message from checkBlockLocalPathAccess

2016-05-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10417: -- Comment: was deleted (was: Committed to trunk, branch-2 and branch-2.8. Thanks for improving the error

[jira] [Reopened] (HDFS-10417) Improve error message from checkBlockLocalPathAccess

2016-05-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reopened HDFS-10417: --- > Improve error message from checkBlockLocalPathAccess >

[jira] [Updated] (HDFS-10417) Improve error message from checkBlockLocalPathAccess

2016-05-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10417: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved

[jira] [Commented] (HDFS-10417) Improve error message from checkBlockLocalPathAccess

2016-05-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15287461#comment-15287461 ] Kihwal Lee commented on HDFS-10417: --- +1. BTW, this is for the "legacy" short-circuit reads. The newer

[jira] [Updated] (HDFS-10417) Improve error message from checkBlockLocalPathAccess

2016-05-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10417: -- Summary: Improve error message from checkBlockLocalPathAccess (was: Actionable msgs for

[jira] [Updated] (HDFS-10417) Actionable msgs for checkBlockLocalPathAccess

2016-05-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10417: -- Assignee: Tianyin Xu > Actionable msgs for checkBlockLocalPathAccess >

[jira] [Updated] (HDFS-10303) DataStreamer#ResponseProcessor calculates packet ack latency incorrectly.

2016-05-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10303: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved

[jira] [Updated] (HDFS-10303) DataStreamer#ResponseProcessor calculates packet ack latency incorrectly.

2016-05-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10303: -- Summary: DataStreamer#ResponseProcessor calculates packet ack latency incorrectly. (was:

[jira] [Commented] (HDFS-10303) DataStreamer#ResponseProcessor calculate packet acknowledge duration wrongly.

2016-05-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15286636#comment-15286636 ] Kihwal Lee commented on HDFS-10303: --- +1 The latest patch looks good. The failed test cases are unrelated

[jira] [Updated] (HDFS-10400) hdfs dfs -put exits with zero on error

2016-05-16 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10400: -- Assignee: Yiqun Lin > hdfs dfs -put exits with zero on error > -- >

[jira] [Commented] (HDFS-10400) hdfs dfs -put exits with zero on error

2016-05-16 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284641#comment-15284641 ] Kihwal Lee commented on HDFS-10400: --- [~linyiqun], thanks for working in this. I assigned it to you. >

[jira] [Commented] (HDFS-8772) Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails

2016-05-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15281687#comment-15281687 ] Kihwal Lee commented on HDFS-8772: -- Cherry-picked to branch-2.7 as this test case occasionally fails the

[jira] [Updated] (HDFS-8772) Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails

2016-05-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-8772: - Fix Version/s: (was: 2.8.0) 2.7.3 > Fix TestStandbyIsHot#testDatanodeRestarts which

[jira] [Updated] (HDFS-10220) A large number of expired leases can make namenode unresponsive and cause failover

2016-05-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10220: -- Summary: A large number of expired leases can make namenode unresponsive and cause failover (was:

[jira] [Comment Edited] (HDFS-10220) Namenode failover due to too long loking in LeaseManager.Monitor

2016-05-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15281592#comment-15281592 ] Kihwal Lee edited comment on HDFS-10220 at 5/12/16 3:16 PM: The throughput is

[jira] [Commented] (HDFS-10220) Namenode failover due to too long loking in LeaseManager.Monitor

2016-05-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15281592#comment-15281592 ] Kihwal Lee commented on HDFS-10220: --- The throughput is pathetic, but it seems in the ballpark of what I

<    7   8   9   10   11   12   13   14   15   16   >