[jira] [Commented] (HDFS-10340) data node sudden killed

2016-08-23 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15432830#comment-15432830 ] Kihwal Lee commented on HDFS-10340: --- I checked the oom killer code and it is SIGKILL as you pointed out.

[jira] [Commented] (HDFS-10762) Pass IIP for file status related methods

2016-08-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15431684#comment-15431684 ] Kihwal Lee commented on HDFS-10762: --- Although it is functionally correct, it looks like the patch undid

[jira] [Commented] (HDFS-10762) Pass IIP for file status related methods

2016-08-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15431588#comment-15431588 ] Kihwal Lee commented on HDFS-10762: --- [~daryn], please post a branch-2 version of the patch. > Pass IIP

[jira] [Updated] (HDFS-10762) Pass IIP for file status related methods

2016-08-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10762: -- Fix Version/s: 3.0.0-alpha2 > Pass IIP for file status related methods >

[jira] [Commented] (HDFS-10762) Pass IIP for file status related methods

2016-08-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15431553#comment-15431553 ] Kihwal Lee commented on HDFS-10762: --- Nice change. I am convinced that there will be no case where

[jira] [Updated] (HDFS-10738) Fix TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration test failure

2016-08-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10738: -- Fix Version/s: (was: 2.9.0) 2.8.0 > Fix

[jira] [Commented] (HDFS-10762) Pass IIP for file status related methods

2016-08-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15430733#comment-15430733 ] Kihwal Lee commented on HDFS-10762: --- I will try finishing the review today. > Pass IIP for file status

[jira] [Updated] (HDFS-7959) WebHdfs logging is missing on Datanode

2016-08-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-7959: - Fix Version/s: (was: 2.9.0) 2.8.0 > WebHdfs logging is missing on Datanode >

[jira] [Commented] (HDFS-7959) WebHdfs logging is missing on Datanode

2016-08-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15428770#comment-15428770 ] Kihwal Lee commented on HDFS-7959: -- Cherry-picked to 2.8. > WebHdfs logging is missing on Datanode >

[jira] [Updated] (HDFS-10711) Optimize FSPermissionChecker group membership check

2016-08-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10711: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10711) Optimize FSPermissionChecker group membership check

2016-08-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15428240#comment-15428240 ] Kihwal Lee commented on HDFS-10711: --- and also the test passes fine. {noformat}

[jira] [Commented] (HDFS-10711) Optimize FSPermissionChecker group membership check

2016-08-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15428219#comment-15428219 ] Kihwal Lee commented on HDFS-10711: --- +1 The change looks good. > Optimize FSPermissionChecker group

[jira] [Updated] (HDFS-10768) Optimize mkdir ops

2016-08-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10768: -- Status: Patch Available (was: Open) > Optimize mkdir ops > -- > > Key:

[jira] [Commented] (HDFS-9395) Make HDFS audit logging consistant

2016-08-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15427286#comment-15427286 ] Kihwal Lee commented on HDFS-9395: -- [~vinodkv], I also reverted it from branch-2.7. > Make HDFS audit

[jira] [Updated] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10763: -- Resolution: Fixed Status: Resolved (was: Patch Available) Committed the patch to branch-2.7.

[jira] [Commented] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15427188#comment-15427188 ] Kihwal Lee commented on HDFS-10763: --- The test passes reliably when run on my box. {noformat}

[jira] [Commented] (HDFS-10776) Revisit Hdfs Audit Logging

2016-08-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15426946#comment-15426946 ] Kihwal Lee commented on HDFS-10776: --- Regarding bq.1) What is the correct way of doing the hdfs audit

[jira] [Comment Edited] (HDFS-10776) Revisit Hdfs Audit Logging

2016-08-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15426946#comment-15426946 ] Kihwal Lee edited comment on HDFS-10776 at 8/18/16 6:22 PM: Regarding bq.1)

[jira] [Updated] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10763: -- Attachment: HDFS-10763.branch-2.7.v2.patch Reverted the original commit from branch-2.7. Attaching a

[jira] [Updated] (HDFS-10776) Revisit Hdfs Audit Logging

2016-08-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10776: -- Description: HDFS audit logging is considered critical in many scenarios, but it hasn't been always

[jira] [Commented] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15426896#comment-15426896 ] Kihwal Lee commented on HDFS-10763: --- Ouch. I meant to reuse {{path}}, but apparently I didn't. > Open

[jira] [Commented] (HDFS-10711) Optimize FSPermissionChecker group membership check

2016-08-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15426889#comment-15426889 ] Kihwal Lee commented on HDFS-10711: --- Kicked the build > Optimize FSPermissionChecker group membership

[jira] [Updated] (HDFS-10745) Directly resolve paths into INodesInPath

2016-08-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10745: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10745) Directly resolve paths into INodesInPath

2016-08-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15426693#comment-15426693 ] Kihwal Lee commented on HDFS-10745: --- The branch-2 patch looks good. > Directly resolve paths into

[jira] [Commented] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15426578#comment-15426578 ] Kihwal Lee commented on HDFS-10763: --- Other tests run fine, except {{TestDataNodeVolumeFailure}}. But it

[jira] [Commented] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15426472#comment-15426472 ] Kihwal Lee commented on HDFS-10763: --- Going through the test failures. {{TestRenameWithSnapshots}}

[jira] [Commented] (HDFS-10745) Directly resolve paths into INodesInPath

2016-08-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425382#comment-15425382 ] Kihwal Lee commented on HDFS-10745: --- Committed to trunk, but need a patch for branch-2. > Directly

[jira] [Commented] (HDFS-10745) Directly resolve paths into INodesInPath

2016-08-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425359#comment-15425359 ] Kihwal Lee commented on HDFS-10745: --- +1 lgtm > Directly resolve paths into INodesInPath >

[jira] [Commented] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425291#comment-15425291 ] Kihwal Lee commented on HDFS-10763: --- Removed an aborted jenkins run. > Open files can leak permanently

[jira] [Issue Comment Deleted] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10763: -- Comment: was deleted (was: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime

[jira] [Updated] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10763: -- Status: Patch Available (was: Reopened) > Open files can leak permanently due to inconsistent lease

[jira] [Commented] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425099#comment-15425099 ] Kihwal Lee commented on HDFS-10763: --- As pointed out by [~zhz], {{TestOpenFilesWithSnapshot}} fails in

[jira] [Updated] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10763: -- Attachment: HDFS-10763.branch-2.7.supplement.patch Attaching a supplemental patch for branch-2.7. This

[jira] [Reopened] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-17 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reopened HDFS-10763: --- Reopening to fix branch-2.7. Apparently we can't simply eliminate open file leak. A deleted file in a

[jira] [Commented] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-16 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423574#comment-15423574 ] Kihwal Lee commented on HDFS-10763: --- It seems to have introduced a bug to branch-2.7 when there are

[jira] [Comment Edited] (HDFS-9696) Garbage snapshot records lingering forever

2016-08-16 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423568#comment-15423568 ] Kihwal Lee edited comment on HDFS-9696 at 8/16/16 11:13 PM: I think it is

[jira] [Commented] (HDFS-9696) Garbage snapshot records lingering forever

2016-08-16 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423568#comment-15423568 ] Kihwal Lee commented on HDFS-9696: -- I think it is broken by HDFS-10763 and a real bug. I will fix it soon.

[jira] [Commented] (HDFS-9696) Garbage snapshot records lingering forever

2016-08-16 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423538#comment-15423538 ] Kihwal Lee commented on HDFS-9696: -- [~zhz] I will take a look. > Garbage snapshot records lingering

[jira] [Comment Edited] (HDFS-10745) Directly resolve paths into INodesInPath

2016-08-16 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423531#comment-15423531 ] Kihwal Lee edited comment on HDFS-10745 at 8/16/16 10:44 PM: - It looks like

[jira] [Commented] (HDFS-10745) Directly resolve paths into INodesInPath

2016-08-16 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423531#comment-15423531 ] Kihwal Lee commented on HDFS-10745: --- It loks like {{TestBootstrapStandby}} has been failing in trunk

[jira] [Commented] (HDFS-10729) NameNode crashes when loading edits because max directory items is exceeded

2016-08-16 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15422827#comment-15422827 ] Kihwal Lee commented on HDFS-10729: --- I just hit the submit patch button. > NameNode crashes when

[jira] [Updated] (HDFS-10729) NameNode crashes when loading edits because max directory items is exceeded

2016-08-16 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10729: -- Status: Patch Available (was: Open) > NameNode crashes when loading edits because max directory items

[jira] [Updated] (HDFS-10342) BlockManager#createLocatedBlocks should not check corrupt replicas if none are corrupt

2016-08-16 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10342: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved

[jira] [Updated] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10763: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Updated] (HDFS-10744) Internally optimize path component resolution

2016-08-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10744: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10744) Internally optimize path component resolution

2016-08-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421709#comment-15421709 ] Kihwal Lee commented on HDFS-10744: --- +1 LGTM. > Internally optimize path component resolution >

[jira] [Updated] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10763: -- Status: Patch Available (was: Open) > Open files can leak permanently due to inconsistent lease update

[jira] [Updated] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10763: -- Attachment: HDFS-10763.patch HDFS-10763.br27.patch > Open files can leak permanently

[jira] [Updated] (HDFS-9696) Garbage snapshot records lingering forever

2016-08-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-9696: - Resolution: Fixed Fix Version/s: 3.0.0-alpha2 2.7.4 Status: Resolved

[jira] [Commented] (HDFS-9696) Garbage snapshot records lingering forever

2016-08-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421376#comment-15421376 ] Kihwal Lee commented on HDFS-9696: -- Thanks for the review, [~szetszwo]. I will commit it shortly. >

[jira] [Commented] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421300#comment-15421300 ] Kihwal Lee commented on HDFS-10763: --- Regarding 2), trunk through branch-2 (2.8) can be fixed by simply

[jira] [Assigned] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reassigned HDFS-10763: - Assignee: Kihwal Lee > Open files can leak permanently due to inconsistent lease update >

[jira] [Created] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-15 Thread Kihwal Lee (JIRA)
Kihwal Lee created HDFS-10763: - Summary: Open files can leak permanently due to inconsistent lease update Key: HDFS-10763 URL: https://issues.apache.org/jira/browse/HDFS-10763 Project: Hadoop HDFS

[jira] [Commented] (HDFS-5770) Exception hit in TestPersistBlocks

2016-08-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421045#comment-15421045 ] Kihwal Lee commented on HDFS-5770: -- Saw this in a precommit build. {noformat} 2016-08-12 23:40:13,900

[jira] [Commented] (HDFS-9696) Garbage snapshot records lingering forever

2016-08-15 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420974#comment-15420974 ] Kihwal Lee commented on HDFS-9696: -- {noformat} --- T

[jira] [Updated] (HDFS-9696) Garbage snapshot records lingering forever

2016-08-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-9696: - Attachment: HDFS-9696.v2.patch > Garbage snapshot records lingering forever >

[jira] [Commented] (HDFS-9696) Garbage snapshot records lingering forever

2016-08-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419584#comment-15419584 ] Kihwal Lee commented on HDFS-9696: -- It looks like these tests failed because the snapshot section wasn't

[jira] [Updated] (HDFS-9696) Garbage snapshot records lingering forever

2016-08-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-9696: - Attachment: HDFS-9696.patch Attaching a patch containing a unit test. > Garbage snapshot records

[jira] [Updated] (HDFS-9696) Garbage snapshot records lingering forever

2016-08-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-9696: - Status: Patch Available (was: Reopened) > Garbage snapshot records lingering forever >

[jira] [Updated] (HDFS-9696) Garbage snapshot records lingering forever

2016-08-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-9696: - Target Version/s: 2.6.5, 2.7.4 > Garbage snapshot records lingering forever >

[jira] [Commented] (HDFS-9696) Garbage snapshot records lingering forever

2016-08-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419099#comment-15419099 ] Kihwal Lee commented on HDFS-9696: -- Does something like this make sense? Saving a diff section involves

[jira] [Assigned] (HDFS-9696) Garbage snapshot records lingering forever

2016-08-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reassigned HDFS-9696: Assignee: Kihwal Lee > Garbage snapshot records lingering forever >

[jira] [Commented] (HDFS-9696) Garbage snapshot records lingering forever

2016-08-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419080#comment-15419080 ] Kihwal Lee commented on HDFS-9696: -- One basic sanity check can be done for cases where there is no

[jira] [Commented] (HDFS-9696) Garbage snapshot records lingering forever

2016-08-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418942#comment-15418942 ] Kihwal Lee commented on HDFS-9696: -- It turns out that HDFS-9406 is not related to this issue. The garbage

[jira] [Reopened] (HDFS-9696) Garbage snapshot records lingering forever

2016-08-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reopened HDFS-9696: -- Assignee: (was: Yongjun Zhang) > Garbage snapshot records lingering forever >

[jira] [Updated] (HDFS-10738) Fix TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration test failure

2016-08-09 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10738: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10738) Fix TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration test failure

2016-08-09 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414127#comment-15414127 ] Kihwal Lee commented on HDFS-10738: --- +1 the patch looks good. {{TestTracing}} was just fixed by

[jira] [Updated] (HDFS-10342) BlockManager#createLocatedBlocks should not check corrupt replicas if none are corrupt

2016-08-09 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10342: -- Target Version/s: 2.8.0 > BlockManager#createLocatedBlocks should not check corrupt replicas if none >

[jira] [Updated] (HDFS-10342) BlockManager#createLocatedBlocks should not check corrupt replicas if none are corrupt

2016-08-09 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10342: -- Fix Version/s: 3.0.0-alpha2 > BlockManager#createLocatedBlocks should not check corrupt replicas if

[jira] [Commented] (HDFS-10342) BlockManager#createLocatedBlocks should not check corrupt replicas if none are corrupt

2016-08-09 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413838#comment-15413838 ] Kihwal Lee commented on HDFS-10342: --- Committed this to trunk. > BlockManager#createLocatedBlocks should

[jira] [Commented] (HDFS-10342) BlockManager#createLocatedBlocks should not check corrupt replicas if none are corrupt

2016-08-09 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413835#comment-15413835 ] Kihwal Lee commented on HDFS-10342: --- +1 lgtm. This is a simple optimization for the common case, but

[jira] [Commented] (HDFS-10733) NameNode terminated after full GC thinking QJM is unresponsive.

2016-08-09 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413811#comment-15413811 ] Kihwal Lee commented on HDFS-10733: --- Can we something similar to HDFS-9107? > NameNode terminated after

[jira] [Commented] (HDFS-10342) BlockManager#createLocatedBlocks should not check corrupt replicas if none are corrupt

2016-08-09 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413785#comment-15413785 ] Kihwal Lee commented on HDFS-10342: --- TestTracing: HDFS-10732 TestRefreshUserMappings: HDFS-10738 >

[jira] [Moved] (HDFS-10738) Fix TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration test failure

2016-08-09 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee moved HADOOP-13469 to HDFS-10738: Component/s: (was: test) test Key:

[jira] [Commented] (HDFS-10723) TestRenameWhileOpen tries to restart NN with the same rpc port

2016-08-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412331#comment-15412331 ] Kihwal Lee commented on HDFS-10723: --- When a {{MiniDFSCluster}} is brought up without setting the rpc

[jira] [Comment Edited] (HDFS-10723) TestRenameWhileOpen tries to restart NN with the same rpc port

2016-08-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409513#comment-15409513 ] Kihwal Lee edited comment on HDFS-10723 at 8/8/16 7:10 PM: --- It is more likely

[jira] [Updated] (HDFS-10730) Fix some failed tests due to BindException

2016-08-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10730: -- Description: In HDFS-10723, [~kihwal] suggested that {quote} it is not a good idea to hard-code or

[jira] [Commented] (HDFS-10342) BlockManager#createLocatedBlocks should not check corrupt replicas if none are corrupt

2016-08-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412148#comment-15412148 ] Kihwal Lee commented on HDFS-10342: --- [~kshukla], can you rebase the patch? >

[jira] [Commented] (HDFS-10729) NameNode crashes when loading edits because max directory items is exceeded

2016-08-08 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411901#comment-15411901 ] Kihwal Lee commented on HDFS-10729: --- Do you know how it got into this state? If the serving namenode

[jira] [Commented] (HDFS-9958) BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages.

2016-08-05 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409698#comment-15409698 ] Kihwal Lee commented on HDFS-9958: -- I found out missed branch-2.8. Just picked it to branch-2.8. >

[jira] [Updated] (HDFS-10723) TestRenameWhileOpen tries to restart NN with the same rpc port

2016-08-05 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10723: -- Attachment: testWhileOpenRenameParentToNonexistentDir.txt Here is the log from a failed test run. >

[jira] [Commented] (HDFS-10723) TestRenameWhileOpen tries to restart NN with the same rpc port

2016-08-05 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409513#comment-15409513 ] Kihwal Lee commented on HDFS-10723: --- It is more likely that another process grabbed the port. In general

[jira] [Commented] (HDFS-10727) Block move is failed: opReplaceBlock - Mover

2016-08-05 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409456#comment-15409456 ] Kihwal Lee commented on HDFS-10727: --- bq. 2) java.io.EOFException: Premature EOF: no length prefix

[jira] [Updated] (HDFS-10673) Optimize FSPermissionChecker's internal path usage

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10673: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10673) Optimize FSPermissionChecker's internal path usage

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408501#comment-15408501 ] Kihwal Lee commented on HDFS-10673: --- Committed this to trunk and branch-2. Thanks for the reviews and

[jira] [Updated] (HDFS-10343) BlockManager#createLocatedBlocks may return blocks on failed storages

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10343: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10343) BlockManager#createLocatedBlocks may return blocks on failed storages

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408421#comment-15408421 ] Kihwal Lee commented on HDFS-10343: --- The array trimming should be a rare event, as failed storage is

[jira] [Commented] (HDFS-10722) Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408415#comment-15408415 ] Kihwal Lee commented on HDFS-10722: --- Thanks for catching that, [~arpitagarwal]. I try building/testing

[jira] [Updated] (HDFS-10717) Fix findbugs warnings of hadoop-hdfs-client in branch-2

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10717: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved

[jira] [Commented] (HDFS-10717) Fix findbugs warnings of hadoop-hdfs-client in branch-2

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408177#comment-15408177 ] Kihwal Lee commented on HDFS-10717: --- The one in {{DataStreamer}} seems to have been introduced by

[jira] [Updated] (HDFS-10722) Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10722: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10722) Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408108#comment-15408108 ] Kihwal Lee commented on HDFS-10722: --- +1 > Fix race condition in

[jira] [Commented] (HDFS-10722) Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408107#comment-15408107 ] Kihwal Lee commented on HDFS-10722: --- Filed HDFS-10723 for {{TestRenameWhileOpen}}. HDFS-6532 is for

[jira] [Created] (HDFS-10723) TestRenameWhileOpen tries to restart NN with the same rpc port

2016-08-04 Thread Kihwal Lee (JIRA)
Kihwal Lee created HDFS-10723: - Summary: TestRenameWhileOpen tries to restart NN with the same rpc port Key: HDFS-10723 URL: https://issues.apache.org/jira/browse/HDFS-10723 Project: Hadoop HDFS

[jira] [Commented] (HDFS-6532) Intermittent test failure org.apache.hadoop.hdfs.TestCrcCorruption.testCorruptionDuringWrt

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408077#comment-15408077 ] Kihwal Lee commented on HDFS-6532: -- Still sometimes failing in trunk the same way. When it is working,

[jira] [Commented] (HDFS-10715) NPE when applying AvailableSpaceBlockPlacementPolicy

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408002#comment-15408002 ] Kihwal Lee commented on HDFS-10715: --- [~zhuguangbin86], please read

[jira] [Commented] (HDFS-10673) Optimize FSPermissionChecker's internal path usage

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407880#comment-15407880 ] Kihwal Lee commented on HDFS-10673: --- Let's preserve the old behavior for now and discuss potential

[jira] [Updated] (HDFS-10662) Optimize UTF8 string/byte conversions

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10662: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Updated] (HDFS-10707) Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10707: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved

[jira] [Commented] (HDFS-10707) Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets

2016-08-04 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15407846#comment-15407846 ] Kihwal Lee commented on HDFS-10707: --- +1 for the branch-2 patch. It looks like the checkstyle issue is

<    6   7   8   9   10   11   12   13   14   15   >