[jira] [Updated] (HDFS-10945) Fix the Findbugwaring FSNamesystem#renameTo(String, String, boolean) in branch-2

2016-10-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10945: -- Assignee: Brahma Reddy Battula > Fix the Findbugwaring FSNamesystem#renameTo(String, String, boolean)

[jira] [Commented] (HDFS-10943) rollEditLog expects empty EditsDoubleBuffer.bufCurrent which is not guaranteed

2016-10-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542701#comment-15542701 ] Kihwal Lee commented on HDFS-10943: --- Which version are you looking at? {{rollEditLog()}} does not and

[jira] [Commented] (HDFS-10714) Issue in handling checksum errors in write pipeline when fault DN is LAST_IN_PIPELINE

2016-10-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542658#comment-15542658 ] Kihwal Lee commented on HDFS-10714: --- +1 on DN remembering what it did during a recovery and being more

[jira] [Updated] (HDFS-10619) Cache path in InodesInPath

2016-10-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10619: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10619) Cache path in InodesInPath

2016-10-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542512#comment-15542512 ] Kihwal Lee commented on HDFS-10619: --- +1 simple and effective. > Cache path in InodesInPath >

[jira] [Commented] (HDFS-10851) FSDirStatAndListingOp: stop passing path as string

2016-10-03 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542504#comment-15542504 ] Kihwal Lee commented on HDFS-10851: --- The failed branch-2 test case passes reliably on my machine.

[jira] [Updated] (HDFS-10851) FSDirStatAndListingOp: stop passing path as string

2016-09-30 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10851: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10851) FSDirStatAndListingOp: stop passing path as string

2016-09-30 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15537052#comment-15537052 ] Kihwal Lee commented on HDFS-10851: --- The branch-2 patch looks good. The only difference is the

[jira] [Commented] (HDFS-10851) FSDirStatAndListingOp: stop passing path as string

2016-09-30 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15536610#comment-15536610 ] Kihwal Lee commented on HDFS-10851: --- Committed to trunk, but it does not apply to branch-2. >

[jira] [Commented] (HDFS-10851) FSDirStatAndListingOp: stop passing path as string

2016-09-30 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15536593#comment-15536593 ] Kihwal Lee commented on HDFS-10851: --- Other than the IIP/src optimizations, I see nice clean-ups and

[jira] [Updated] (HDFS-10850) getEZForPath should NOT throw FNF

2016-09-30 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10850: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10850) getEZForPath should NOT throw FNF

2016-09-30 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15536040#comment-15536040 ] Kihwal Lee commented on HDFS-10850: --- +1 for the patch. > getEZForPath should NOT throw FNF >

[jira] [Issue Comment Deleted] (HDFS-10850) getEZForPath should NOT throw FNF

2016-09-30 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10850: -- Comment: was deleted (was: +1 for the latest patch.) > getEZForPath should NOT throw FNF >

[jira] [Commented] (HDFS-10850) getEZForPath should NOT throw FNF

2016-09-30 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15536034#comment-15536034 ] Kihwal Lee commented on HDFS-10850: --- +1 for the latest patch. > getEZForPath should NOT throw FNF >

[jira] [Commented] (HDFS-10850) getEZForPath should NOT throw FNF

2016-09-29 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533884#comment-15533884 ] Kihwal Lee commented on HDFS-10850: --- I tried reverting and compared with the patch. It looks like the

[jira] [Commented] (HDFS-10789) Route webhdfs through the RPC call queue

2016-09-29 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533868#comment-15533868 ] Kihwal Lee commented on HDFS-10789: --- The patch does not apply anymore. Please rebase/update. > Route

[jira] [Commented] (HDFS-10850) getEZForPath should NOT throw FNF

2016-09-29 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533208#comment-15533208 ] Kihwal Lee commented on HDFS-10850: --- +1 on revert. We can then reopen HDFS-9433 and decide what to do.

[jira] [Updated] (HDFS-10779) Rename does not need to re-solve destination

2016-09-28 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10779: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10779) Rename does not need to re-solve destination

2016-09-28 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15531097#comment-15531097 ] Kihwal Lee commented on HDFS-10779: --- +1 nice improvement to not re-resolve the same thing multiple

[jira] [Commented] (HDFS-10911) Change edit log OP_UPDATE_BLOCKS to store delta blocks only.

2016-09-28 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15530215#comment-15530215 ] Kihwal Lee commented on HDFS-10911: --- IIRC, namenode didn't use to log all blocks all the time, but it

[jira] [Commented] (HDFS-10919) Provide admin/debug tool to dump out info of a given block

2016-09-28 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15530196#comment-15530196 ] Kihwal Lee commented on HDFS-10919: --- Given a block ID you can use {{fsck -blockId}} to get the file it

[jira] [Commented] (HDFS-10887) Provide admin/debug tool to dump block map

2016-09-26 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523305#comment-15523305 ] Kihwal Lee commented on HDFS-10887: --- Missing blocks during start-up is caused by datanodes not sending

[jira] [Updated] (HDFS-10887) Provide admin/debug tool to dump block map

2016-09-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10887: -- Issue Type: New Feature (was: Bug) > Provide admin/debug tool to dump block map >

[jira] [Commented] (HDFS-10887) Provide admin/debug tool to dump block map

2016-09-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513408#comment-15513408 ] Kihwal Lee commented on HDFS-10887: --- Perhaps, a less expensive diagnostics will be looking at

[jira] [Commented] (HDFS-10857) Rolling upgrade can make data unavailable when the cluster has many failed volumes

2016-09-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504768#comment-15504768 ] Kihwal Lee commented on HDFS-10857: --- The patch partially brings in changes available in branch-2.7 to

[jira] [Updated] (HDFS-10857) Rolling upgrade can make data unavailable when the cluster has many failed volumes

2016-09-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10857: -- Status: Patch Available (was: Open) > Rolling upgrade can make data unavailable when the cluster has

[jira] [Updated] (HDFS-10857) Rolling upgrade can make data unavailable when the cluster has many failed volumes

2016-09-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10857: -- Attachment: HDFS-10857.branch-2.6.patch > Rolling upgrade can make data unavailable when the cluster

[jira] [Commented] (HDFS-8865) Improve quota initialization performance

2016-09-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504488#comment-15504488 ] Kihwal Lee commented on HDFS-8865: -- Just added you as a HDFS contributor. > Improve quota initialization

[jira] [Commented] (HDFS-8865) Improve quota initialization performance

2016-09-19 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503897#comment-15503897 ] Kihwal Lee commented on HDFS-8865: -- [~nfraison.criteo], if you plan to contribute code to the HDFS project

[jira] [Commented] (HDFS-10475) Adding metrics for long FSNamesystem read and write locks

2016-09-14 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15490522#comment-15490522 ] Kihwal Lee commented on HDFS-10475: --- While I think this is a very useful feature, I am afraid it might

[jira] [Commented] (HDFS-10857) Rolling upgrade can make data unavailable when the cluster has many failed volumes

2016-09-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484851#comment-15484851 ] Kihwal Lee commented on HDFS-10857: --- We can have {{FsDatasetImpl#checkDataDir()}} remove the failed

[jira] [Updated] (HDFS-10857) Rolling upgrade can make data unavailable when the cluster has many failed volumes

2016-09-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10857: -- Target Version/s: 2.6.5 (was: 2.6.4) > Rolling upgrade can make data unavailable when the cluster has

[jira] [Updated] (HDFS-10857) Rolling upgrade can make data unavailable when the cluster has many failed volumes

2016-09-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10857: -- Affects Version/s: (was: 2.7.4) 2.6.4 > Rolling upgrade can make data

[jira] [Comment Edited] (HDFS-10857) Rolling upgrade can make data unavailable when the cluster has many failed volumes

2016-09-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484751#comment-15484751 ] Kihwal Lee edited comment on HDFS-10857 at 9/12/16 6:11 PM: It looks like it

[jira] [Updated] (HDFS-10857) Rolling upgrade can make data unavailable when the cluster has many failed volumes

2016-09-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10857: -- Assignee: (was: Kihwal Lee) > Rolling upgrade can make data unavailable when the cluster has many

[jira] [Updated] (HDFS-10857) Rolling upgrade can make data unavailable when the cluster has many failed volumes

2016-09-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10857: -- Target Version/s: 2.6.4 (was: 2.7.4) > Rolling upgrade can make data unavailable when the cluster has

[jira] [Commented] (HDFS-10857) Rolling upgrade can make data unavailable when the cluster has many failed volumes

2016-09-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484751#comment-15484751 ] Kihwal Lee commented on HDFS-10857: --- It looks like it is fixed in 2.8 and later.

[jira] [Updated] (HDFS-10857) Rolling upgrade can make data unavailable when the cluster has many failed volumes

2016-09-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10857: -- Target Version/s: 2.7.4 (was: 2.8.0, 2.7.4) > Rolling upgrade can make data unavailable when the

[jira] [Updated] (HDFS-10857) Rolling upgrade can make data unavailable when the cluster has many failed volumes

2016-09-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10857: -- Affects Version/s: 2.7.4 > Rolling upgrade can make data unavailable when the cluster has many failed

[jira] [Assigned] (HDFS-10857) Rolling upgrade can make data unavailable when the cluster has many failed volumes

2016-09-12 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reassigned HDFS-10857: - Assignee: Kihwal Lee > Rolling upgrade can make data unavailable when the cluster has many

[jira] [Created] (HDFS-10857) Rolling upgrade can make data unavailable when the cluster has many failed volumes

2016-09-12 Thread Kihwal Lee (JIRA)
Kihwal Lee created HDFS-10857: - Summary: Rolling upgrade can make data unavailable when the cluster has many failed volumes Key: HDFS-10857 URL: https://issues.apache.org/jira/browse/HDFS-10857 Project:

[jira] [Updated] (HDFS-10842) Prevent IncrementalBlockReport processing before the first FullBlockReport has been received

2016-09-07 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10842: -- Description: In a scenario where the initial full block report of a Datanode is delayed after a

[jira] [Commented] (HDFS-10841) Remove duplicate or unused variable in appendFile()

2016-09-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467875#comment-15467875 ] Kihwal Lee commented on HDFS-10841: --- >From https://builds.apache.org/job/PreCommit-Admin/263621

[jira] [Updated] (HDFS-10841) Remove duplicate or unused variable in appendFile()

2016-09-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10841: -- Priority: Minor (was: Major) > Remove duplicate or unused variable in appendFile() >

[jira] [Updated] (HDFS-10841) Remove duplicate or unused variable in appendFile()

2016-09-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10841: -- Status: Patch Available (was: Open) > Remove duplicate or unused variable in appendFile() >

[jira] [Updated] (HDFS-10841) Remove duplicate or unused variable in appendFile()

2016-09-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10841: -- Attachment: HDFS-10841.patch Even in trunk, {{src}} is only used for a logging statement in the method.

[jira] [Updated] (HDFS-10841) Remove duplicate or unused variable in appendFile()

2016-09-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10841: -- Summary: Remove duplicate or unused variable in appendFile() (was: Fix findbugs warning in branch-2

[jira] [Updated] (HDFS-10841) Fix findbugs warning in branch-2 introduced by path handling improvements

2016-09-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10841: -- Target Version/s: 2.8.0 (was: 2.8.0, 2.9.0) > Fix findbugs warning in branch-2 introduced by path

[jira] [Assigned] (HDFS-10841) Fix findbugs warning in branch-2 introduced by path handling improvements

2016-09-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reassigned HDFS-10841: - Assignee: Kihwal Lee (was: Daryn Sharp) > Fix findbugs warning in branch-2 introduced by path

[jira] [Updated] (HDFS-10841) Fix findbugs warning in branch-2 introduced by path handling improvements

2016-09-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10841: -- Assignee: Daryn Sharp > Fix findbugs warning in branch-2 introduced by path handling improvements >

[jira] [Updated] (HDFS-10841) Fix findbugs warning in branch-2 introduced by path handling improvements

2016-09-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10841: -- Description: A new findbugs warning was reported (See HDFS-10745). branch-2.8 might have the same

[jira] [Commented] (HDFS-10745) Directly resolve paths into INodesInPath

2016-09-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467655#comment-15467655 ] Kihwal Lee commented on HDFS-10745: --- Hmm the branch-2 precommit didn't complain about the new findbugs

[jira] [Updated] (HDFS-10841) Fix findbugs warning in branch-2 introduced by path handling improvements

2016-09-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10841: -- Target Version/s: 2.8.0, 2.9.0 > Fix findbugs warning in branch-2 introduced by path handling

[jira] [Created] (HDFS-10841) Fix findbugs warning in branch-2 introduced by path handling improvements

2016-09-06 Thread Kihwal Lee (JIRA)
Kihwal Lee created HDFS-10841: - Summary: Fix findbugs warning in branch-2 introduced by path handling improvements Key: HDFS-10841 URL: https://issues.apache.org/jira/browse/HDFS-10841 Project: Hadoop

[jira] [Commented] (HDFS-10789) Route webhdfs through the RPC call queue

2016-09-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459176#comment-15459176 ] Kihwal Lee commented on HDFS-10789: --- What happens when the call queue is full and backoff is on? What

[jira] [Commented] (HDFS-10772) Reduce byte/string conversions for get listing

2016-09-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459119#comment-15459119 ] Kihwal Lee commented on HDFS-10772: --- Found its way to branch-2.8. > Reduce byte/string conversions for

[jira] [Updated] (HDFS-10772) Reduce byte/string conversions for get listing

2016-09-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10772: -- Fix Version/s: (was: 2.9.0) 2.8.0 > Reduce byte/string conversions for get

[jira] [Updated] (HDFS-10768) Optimize mkdir ops

2016-09-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10768: -- Fix Version/s: (was: 2.9.0) 2.8.0 > Optimize mkdir ops > -- > >

[jira] [Commented] (HDFS-10768) Optimize mkdir ops

2016-09-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459109#comment-15459109 ] Kihwal Lee commented on HDFS-10768: --- Picked to 2.8. > Optimize mkdir ops > -- > >

[jira] [Updated] (HDFS-10762) Pass IIP for file status related methods

2016-09-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10762: -- Fix Version/s: (was: 2.9.0) 2.8.0 > Pass IIP for file status related methods >

[jira] [Commented] (HDFS-10762) Pass IIP for file status related methods

2016-09-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459093#comment-15459093 ] Kihwal Lee commented on HDFS-10762: --- Commited this to branch-2.8. > Pass IIP for file status related

[jira] [Commented] (HDFS-10762) Pass IIP for file status related methods

2016-09-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459087#comment-15459087 ] Kihwal Lee commented on HDFS-10762: --- Cherry-picked HDFS-9621 to branch-2.8, which moves target into IIP.

[jira] [Updated] (HDFS-9621) getListing wrongly associates Erasure Coding policy to pre-existing replicated files under an EC directory

2016-09-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-9621: - Fix Version/s: (was: 2.9.0) 2.8.0 > getListing wrongly associates Erasure Coding

[jira] [Commented] (HDFS-9621) getListing wrongly associates Erasure Coding policy to pre-existing replicated files under an EC directory

2016-09-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459084#comment-15459084 ] Kihwal Lee commented on HDFS-9621: -- Picked to branch-2.8. Although it does not have the same bug, but it

[jira] [Updated] (HDFS-10745) Directly resolve paths into INodesInPath

2016-09-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10745: -- Fix Version/s: (was: 2.9.0) 2.8.0 > Directly resolve paths into INodesInPath >

[jira] [Commented] (HDFS-10745) Directly resolve paths into INodesInPath

2016-09-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459012#comment-15459012 ] Kihwal Lee commented on HDFS-10745: --- Committed to branch-2.8. > Directly resolve paths into

[jira] [Updated] (HDFS-10744) Internally optimize path component resolution

2016-09-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10744: -- Fix Version/s: (was: 2.9.0) 2.8.0 > Internally optimize path component

[jira] [Commented] (HDFS-10744) Internally optimize path component resolution

2016-09-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458990#comment-15458990 ] Kihwal Lee commented on HDFS-10744: --- Committed to branch-2.8. > Internally optimize path component

[jira] [Updated] (HDFS-10711) Optimize FSPermissionChecker group membership check

2016-09-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10711: -- Fix Version/s: (was: 2.9.0) 2.8.0 > Optimize FSPermissionChecker group

[jira] [Commented] (HDFS-10711) Optimize FSPermissionChecker group membership check

2016-09-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458975#comment-15458975 ] Kihwal Lee commented on HDFS-10711: --- Committed to 2.8. > Optimize FSPermissionChecker group membership

[jira] [Commented] (HDFS-10662) Optimize UTF8 string/byte conversions

2016-09-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15455967#comment-15455967 ] Kihwal Lee commented on HDFS-10662: --- [~zhz], the patch looks good in general. There is an unused import

[jira] [Commented] (HDFS-10662) Optimize UTF8 string/byte conversions

2016-09-01 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15455869#comment-15455869 ] Kihwal Lee commented on HDFS-10662: --- [~zhz], I will take a look. The 2.7 precommit has more test

[jira] [Commented] (HDFS-10822) Log DataNodes in the write pipeline

2016-08-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453244#comment-15453244 ] Kihwal Lee commented on HDFS-10822: --- bq. Trying to diagnose a slow HDFS flush, taking longer than 10

[jira] [Updated] (HDFS-10729) Improve log message for edit loading failures caused by FS limit checks.

2016-08-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10729: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Updated] (HDFS-10729) Improve log message for edit loading failures caused by FS limit checks.

2016-08-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10729: -- Priority: Major (was: Critical) > Improve log message for edit loading failures caused by FS limit

[jira] [Commented] (HDFS-10729) Improve log message for edit loading failures caused by FS limit checks.

2016-08-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453084#comment-15453084 ] Kihwal Lee commented on HDFS-10729: --- +1 the patch looks good. > Improve log message for edit loading

[jira] [Updated] (HDFS-10729) Improve log message for edit loading failures caused by FS limit checks.

2016-08-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10729: -- Summary: Improve log message for edit loading failures caused by FS limit checks. (was: NameNode

[jira] [Commented] (HDFS-10729) NameNode crashes when loading edits because max directory items is exceeded

2016-08-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15452990#comment-15452990 ] Kihwal Lee commented on HDFS-10729: --- They pass for me too. {noformat}

[jira] [Commented] (HDFS-10480) Add an admin command to list currently open files

2016-08-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15452714#comment-15452714 ] Kihwal Lee commented on HDFS-10480: --- The superuser-only admin command is certainly useful, but letting

[jira] [Updated] (HDFS-10768) Optimize mkdir ops

2016-08-26 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10768: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10768) Optimize mkdir ops

2016-08-26 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439788#comment-15439788 ] Kihwal Lee commented on HDFS-10768: --- +1 the patch looks good. > Optimize mkdir ops > --

[jira] [Commented] (HDFS-9145) Tracking methods that hold FSNamesytemLock for too long

2016-08-26 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439242#comment-15439242 ] Kihwal Lee commented on HDFS-9145: -- I noticed that 2.7 CHANGES.txt has no entry for this and few other

[jira] [Updated] (HDFS-8915) TestFSNamesystem.testFSLockGetWaiterCount fails intermittently in jenkins

2016-08-26 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-8915: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2 2.7.4

[jira] [Commented] (HDFS-8915) TestFSNamesystem.testFSLockGetWaiterCount fails intermittently in jenkins

2016-08-26 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439180#comment-15439180 ] Kihwal Lee commented on HDFS-8915: -- {{testFSLockGetWaiterCount}} is very flaky in branch-2.7 for me and

[jira] [Commented] (HDFS-9145) Tracking methods that hold FSNamesytemLock for too long

2016-08-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15437863#comment-15437863 ] Kihwal Lee commented on HDFS-9145: -- I can reproduce it consistently. But when I tried the approach in the

[jira] [Updated] (HDFS-9145) Tracking methods that hold FSNamesytemLock for too long

2016-08-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-9145: - Attachment: testlog.txt I am not sure how helpful this log will be. I am using openjdk 1.8.0_101 on my

[jira] [Commented] (HDFS-9145) Tracking methods that hold FSNamesytemLock for too long

2016-08-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15437840#comment-15437840 ] Kihwal Lee commented on HDFS-9145: -- For me it fails when I run {{TestFSNamesystem}}, but passes when I

[jira] [Commented] (HDFS-9145) Tracking methods that hold FSNamesytemLock for too long

2016-08-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15437742#comment-15437742 ] Kihwal Lee commented on HDFS-9145: -- There is a test failure in branch-2.7 after this. {noformat}

[jira] [Commented] (HDFS-10768) Optimize mkdir ops

2016-08-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15437708#comment-15437708 ] Kihwal Lee commented on HDFS-10768: --- I will review the latest patch by tomorrow. > Optimize mkdir ops >

[jira] [Commented] (HDFS-10768) Optimize mkdir ops

2016-08-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15437701#comment-15437701 ] Kihwal Lee commented on HDFS-10768: --- {{TestEditLogJournalFailures}} was broken by HADOOP-13465. It has

[jira] [Commented] (HDFS-10768) Optimize mkdir ops

2016-08-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15437169#comment-15437169 ] Kihwal Lee commented on HDFS-10768: --- The test failure was already reported in HDFS-10498. I've added the

[jira] [Updated] (HDFS-10498) Intermittent test failure org.apache.hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength.testSnapshotfileLength

2016-08-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10498: -- Attachment: test_failure.txt Attaching a test log from a precommit build. The test calls {{append()}}

[jira] [Commented] (HDFS-10619) Cache path in InodesInPath

2016-08-25 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15437129#comment-15437129 ] Kihwal Lee commented on HDFS-10619: --- [~daryn], you need to rebase the patch. > Cache path in

[jira] [Commented] (HDFS-10768) Optimize mkdir ops

2016-08-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435633#comment-15435633 ] Kihwal Lee commented on HDFS-10768: --- The patch can apply now. Kicked a precommit build. > Optimize

[jira] [Updated] (HDFS-10772) Reduce byte/string conversions for get listing

2016-08-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10772: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10772) Reduce byte/string conversions for get listing

2016-08-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435606#comment-15435606 ] Kihwal Lee commented on HDFS-10772: --- +1 it looks straight-forward. > Reduce byte/string conversions for

[jira] [Commented] (HDFS-10788) fsck NullPointerException when it encounters corrupt replicas

2016-08-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435537#comment-15435537 ] Kihwal Lee commented on HDFS-10788: --- [~kshukla], I think one of your jiras fixed it. Please verify. >

[jira] [Commented] (HDFS-10763) Open files can leak permanently due to inconsistent lease update

2016-08-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435124#comment-15435124 ] Kihwal Lee commented on HDFS-10763: --- The one thing I had to do in the latest patch for branch-2.7 was to

[jira] [Updated] (HDFS-9696) Garbage snapshot records lingering forever

2016-08-24 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-9696: - Attachment: HDFS-9696.branch-2.6.patch I think it is worth having in branch-2.6. We would if we were still

[jira] [Commented] (HDFS-10762) Pass IIP for file status related methods

2016-08-23 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433628#comment-15433628 ] Kihwal Lee commented on HDFS-10762: --- +1 > Pass IIP for file status related methods >

<    5   6   7   8   9   10   11   12   13   14   >