[jira] [Comment Edited] (HDFS-8088) Reduce the number of HTrace spans generated by HDFS reads

2015-04-08 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14486605#comment-14486605 ] Yi Liu edited comment on HDFS-8088 at 4/9/15 2:53 AM: -- Colin, thanks

[jira] [Commented] (HDFS-8088) Reduce the number of HTrace spans generated by HDFS reads

2015-04-08 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14486605#comment-14486605 ] Yi Liu commented on HDFS-8088: -- Colin, seems today is the deadline of 2.7 release? I have not

[jira] [Commented] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

2015-04-07 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482681#comment-14482681 ] Yi Liu commented on HDFS-8058: -- Thanks Jing for the comments, I will close it as invalid.

[jira] [Commented] (HDFS-8059) Erasure coding: move dataBlockNum and parityBlockNum from BlockInfoStriped to INodeFile

2015-04-07 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482673#comment-14482673 ] Yi Liu commented on HDFS-8059: -- Thanks Jing for your discussion! let's leave it firstly, and

[jira] [Resolved] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

2015-04-07 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu resolved HDFS-8058. -- Resolution: Invalid Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

[jira] [Updated] (HDFS-8065) Erasure coding: Support truncate at striped group boundary.

2015-04-07 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8065: - Parent Issue: HDFS-8031 (was: HDFS-7285) Erasure coding: Support truncate at striped group boundary.

[jira] [Commented] (HDFS-7348) Erasure Coding: perform stripping erasure decoding/recovery work given block reader and writer

2015-04-07 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484434#comment-14484434 ] Yi Liu commented on HDFS-7348: -- Thanks Kai, Zhe. Erasure Coding: perform stripping erasure

[jira] [Created] (HDFS-8087) Erasure Coding: Add more EC zone management APIs (get/list EC zone(s))

2015-04-07 Thread Yi Liu (JIRA)
Yi Liu created HDFS-8087: Summary: Erasure Coding: Add more EC zone management APIs (get/list EC zone(s)) Key: HDFS-8087 URL: https://issues.apache.org/jira/browse/HDFS-8087 Project: Hadoop HDFS

[jira] [Commented] (HDFS-8081) Split getAdditionalBlock() into two methods.

2015-04-07 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484756#comment-14484756 ] Yi Liu commented on HDFS-8081: -- Hi Konst, the patch looks good. Few nits: *1.* {quote} //

[jira] [Commented] (HDFS-8059) Erasure coding: move dataBlockNum and parityBlockNum from BlockInfoStriped to INodeFile

2015-04-06 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482483#comment-14482483 ] Yi Liu commented on HDFS-8059: -- Jing, Thanks for your detailed and nice comments ! I agree

[jira] [Updated] (HDFS-8071) Redundant checkFileProgress() in PART II of getAdditionalBlock()

2015-04-06 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8071: - Assignee: Konstantin Shvachko Redundant checkFileProgress() in PART II of getAdditionalBlock()

[jira] [Commented] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

2015-04-06 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482423#comment-14482423 ] Yi Liu commented on HDFS-8058: -- Zhe, Jing, thanks for your comments! {quote} The only concern

[jira] [Commented] (HDFS-7969) Erasure coding: NameNode support for lease recovery of striped block groups

2015-04-06 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14480995#comment-14480995 ] Yi Liu commented on HDFS-7969: -- Hi Zhe, The logic in {{initializeBlockRecovery}} to choose

[jira] [Resolved] (HDFS-8066) Erasure coding: Decommission handle for EC blocks.

2015-04-06 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu resolved HDFS-8066. -- Resolution: Implemented Take a look at the latest branch, this has been implemented in branch, so close it as

[jira] [Created] (HDFS-8065) Support truncate at striped group boundary.

2015-04-06 Thread Yi Liu (JIRA)
Yi Liu created HDFS-8065: Summary: Support truncate at striped group boundary. Key: HDFS-8065 URL: https://issues.apache.org/jira/browse/HDFS-8065 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-8064) Erasure coding: DataNode support for block recovery of striped block groups

2015-04-06 Thread Yi Liu (JIRA)
Yi Liu created HDFS-8064: Summary: Erasure coding: DataNode support for block recovery of striped block groups Key: HDFS-8064 URL: https://issues.apache.org/jira/browse/HDFS-8064 Project: Hadoop HDFS

[jira] [Updated] (HDFS-8065) Erasure coding: Support truncate at striped group boundary.

2015-04-06 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8065: - Summary: Erasure coding: Support truncate at striped group boundary. (was: Support truncate at striped group

[jira] [Created] (HDFS-8066) Erasure coding: Decommission handle for EC blocks.

2015-04-06 Thread Yi Liu (JIRA)
Yi Liu created HDFS-8066: Summary: Erasure coding: Decommission handle for EC blocks. Key: HDFS-8066 URL: https://issues.apache.org/jira/browse/HDFS-8066 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Updated] (HDFS-8059) Erasure coding: move dataBlockNum and parityBlockNum from BlockInfoStriped to INodeFile

2015-04-05 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8059: - Attachment: HDFS-8059.001.patch Yes Zhe. Upload the patch, it requires HDFS-8058. Erasure coding: move

[jira] [Commented] (HDFS-8032) Erasure coding: shrink size of indices in BlockInfoStriped

2015-04-05 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14396121#comment-14396121 ] Yi Liu commented on HDFS-8032: -- We can use 4bits to represent an index, then few bytes

[jira] [Work started] (HDFS-8059) Move dataBlockNum and parityBlockNum from BlockInfoStriped to INodeFile

2015-04-03 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-8059 started by Yi Liu. Move dataBlockNum and parityBlockNum from BlockInfoStriped to INodeFile

[jira] [Created] (HDFS-8059) Move dataBlockNum and parityBlockNum from BlockInfoStriped to INodeFile

2015-04-03 Thread Yi Liu (JIRA)
Yi Liu created HDFS-8059: Summary: Move dataBlockNum and parityBlockNum from BlockInfoStriped to INodeFile Key: HDFS-8059 URL: https://issues.apache.org/jira/browse/HDFS-8059 Project: Hadoop HDFS

[jira] [Updated] (HDFS-8059) Erasure coding: move dataBlockNum and parityBlockNum from BlockInfoStriped to INodeFile

2015-04-03 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8059: - Summary: Erasure coding: move dataBlockNum and parityBlockNum from BlockInfoStriped to INodeFile (was: Move

[jira] [Created] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

2015-04-03 Thread Yi Liu (JIRA)
Yi Liu created HDFS-8058: Summary: Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile Key: HDFS-8058 URL: https://issues.apache.org/jira/browse/HDFS-8058 Project: Hadoop HDFS

[jira] [Updated] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

2015-04-03 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8058: - Attachment: HDFS-8058.001.patch Attach the patch. In current logic, If file is striped, only

[jira] [Comment Edited] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

2015-04-03 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394532#comment-14394532 ] Yi Liu edited comment on HDFS-8058 at 4/3/15 2:59 PM: -- Attach the

[jira] [Work started] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

2015-04-03 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-8058 started by Yi Liu. Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

[jira] [Updated] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

2015-04-03 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8058: - Description: This JIRA is to use {{BlockInfo[] blocks}} for both striped and contiguous blocks in INodeFile.

[jira] [Commented] (HDFS-8059) Erasure coding: move dataBlockNum and parityBlockNum from BlockInfoStriped to INodeFile

2015-04-03 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394642#comment-14394642 ] Yi Liu commented on HDFS-8059: -- Will upload the patch later. Erasure coding: move

[jira] [Updated] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

2015-04-03 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8058: - Attachment: HDFS-8058.002.patch Fix test failure of original code and NPE. Erasure coding: use BlockInfo[] for

[jira] [Commented] (HDFS-7916) 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop

2015-04-01 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390480#comment-14390480 ] Yi Liu commented on HDFS-7916: -- Thanks Vinay for the patch, it looks good, one comment, +1

[jira] [Commented] (HDFS-7916) 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop

2015-04-01 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14391839#comment-14391839 ] Yi Liu commented on HDFS-7916: -- +1 'reportBadBlocks' from datanodes to standby Node

[jira] [Updated] (HDFS-7997) The first non-existing xattr should also throw IOException

2015-03-31 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7997: - Resolution: Fixed Fix Version/s: 2.8.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch

[jira] [Commented] (HDFS-8026) Trace FSOutputSummer#writeChecksumChunks rather than DFSOutputStream#writeChunk

2015-03-31 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14388512#comment-14388512 ] Yi Liu commented on HDFS-8026: -- TestTracing fails again. Trace

[jira] [Commented] (HDFS-8026) Trace FSOutputSummer#writeChecksumChunks rather than DFSOutputStream#writeChunk

2015-03-31 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14389778#comment-14389778 ] Yi Liu commented on HDFS-8026: -- Yes, please go ahead to commit it, thanks Colin. I remember

[jira] [Comment Edited] (HDFS-8026) Trace FSOutputSummer#writeChecksumChunks rather than DFSOutputStream#writeChunk

2015-03-31 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14389778#comment-14389778 ] Yi Liu edited comment on HDFS-8026 at 4/1/15 12:42 AM: --- Yes, please

[jira] [Commented] (HDFS-7964) Add support for async edit logging

2015-03-30 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14386638#comment-14386638 ] Yi Liu commented on HDFS-7964: -- [~daryn], this is a cool and important improvement. Besides

[jira] [Commented] (HDFS-8026) Trace FSOutputSummer#writeChecksumChunks rather than DFSOutputStream#writeChunk

2015-03-30 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387903#comment-14387903 ] Yi Liu commented on HDFS-8026: -- +1 Trace FSOutputSummer#writeChecksumChunks rather than

[jira] [Commented] (HDFS-8007) Lower RollingWindowManager log to debug

2015-03-29 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14386082#comment-14386082 ] Yi Liu commented on HDFS-8007: -- +1, thanks Andrew. Lower RollingWindowManager log to debug

[jira] [Commented] (HDFS-7997) The first non-existing xattr should also throw IOException

2015-03-29 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14386177#comment-14386177 ] Yi Liu commented on HDFS-7997: -- [~sinago], the example you gave is not correct, the xattr

[jira] [Updated] (HDFS-7997) The first non-existing xattr should also throw IOException

2015-03-29 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7997: - Priority: Minor (was: Major) Target Version/s: 2.8.0 The first non-existing xattr should also throw

[jira] [Commented] (HDFS-7960) The full block report should prune zombie storages even if they're not empty

2015-03-24 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377728#comment-14377728 ] Yi Liu commented on HDFS-7960: -- Thanks Colin and Eddy for the patch, and Andrew for reviewing.

[jira] [Commented] (HDFS-7960) The full block report should prune zombie storages even if they're not empty

2015-03-23 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14375781#comment-14375781 ] Yi Liu commented on HDFS-7960: -- This is a good fix and improvement. Some comments: *1.* In

[jira] [Commented] (HDFS-7960) The full block report should prune zombie storages even if they're not empty

2015-03-23 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14376712#comment-14376712 ] Yi Liu commented on HDFS-7960: -- Some incorrect in my second comment: wo could not use

[jira] [Commented] (HDFS-7960) The full block report should prune zombie storages even if they're not empty

2015-03-23 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377060#comment-14377060 ] Yi Liu commented on HDFS-7960: -- Thanks Colin, Lei for updating the patch. It looks really

[jira] [Updated] (HDFS-7962) Remove duplicated logs in BlockManager

2015-03-20 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7962: - Resolution: Fixed Fix Version/s: 2.7.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch

[jira] [Commented] (HDFS-7930) commitBlockSynchronization() does not remove locations

2015-03-19 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368562#comment-14368562 ] Yi Liu commented on HDFS-7930: -- I think it's fine to remove the modification to

[jira] [Updated] (HDFS-7930) commitBlockSynchronization() does not remove locations

2015-03-19 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7930: - Attachment: HDFS-7930.003.patch commitBlockSynchronization() does not remove locations

[jira] [Comment Edited] (HDFS-7930) commitBlockSynchronization() does not remove locations

2015-03-19 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368539#comment-14368539 ] Yi Liu edited comment on HDFS-7930 at 3/19/15 6:14 AM: --- [~shv],

[jira] [Comment Edited] (HDFS-7930) commitBlockSynchronization() does not remove locations

2015-03-19 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368539#comment-14368539 ] Yi Liu edited comment on HDFS-7930 at 3/19/15 6:12 AM: --- [~shv],

[jira] [Commented] (HDFS-7256) Encryption Key created in Java Key Store after Namenode start unavailable for EZ Creation

2015-03-19 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370483#comment-14370483 ] Yi Liu commented on HDFS-7256: -- Hi Xiaoyu, {{hadoop.security.key.provider.path}} is only used

[jira] [Updated] (HDFS-7930) commitBlockSynchronization() does not remove locations

2015-03-19 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7930: - Resolution: Fixed Fix Version/s: 2.7.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch

[jira] [Issue Comment Deleted] (HDFS-7930) commitBlockSynchronization() does not remove locations

2015-03-19 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7930: - Comment: was deleted (was: {color:red}-1 overall{color}. Here are the results of testing the latest attachment

[jira] [Commented] (HDFS-7930) commitBlockSynchronization() does not remove locations

2015-03-19 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369254#comment-14369254 ] Yi Liu commented on HDFS-7930: -- {quote} If I add triggerBlockReports() before

[jira] [Updated] (HDFS-7962) Remove duplicated logs in BlockManager

2015-03-19 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7962: - Summary: Remove duplicated logs in BlockManager (was: Remove duplicated log in BlockManager) Remove duplicated

[jira] [Updated] (HDFS-7962) Remove duplicated log in BlockManager

2015-03-19 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7962: - Status: Patch Available (was: Open) Remove duplicated log in BlockManager -

[jira] [Updated] (HDFS-7962) Remove duplicated log in BlockManager

2015-03-19 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7962: - Attachment: HDFS-7962.001.patch Remove duplicated log in BlockManager -

[jira] [Created] (HDFS-7962) Remove duplicated log in BlockManager

2015-03-19 Thread Yi Liu (JIRA)
Yi Liu created HDFS-7962: Summary: Remove duplicated log in BlockManager Key: HDFS-7962 URL: https://issues.apache.org/jira/browse/HDFS-7962 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Commented] (HDFS-7930) commitBlockSynchronization() does not remove locations

2015-03-18 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368235#comment-14368235 ] Yi Liu commented on HDFS-7930: -- Thanks [~shv] for comments, I will update the patch and

[jira] [Commented] (HDFS-7930) commitBlockSynchronization() does not remove locations

2015-03-18 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368539#comment-14368539 ] Yi Liu commented on HDFS-7930: -- [~shv], thanks for your nice discussion. I took further look

[jira] [Updated] (HDFS-7838) Expose truncate API for libhdfs

2015-03-17 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7838: - Resolution: Fixed Fix Version/s: 2.7.0 Target Version/s: 2.7.0 Hadoop Flags: Reviewed

[jira] [Updated] (HDFS-7930) commitBlockSynchronization() does not remove locations

2015-03-17 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7930: - Attachment: HDFS-7930.002.patch Thanks [~shv] for review, update the patch to address all your comments.

[jira] [Updated] (HDFS-7930) commitBlockSynchronization() does not remove locations

2015-03-16 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7930: - Status: Patch Available (was: Open) commitBlockSynchronization() does not remove locations

[jira] [Updated] (HDFS-7930) commitBlockSynchronization() does not remove locations

2015-03-16 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7930: - Summary: commitBlockSynchronization() does not remove locations (was: commitBlockSynchronization() does not

[jira] [Updated] (HDFS-7930) commitBlockSynchronization() does not remove locations, which were not confirmed

2015-03-16 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7930: - Attachment: HDFS-7930.001.patch commitBlockSynchronization() does not remove locations, which were not

[jira] [Commented] (HDFS-7930) commitBlockSynchronization() does not remove locations

2015-03-16 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362929#comment-14362929 ] Yi Liu commented on HDFS-7930: -- I add a test in {{TestFileTruncate}} since it includes block

[jira] [Commented] (HDFS-7930) commitBlockSynchronization() does not remove locations

2015-03-16 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362924#comment-14362924 ] Yi Liu commented on HDFS-7930: -- [~shv], you are right. After block recovery, if

[jira] [Comment Edited] (HDFS-7930) commitBlockSynchronization() does not remove locations

2015-03-16 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362924#comment-14362924 ] Yi Liu edited comment on HDFS-7930 at 3/16/15 8:22 AM: --- [~shv], you

[jira] [Commented] (HDFS-7886) TestFileTruncate#testTruncateWithDataNodesRestart runs timeout sometimes

2015-03-16 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362931#comment-14362931 ] Yi Liu commented on HDFS-7886: -- Thanks Konst for the explanation, you are right. We still need

[jira] [Updated] (HDFS-7838) Expose truncate API for libhdfs

2015-03-16 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7838: - Attachment: HDFS-7838.004.patch I get your point now, update the patch. Thanks [~cmccabe] for review. Expose

[jira] [Commented] (HDFS-7926) NameNode implementation of ClientProtocol.truncate(..) is not idempotent

2015-03-13 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361552#comment-14361552 ] Yi Liu commented on HDFS-7926: -- You have a point there, thanks. NameNode implementation of

[jira] [Commented] (HDFS-7926) NameNode implementation of ClientProtocol.truncate(..) is not idempotent

2015-03-13 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361389#comment-14361389 ] Yi Liu commented on HDFS-7926: -- Thanks [~szetszwo] for the fix. Actually I agree that

[jira] [Assigned] (HDFS-7930) commitBlockSynchronization() does not remove locations, which were not confirmed

2015-03-13 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu reassigned HDFS-7930: Assignee: Yi Liu commitBlockSynchronization() does not remove locations, which were not confirmed

[jira] [Comment Edited] (HDFS-7926) NameNode implementation of ClientProtocol.truncate(..) is not idempotent

2015-03-13 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361389#comment-14361389 ] Yi Liu edited comment on HDFS-7926 at 3/14/15 12:29 AM: Thanks

[jira] [Commented] (HDFS-7926) NameNode implementation of ClientProtocol.truncate(..) is not idempotent

2015-03-13 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360020#comment-14360020 ] Yi Liu commented on HDFS-7926: -- {quote} No the file is for this JIRA. Could you take a look

[jira] [Commented] (HDFS-7926) NameNode implementation of ClientProtocol.truncate(..) is not idempotent

2015-03-12 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359854#comment-14359854 ] Yi Liu commented on HDFS-7926: -- Hi Nicholas, the patch is not for this JIRA, can you check?

[jira] [Assigned] (HDFS-7924) NameNode goes into infinite lease recovery

2015-03-12 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu reassigned HDFS-7924: Assignee: Yi Liu NameNode goes into infinite lease recovery --

[jira] [Commented] (HDFS-7921) FileSystem listFiles doesn't list the directories if recursive is false

2015-03-12 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359974#comment-14359974 ] Yi Liu commented on HDFS-7921: -- {quote} Is this the intended behavior? It is weird just to

[jira] [Commented] (HDFS-7886) TestFileTruncate#testTruncateWithDataNodesRestart runs timeout sometimes

2015-03-12 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359810#comment-14359810 ] Yi Liu commented on HDFS-7886: -- Konst, the patch looks good to me, just one nit: In

[jira] [Commented] (HDFS-7911) Buffer Overflow when running HBase on HDFS Encryption Zone

2015-03-12 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359720#comment-14359720 ] Yi Liu commented on HDFS-7911: -- As discussed in HADOOP-11708, I +1 for changing

[jira] [Resolved] (HDFS-7911) Buffer Overflow when running HBase on HDFS Encryption Zone

2015-03-12 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu resolved HDFS-7911. -- Resolution: Duplicate Buffer Overflow when running HBase on HDFS Encryption Zone

[jira] [Commented] (HDFS-7838) Expose truncate API for libhdfs

2015-03-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357879#comment-14357879 ] Yi Liu commented on HDFS-7838: -- {quote} +1 once those are addressed. {quote} Hi, [~cmccabe],

[jira] [Comment Edited] (HDFS-7838) Expose truncate API for libhdfs

2015-03-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357879#comment-14357879 ] Yi Liu edited comment on HDFS-7838 at 3/12/15 12:40 AM: {quote} +1

[jira] [Commented] (HDFS-7886) TestFileTruncate#testTruncateWithDataNodesRestart runs timeout sometimes

2015-03-11 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357877#comment-14357877 ] Yi Liu commented on HDFS-7886: -- Yes Konst, I agree with you, thanks.

[jira] [Commented] (HDFS-7838) Expose truncate API for libhdfs

2015-03-10 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354818#comment-14354818 ] Yi Liu commented on HDFS-7838: -- {quote} It seems confusing to translate false = 1 and true =

[jira] [Commented] (HDFS-7844) Create an off-heap hash table implementation

2015-03-10 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354853#comment-14354853 ] Yi Liu commented on HDFS-7844: -- Thanks Colin for updating the patch. {quote} I'm going to

[jira] [Commented] (HDFS-7911) Buffer Overflow when running HBase on HDFS Encryption Zone

2015-03-10 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354833#comment-14354833 ] Yi Liu commented on HDFS-7911: -- {{CryptoOutputStream}} is not thread-safe as it declares. To

[jira] [Updated] (HDFS-7902) Expose truncate API for libwebhdfs

2015-03-10 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7902: - Attachment: HDFS-7902.002.patch Expose truncate API for libwebhdfs --

[jira] [Commented] (HDFS-7846) Create off-heap BlocksMap and BlockData structures

2015-03-10 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354936#comment-14354936 ] Yi Liu commented on HDFS-7846: -- Thanks Colin for the patch, it looks good. Each shard includes

[jira] [Updated] (HDFS-7838) Expose truncate API for libhdfs

2015-03-10 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7838: - Attachment: HDFS-7838.003.patch Expose truncate API for libhdfs ---

[jira] [Comment Edited] (HDFS-7847) Modify NNThroughputBenchmark to be able to operate on a remote NameNode

2015-03-10 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353850#comment-14353850 ] Yi Liu edited comment on HDFS-7847 at 3/10/15 2:25 PM: --- Thanks

[jira] [Commented] (HDFS-7846) Create off-heap BlocksMap and BlockData structures

2015-03-10 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354980#comment-14354980 ] Yi Liu commented on HDFS-7846: -- Another comment is about block replicas: the replication

[jira] [Commented] (HDFS-7886) TestFileTruncate#testTruncateWithDataNodesRestart runs timeout sometimes

2015-03-10 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14355028#comment-14355028 ] Yi Liu commented on HDFS-7886: -- Thanks Konst and Plamen for the comments. Following is reply

[jira] [Commented] (HDFS-7844) Create an off-heap hash table implementation

2015-03-09 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352979#comment-14352979 ] Yi Liu commented on HDFS-7844: -- I think you can also address my new comments in following-on

[jira] [Commented] (HDFS-7886) TestFileTruncate#testTruncateWithDataNodesRestart runs timeout sometimes

2015-03-09 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352946#comment-14352946 ] Yi Liu commented on HDFS-7886: -- Thanks Konst for the detailed analyzing, also thanks Plamen.

[jira] [Comment Edited] (HDFS-7886) TestFileTruncate#testTruncateWithDataNodesRestart runs timeout sometimes

2015-03-09 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352946#comment-14352946 ] Yi Liu edited comment on HDFS-7886 at 3/9/15 1:05 PM: -- Thanks Konst

[jira] [Commented] (HDFS-7844) Create an off-heap hash table implementation

2015-03-09 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352975#comment-14352975 ] Yi Liu commented on HDFS-7844: -- Thanks Colin for addressing the comments, it's cool and only

[jira] [Assigned] (HDFS-7911) Buffer Overflow when running HBase on HDFS Encryption Zone

2015-03-09 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu reassigned HDFS-7911: Assignee: Yi Liu Buffer Overflow when running HBase on HDFS Encryption Zone

[jira] [Commented] (HDFS-7911) Buffer Overflow when running HBase on HDFS Encryption Zone

2015-03-09 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353967#comment-14353967 ] Yi Liu commented on HDFS-7911: -- Thanks [~xyao] for reporting this, I will take a look at this

[jira] [Updated] (HDFS-7902) Expose truncate API for libwebhdfs

2015-03-07 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7902: - Attachment: HDFS-7902.001.patch Upload the patch. Will submit it after HDFS-7838. Expose truncate API for

<    2   3   4   5   6   7   8   9   10   11   >