[jira] [Commented] (HDFS-11284) [SPS]: Avoid running SPS under safemode and fix issues in target node choosing.

2016-12-31 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15790521#comment-15790521 ] Yuanbo Liu commented on HDFS-11284: --- [~umamaheswararao] Thanks for your response. * The

[jira] [Comment Edited] (HDFS-11284) [SPS]: Avoid running SPS under safemode and fix issues in target node choosing.

2016-12-31 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15790521#comment-15790521 ] Yuanbo Liu edited comment on HDFS-11284 at 1/1/17 4:30 AM: --- [~um

[jira] [Comment Edited] (HDFS-11284) [SPS]: Avoid running SPS under safemode and fix issues in target node choosing.

2016-12-31 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15790521#comment-15790521 ] Yuanbo Liu edited comment on HDFS-11284 at 1/1/17 4:31 AM: --- [~um

[jira] [Commented] (HDFS-11284) [SPS]: Avoid running SPS under safemode and fix issues in target node choosing.

2016-12-31 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15790659#comment-15790659 ] Yuanbo Liu commented on HDFS-11284: --- [~umamaheswararao] Thanks for your quick response.

[jira] [Commented] (HDFS-11284) [SPS]: Avoid running SPS under safemode and fix issues in target node choosing.

2017-01-02 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15793937#comment-15793937 ] Yuanbo Liu commented on HDFS-11284: --- [~umamaheswararao] {quote} this JIRA can not be a

[jira] [Commented] (HDFS-11284) [SPS]: Avoid running SPS under safemode and fix issues in target node choosing.

2017-01-02 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15794427#comment-15794427 ] Yuanbo Liu commented on HDFS-11284: --- [~rakesh_r] Thanks for your response. I've attached

[jira] [Updated] (HDFS-11284) [SPS]: Avoid running SPS under safemode and fix issues in target node choosing.

2017-01-02 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-11284: -- Attachment: TestSatisfier.java > [SPS]: Avoid running SPS under safemode and fix issues in target node

[jira] [Commented] (HDFS-11284) [SPS]: Avoid running SPS under safemode and fix issues in target node choosing.

2017-01-03 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15796898#comment-15796898 ] Yuanbo Liu commented on HDFS-11284: --- [~umamaheswararao] Thanks for your response. {code}

[jira] [Created] (HDFS-11293) FsDatasetImpl throws ReplicaAlreadyExistsException in a wrong situation

2017-01-04 Thread Yuanbo Liu (JIRA)
Yuanbo Liu created HDFS-11293: - Summary: FsDatasetImpl throws ReplicaAlreadyExistsException in a wrong situation Key: HDFS-11293 URL: https://issues.apache.org/jira/browse/HDFS-11293 Project: Hadoop HDFS

[jira] [Commented] (HDFS-11293) FsDatasetImpl throws ReplicaAlreadyExistsException in a wrong situation

2017-01-04 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15800624#comment-15800624 ] Yuanbo Liu commented on HDFS-11293: --- [~umamaheswararao] / [~rakeshr] I tag you here beca

[jira] [Commented] (HDFS-11293) FsDatasetImpl throws ReplicaAlreadyExistsException in a wrong situation

2017-01-05 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15800939#comment-15800939 ] Yuanbo Liu commented on HDFS-11293: --- [~umamaheswararao] Thanks for your response. {code}

[jira] [Commented] (HDFS-11293) FsDatasetImpl throws ReplicaAlreadyExistsException in a wrong situation

2017-01-05 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803261#comment-15803261 ] Yuanbo Liu commented on HDFS-11293: --- [~umamaheswararao] Thanks for your response. I'll a

[jira] [Updated] (HDFS-11150) [SPS]: Provide persistence when satisfying storage policy.

2017-01-06 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-11150: -- Attachment: HDFS-11150-HDFS-10285.005.patch uploaded v5 patch for this JIRA. [~umamaheswararao] / [~rake

[jira] [Comment Edited] (HDFS-11150) [SPS]: Provide persistence when satisfying storage policy.

2017-01-06 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15804220#comment-15804220 ] Yuanbo Liu edited comment on HDFS-11150 at 1/6/17 10:20 AM: up

[jira] [Comment Edited] (HDFS-11150) [SPS]: Provide persistence when satisfying storage policy.

2017-01-06 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15804220#comment-15804220 ] Yuanbo Liu edited comment on HDFS-11150 at 1/6/17 10:21 AM: up

[jira] [Comment Edited] (HDFS-11150) [SPS]: Provide persistence when satisfying storage policy.

2017-01-06 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15804220#comment-15804220 ] Yuanbo Liu edited comment on HDFS-11150 at 1/6/17 10:22 AM: up

[jira] [Updated] (HDFS-11293) [SPS]: Local DN should be given prefernce as source node, when target available in same node

2017-01-08 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-11293: -- Assignee: Uma Maheswara Rao G (was: Yuanbo Liu) > [SPS]: Local DN should be given prefernce as source n

[jira] [Commented] (HDFS-11150) [SPS]: Provide persistence when satisfying storage policy.

2017-01-08 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15810475#comment-15810475 ] Yuanbo Liu commented on HDFS-11150: --- [~umamaheswararao] Sure, I will remove my code chan

[jira] [Updated] (HDFS-11293) [SPS]: Local DN should be given prefernce as source node, when target available in same node

2017-01-08 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-11293: -- Description: In {{FsDatasetImpl#createTemporary}}, we use {{volumeMap}} to get replica info by block po

[jira] [Commented] (HDFS-11293) [SPS]: Local DN should be given prefernce as source node, when target available in same node

2017-01-08 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15810584#comment-15810584 ] Yuanbo Liu commented on HDFS-11293: --- [~umamaheswararao] Thanks for your patch. Most part

[jira] [Comment Edited] (HDFS-11293) [SPS]: Local DN should be given prefernce as source node, when target available in same node

2017-01-08 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15810584#comment-15810584 ] Yuanbo Liu edited comment on HDFS-11293 at 1/9/17 3:51 AM: --- [~um

[jira] [Comment Edited] (HDFS-11293) [SPS]: Local DN should be given prefernce as source node, when target available in same node

2017-01-08 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15810584#comment-15810584 ] Yuanbo Liu edited comment on HDFS-11293 at 1/9/17 3:51 AM: --- [~um

[jira] [Updated] (HDFS-11284) [SPS]: Avoid running SPS under safemode and fix issues in target node choosing.

2017-01-08 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-11284: -- Description: When the real replication number of block doesn't match the replication factor. For exampl

[jira] [Updated] (HDFS-11284) [SPS]: fix issue of moving blocks with satisfier while changing replication factor

2017-01-08 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-11284: -- Summary: [SPS]: fix issue of moving blocks with satisfier while changing replication factor (was: [SPS

[jira] [Updated] (HDFS-11293) [SPS]: Local DN should be given preference as source node, when target available in same node

2017-01-09 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-11293: -- Summary: [SPS]: Local DN should be given preference as source node, when target available in same node

[jira] [Commented] (HDFS-11293) [SPS]: Local DN should be given preference as source node, when target available in same node

2017-01-09 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15811079#comment-15811079 ] Yuanbo Liu commented on HDFS-11293: --- [~umamaheswararao] Thanks for your new patch. Test

[jira] [Commented] (HDFS-11150) [SPS]: Provide persistence when satisfying storage policy.

2017-01-09 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15813791#comment-15813791 ] Yuanbo Liu commented on HDFS-11150: --- [~umamaheswararao] Sure. Thanks for your reminder.

[jira] [Updated] (HDFS-11150) [SPS]: Provide persistence when satisfying storage policy.

2017-01-09 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-11150: -- Attachment: HDFS-11150-HDFS-10285.006.patch Uploaded v6 patch: 1. Rebase my patch 2. Delete the code cha

[jira] [Commented] (HDFS-11284) [SPS]: fix issue of moving blocks with satisfier while changing replication factor

2017-01-10 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15814306#comment-15814306 ] Yuanbo Liu commented on HDFS-11284: --- [~umamaheswararao] Thanks for your response. {quote

[jira] [Updated] (HDFS-10999) Use more generic "low redundancy" blocks instead of "under replicated" blocks

2017-01-10 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10999: -- Assignee: (was: Yuanbo Liu) > Use more generic "low redundancy" blocks instead of "under replicated"

[jira] [Commented] (HDFS-10999) Use more generic "low redundancy" blocks instead of "under replicated" blocks

2017-01-10 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816787#comment-15816787 ] Yuanbo Liu commented on HDFS-10999: --- I think I've lost the context of this JIRA. Dismiss

[jira] [Commented] (HDFS-11150) [SPS]: Provide persistence when satisfying storage policy.

2017-01-10 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816821#comment-15816821 ] Yuanbo Liu commented on HDFS-11150: --- [~umamaheswararao] Thanks for your response. {code}

[jira] [Updated] (HDFS-11150) [SPS]: Provide persistence when satisfying storage policy.

2017-01-11 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-11150: -- Attachment: HDFS-11150-HDFS-10285.007.patch upload v7 patch. > [SPS]: Provide persistence when satisfyi

[jira] [Commented] (HDFS-11150) [SPS]: Provide persistence when satisfying storage policy.

2017-01-11 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819978#comment-15819978 ] Yuanbo Liu commented on HDFS-11150: --- Thanks, raise another JIRA to remove xAttr > [SPS]

[jira] [Created] (HDFS-11336) [SPS]: Clean up SPS xAttr in a proper situation

2017-01-11 Thread Yuanbo Liu (JIRA)
Yuanbo Liu created HDFS-11336: - Summary: [SPS]: Clean up SPS xAttr in a proper situation Key: HDFS-11336 URL: https://issues.apache.org/jira/browse/HDFS-11336 Project: Hadoop HDFS Issue Type: Sub

[jira] [Updated] (HDFS-11336) [SPS]: Remove xAttrs when movements done or SPS disabled

2017-02-13 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-11336: -- Attachment: HDFS-11336-HDFS-10285.001.patch upload v1 patch for this JIRA > [SPS]: Remove xAttrs when m

[jira] [Updated] (HDFS-11336) [SPS]: Remove xAttrs when movements done or SPS disabled

2017-02-13 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-11336: -- Status: Patch Available (was: Open) > [SPS]: Remove xAttrs when movements done or SPS disabled > --

[jira] [Updated] (HDFS-11336) [SPS]: Remove xAttrs when movements done or SPS disabled

2017-02-14 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-11336: -- Attachment: HDFS-11336-HDFS-10285.002.patch add v2 patch to address test failures and check-style failur

[jira] [Commented] (HDFS-10276) Different results for exist call for file.ext/name

2016-05-08 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275950#comment-15275950 ] Yuanbo Liu commented on HDFS-10276: --- [~arpitagarwal] Could you please help me review thi

[jira] [Commented] (HDFS-10276) HDFS throws AccessControlException when checking for the existence of /a/b when /a is a file

2016-05-16 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15284293#comment-15284293 ] Yuanbo Liu commented on HDFS-10276: --- [~cmccabe] Thanks for your comment. When I did some

[jira] [Updated] (HDFS-10276) HDFS throws AccessControlException when checking for the existence of /a/b when /a is a file

2016-05-16 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10276: -- Status: In Progress (was: Patch Available) > HDFS throws AccessControlException when checking for the e

[jira] [Updated] (HDFS-10276) HDFS throws AccessControlException when checking for the existence of /a/b when /a is a file

2016-05-16 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10276: -- Attachment: HDFS-10276.005.patch > HDFS throws AccessControlException when checking for the existence of

[jira] [Updated] (HDFS-10276) HDFS throws AccessControlException when checking for the existence of /a/b when /a is a file

2016-05-16 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10276: -- Status: Patch Available (was: In Progress) > HDFS throws AccessControlException when checking for the e

[jira] [Commented] (HDFS-10276) HDFS throws AccessControlException when checking for the existence of /a/b when /a is a file

2016-05-17 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15288119#comment-15288119 ] Yuanbo Liu commented on HDFS-10276: --- Discussed with [~yzhangal] who committed the patch

[jira] [Updated] (HDFS-10276) HDFS throws AccessControlException when checking for the existence of /a/b when /a is a file

2016-05-17 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10276: -- Status: In Progress (was: Patch Available) > HDFS throws AccessControlException when checking for the e

[jira] [Updated] (HDFS-10276) HDFS throws AccessControlException when checking for the existence of /a/b when /a is a file

2016-05-17 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10276: -- Attachment: HDFS-10276.006.patch > HDFS throws AccessControlException when checking for the existence of

[jira] [Updated] (HDFS-10276) HDFS throws AccessControlException when checking for the existence of /a/b when /a is a file

2016-05-17 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10276: -- Status: Patch Available (was: In Progress) > HDFS throws AccessControlException when checking for the e

[jira] [Commented] (HDFS-10276) HDFS throws AccessControlException when checking for the existence of /a/b when /a is a file

2016-05-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15288586#comment-15288586 ] Yuanbo Liu commented on HDFS-10276: --- the failure test cases are not related to the patch

[jira] [Commented] (HDFS-7343) A comprehensive and flexible storage policy engine

2016-06-26 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15350392#comment-15350392 ] Yuanbo Liu commented on HDFS-7343: -- [~drankye] Great proposal. Any updates here? We're int

[jira] [Commented] (HDFS-10574) webhdfs fails with filenames including semicolons

2016-06-27 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15350568#comment-15350568 ] Yuanbo Liu commented on HDFS-10574: --- [~bobhansen] I wrote a test case and test it on Had

[jira] [Commented] (HDFS-10575) webhdfs fails with filenames including semicolons

2016-06-27 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352226#comment-15352226 ] Yuanbo Liu commented on HDFS-10575: --- This is a duplicate jira of HDFS-10574 > webhdfs f

[jira] [Created] (HDFS-10593) MAX_DIR_ITEMS should not be hard coded since RPC buff size is configurable

2016-07-04 Thread Yuanbo Liu (JIRA)
Yuanbo Liu created HDFS-10593: - Summary: MAX_DIR_ITEMS should not be hard coded since RPC buff size is configurable Key: HDFS-10593 URL: https://issues.apache.org/jira/browse/HDFS-10593 Project: Hadoop H

[jira] [Comment Edited] (HDFS-10593) MAX_DIR_ITEMS should not be hard coded since RPC buff size is configurable

2016-07-04 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362065#comment-15362065 ] Yuanbo Liu edited comment on HDFS-10593 at 7/5/16 6:06 AM: --- [~an

[jira] [Commented] (HDFS-10593) MAX_DIR_ITEMS should not be hard coded since RPC buff size is configurable

2016-07-04 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362065#comment-15362065 ] Yuanbo Liu commented on HDFS-10593: --- [~andrew.wang][~cnauroth] I also tag you two in thi

[jira] [Commented] (HDFS-10593) MAX_DIR_ITEMS should not be hard coded since RPC buff size is configurable

2016-07-05 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15363718#comment-15363718 ] Yuanbo Liu commented on HDFS-10593: --- [~andrew.wang]Thank you for your response. You're r

[jira] [Resolved] (HDFS-10593) MAX_DIR_ITEMS should not be hard coded since RPC buff size is configurable

2016-07-05 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu resolved HDFS-10593. --- Resolution: Not A Problem > MAX_DIR_ITEMS should not be hard coded since RPC buff size is configurable

[jira] [Commented] (HDFS-10564) UNDER MIN REPL'D BLOCKS should be prioritized for replication

2016-07-07 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15367308#comment-15367308 ] Yuanbo Liu commented on HDFS-10564: --- [~eclark] Actually blocks which need to be replicat

[jira] [Created] (HDFS-10612) Optimize mechanism when block report size exceed the limit of PB message

2016-07-12 Thread Yuanbo Liu (JIRA)
Yuanbo Liu created HDFS-10612: - Summary: Optimize mechanism when block report size exceed the limit of PB message Key: HDFS-10612 URL: https://issues.apache.org/jira/browse/HDFS-10612 Project: Hadoop HDFS

[jira] [Commented] (HDFS-10612) Optimize mechanism when block report size exceed the limit of PB message

2016-07-12 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372410#comment-15372410 ] Yuanbo Liu commented on HDFS-10612: --- Here are my proposals: 1. Stop writing blocks when

[jira] [Assigned] (HDFS-10612) Optimize mechanism when block report size exceed the limit of PB message

2016-07-13 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu reassigned HDFS-10612: - Assignee: Yuanbo Liu > Optimize mechanism when block report size exceed the limit of PB message >

[jira] [Commented] (HDFS-10612) Optimize mechanism when block report size exceed the limit of PB message

2016-07-13 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376229#comment-15376229 ] Yuanbo Liu commented on HDFS-10612: --- Block report size is difficult to calculate, for th

[jira] [Created] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-17 Thread Yuanbo Liu (JIRA)
Yuanbo Liu created HDFS-10645: - Summary: Make block report size as a metric and add this metric to datanode web ui Key: HDFS-10645 URL: https://issues.apache.org/jira/browse/HDFS-10645 Project: Hadoop HDF

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-17 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Description: Add a new metric called "Max block report size". It's important for administrators to know

[jira] [Comment Edited] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-17 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15381802#comment-15381802 ] Yuanbo Liu edited comment on HDFS-10645 at 7/18/16 6:54 AM: If

[jira] [Commented] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-17 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15381802#comment-15381802 ] Yuanbo Liu commented on HDFS-10645: --- If the cluster grows big enough, it will hit this e

[jira] [Updated] (HDFS-10645) Make max block report size as a metric and add this metric to datanode web ui

2016-07-17 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Summary: Make max block report size as a metric and add this metric to datanode web ui (was: Make block

[jira] [Updated] (HDFS-10645) Make max block report size as a metric and add this metric to datanode web ui

2016-07-17 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Attachment: Selection_046.png > Make max block report size as a metric and add this metric to datanode w

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Summary: Make block report size as a metric and add this metric to datanode web ui (was: Make max block

[jira] [Comment Edited] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15381802#comment-15381802 ] Yuanbo Liu edited comment on HDFS-10645 at 7/18/16 8:52 AM: If

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Description: Record block report size as a metric and show it on datanode UI. It's important for adminis

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Attachment: (was: Selection_046.png) > Make block report size as a metric and add this metric to dat

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Attachment: Selection_047.png > Make block report size as a metric and add this metric to datanode web u

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Attachment: Selection_048.png > Make block report size as a metric and add this metric to datanode web u

[jira] [Commented] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382053#comment-15382053 ] Yuanbo Liu commented on HDFS-10645: --- [~cheersyang] Good point. I submitted a new snapsho

[jira] [Comment Edited] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382053#comment-15382053 ] Yuanbo Liu edited comment on HDFS-10645 at 7/18/16 10:33 AM: -

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Attachment: HDFS-10645.001.patch > Make block report size as a metric and add this metric to datanode we

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-18 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Status: Patch Available (was: Open) > Make block report size as a metric and add this metric to datanod

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-19 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Attachment: HDFS-10645.002.patch > Make block report size as a metric and add this metric to datanode we

[jira] [Commented] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-19 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383942#comment-15383942 ] Yuanbo Liu commented on HDFS-10645: --- [~ajisakaa] Thanks for your comments. I've uploaded

[jira] [Commented] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-19 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15385268#comment-15385268 ] Yuanbo Liu commented on HDFS-10645: --- [~ajisakaa] Thanks for your comments, they are very

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-19 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Attachment: HDFS-10645.003.patch > Make block report size as a metric and add this metric to datanode we

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-20 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Status: Patch Available (was: In Progress) > Make block report size as a metric and add this metric to

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-20 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Status: In Progress (was: Patch Available) > Make block report size as a metric and add this metric to

[jira] [Commented] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-20 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387019#comment-15387019 ] Yuanbo Liu commented on HDFS-10645: --- [~ajisakaa] Thanks for your good guidance. I will c

[jira] [Updated] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-20 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10645: -- Attachment: HDFS-10645.004.patch > Make block report size as a metric and add this metric to datanode we

[jira] [Commented] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-20 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387207#comment-15387207 ] Yuanbo Liu commented on HDFS-10645: --- uploaded v4 patch > Make block report size as a me

[jira] [Assigned] (HDFS-10667) Report more accurate info about data corruption location

2016-07-20 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu reassigned HDFS-10667: - Assignee: Yuanbo Liu > Report more accurate info about data corruption location > ---

[jira] [Commented] (HDFS-10645) Make block report size as a metric and add this metric to datanode web ui

2016-07-21 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15388836#comment-15388836 ] Yuanbo Liu commented on HDFS-10645: --- [~ajisakaa] Thanks for your comments and your patie

[jira] [Commented] (HDFS-10667) Report more accurate info about data corruption location

2016-07-22 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15389290#comment-15389290 ] Yuanbo Liu commented on HDFS-10667: --- [~yzhangal] I did background research about this ji

[jira] [Updated] (HDFS-10667) Report more accurate info about data corruption location

2016-07-22 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10667: -- Attachment: HDFS-10667.001.patch > Report more accurate info about data corruption location > --

[jira] [Work started] (HDFS-10667) Report more accurate info about data corruption location

2016-07-22 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-10667 started by Yuanbo Liu. - > Report more accurate info about data corruption location > --

[jira] [Updated] (HDFS-10667) Report more accurate info about data corruption location

2016-07-22 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10667: -- Status: Patch Available (was: In Progress) > Report more accurate info about data corruption location >

[jira] [Updated] (HDFS-10667) Report more accurate info about data corruption location

2016-07-24 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10667: -- Attachment: HDFS-10667.002.patch > Report more accurate info about data corruption location > --

[jira] [Commented] (HDFS-10667) Report more accurate info about data corruption location

2016-07-24 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15391267#comment-15391267 ] Yuanbo Liu commented on HDFS-10667: --- [~yzhangal] Thanks for your comments. I've uploaded

[jira] [Commented] (HDFS-10667) Report more accurate info about data corruption location

2016-07-25 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15393065#comment-15393065 ] Yuanbo Liu commented on HDFS-10667: --- [~yzhangal] Sure, I've uploaded v3 patch per your c

[jira] [Updated] (HDFS-10667) Report more accurate info about data corruption location

2016-07-25 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu updated HDFS-10667: -- Attachment: HDFS-10667.003.patch > Report more accurate info about data corruption location > --

[jira] [Assigned] (HDFS-10664) layoutVersion mismatch between Namenode VERSION file and Journalnode VERSION file after cluster upgrade

2016-07-26 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu reassigned HDFS-10664: - Assignee: Yuanbo Liu > layoutVersion mismatch between Namenode VERSION file and Journalnode VERSI

[jira] [Assigned] (HDFS-10693) metaSave should print blocks, not LightWeightHashSet

2016-07-26 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu reassigned HDFS-10693: - Assignee: Yuanbo Liu > metaSave should print blocks, not LightWeightHashSet > ---

[jira] [Assigned] (HDFS-10694) BlockManager.processReport() should print blockReportId in each log message.

2016-07-26 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanbo Liu reassigned HDFS-10694: - Assignee: Yuanbo Liu > BlockManager.processReport() should print blockReportId in each log messag

[jira] [Commented] (HDFS-10667) Report more accurate info about data corruption location

2016-07-27 Thread Yuanbo Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15395232#comment-15395232 ] Yuanbo Liu commented on HDFS-10667: --- [~yzhangal] Thanks for your comments {quote} which

<    1   2   3   4   5   6   7   >