[jira] [Commented] (HDFS-8010) Erasure coding: extend UnderReplicatedBlocks to accurately handle striped blocks
[
https://issues.apache.org/jira/browse/HDFS-8010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14512893#comment-14512893
]
Zhe Zhang commented on HDFS-8010:
-
Thanks for the reviews [~jingzhao] and [~rakeshr]! And sorry for getting to
this late.
I just submitted a path for the rename under HDFS-8255. Will update the patch
for the {{getPriority}} part soon.
Erasure coding: extend UnderReplicatedBlocks to accurately handle striped
blocks
Key: HDFS-8010
URL: https://issues.apache.org/jira/browse/HDFS-8010
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Zhe Zhang
Assignee: Zhe Zhang
Attachments: HDFS-8010-000.patch
This JIRA tracks efforts to accurately assess the _risk level_ of a striped
block groups with missing blocks, when added to {{UnderReplicatedBlocks}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8010) Erasure coding: extend UnderReplicatedBlocks to accurately handle striped blocks
[
https://issues.apache.org/jira/browse/HDFS-8010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14493261#comment-14493261
]
Jing Zhao commented on HDFS-8010:
-
Thanks for working on this, Zhe! The patch looks good to me. Some minors:
# Please see if you want to do the rename work in trunk. But keeping the change
here is also fine to me.
# The patch needs rebase. Besides, {{getPriority}} now needs to use ECSchema
information instead of hard-coded numbers.
# After using ECSchema information for priority computation, we may need to add
several more unit tests (to cover cases with different schemas).
# As suggested by [~rakeshr], we can move the conditions into two separate
functions.
Erasure coding: extend UnderReplicatedBlocks to accurately handle striped
blocks
Key: HDFS-8010
URL: https://issues.apache.org/jira/browse/HDFS-8010
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Zhe Zhang
Assignee: Zhe Zhang
Attachments: HDFS-8010-000.patch
This JIRA tracks efforts to accurately assess the _risk level_ of a striped
block groups with missing blocks, when added to {{UnderReplicatedBlocks}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8010) Erasure coding: extend UnderReplicatedBlocks to accurately handle striped blocks
[
https://issues.apache.org/jira/browse/HDFS-8010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14388555#comment-14388555
]
Rakesh R commented on HDFS-8010:
Minor mistake in my above comments, please read the suggested way as :
{code}
private boolean veryUnderReplicated(int curReplicas, int expectedReplicas,
boolean isStriped) {
if (!isStriped) {
return (curReplicas * 3) expectedReplicas;
} else {
return curReplicas = HdfsConstants.NUM_DATA_BLOCKS + 2;
}
}
private boolean highestPrioirty(int curReplicas, boolean isStriped) {
if(!isStriped){
return curReplicas == 1;
} else {
return curReplicas == HdfsConstants.NUM_DATA_BLOCKS;
}
}
{code}
Erasure coding: extend UnderReplicatedBlocks to accurately handle striped
blocks
Key: HDFS-8010
URL: https://issues.apache.org/jira/browse/HDFS-8010
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Zhe Zhang
Assignee: Zhe Zhang
Attachments: HDFS-8010-000.patch
This JIRA tracks efforts to accurately assess the _risk level_ of a striped
block groups with missing blocks, when added to {{UnderReplicatedBlocks}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8010) Erasure coding: extend UnderReplicatedBlocks to accurately handle striped blocks
[
https://issues.apache.org/jira/browse/HDFS-8010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14388552#comment-14388552
]
Rakesh R commented on HDFS-8010:
Thanks [~zhz]. Overall the patch looks good. Just few comments,
# Can we rename {{requiredReplication}} and {{expectedReplication}} variables
as well. This should be helpful to reflect the meaning, right?
# Can we think of simplifying the below conditions to make it more readable.
{code}
+} else if ((!isStriped curReplicas == 1) ||
+(isStriped curReplicas == HdfsConstants.NUM_DATA_BLOCKS)) {
+} else if ((!isStriped (curReplicas * 3) expectedReplicas) ||
+isStriped curReplicas = HdfsConstants.NUM_DATA_BLOCKS + 2) {
{code}
One suggestion that comes in my mind is moving to another private methods like
below,
{code}
private boolean veryUnderReplicated(int curReplicas, int expectedReplicas,
boolean isStriped) {
if (!isStriped) {
return curReplicas = HdfsConstants.NUM_DATA_BLOCKS + 2;
} else {
return (curReplicas * 3) expectedReplicas;
}
}
private boolean highestPrioirty(int curReplicas, boolean isStriped) {
if(isStriped){
return curReplicas == 1;
} else {
return curReplicas == HdfsConstants.NUM_DATA_BLOCKS;
}
}
{code}
Erasure coding: extend UnderReplicatedBlocks to accurately handle striped
blocks
Key: HDFS-8010
URL: https://issues.apache.org/jira/browse/HDFS-8010
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Zhe Zhang
Assignee: Zhe Zhang
Attachments: HDFS-8010-000.patch
This JIRA tracks efforts to accurately assess the _risk level_ of a striped
block groups with missing blocks, when added to {{UnderReplicatedBlocks}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8010) Erasure coding: extend UnderReplicatedBlocks to accurately handle striped blocks
[
https://issues.apache.org/jira/browse/HDFS-8010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14385161#comment-14385161
]
Zhe Zhang commented on HDFS-8010:
-
My current thoughts are:
# Move the logic of checking expected, actual, and decomm replicas into
{{UnderReplicatedBlocks#add}}. This way it becomes a single place to handle
both contiguous and striped blocks.
# Better placement of striped blocks into priority levels (maybe more levels
are needed)
[~jingzhao] Not sure if you have already started working on this. If so please
feel free to assign to yourself.
Erasure coding: extend UnderReplicatedBlocks to accurately handle striped
blocks
Key: HDFS-8010
URL: https://issues.apache.org/jira/browse/HDFS-8010
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Zhe Zhang
This JIRA tracks efforts to accurately assess the _risk level_ of a striped
block groups with missing blocks, when added to {{UnderReplicatedBlocks}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
