[
https://issues.apache.org/jira/browse/HDFS-12575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16201668#comment-16201668
]
SammiChen commented on HDFS-12575:
----------------------------------
Hi [~eddyxu], sure, I will work on it. I'm not clear about the detail steps to
carry out the "Replay edits after checkpoint" and "Apply edits on SNN". Can you
help to give some hit? Also by SNN, you mean both secondary namenode and
standby namenode, right?
> Improve test coverage for EC related edit logs ops
> --------------------------------------------------
>
> Key: HDFS-12575
> URL: https://issues.apache.org/jira/browse/HDFS-12575
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: erasure-coding
> Affects Versions: 3.0.0-alpha4
> Reporter: Lei (Eddy) Xu
> Assignee: SammiChen
> Labels: hdfs-ec-3.0-nice-to-have
>
> HDFS-12569 found that we have little test coverage for edit logs ops of
> erasure coding.
> And we've seen the following bug bring down SNN in our test environments:
> {code}
> 6:42:18.177 AM ERROR FSEditLogLoader
> Encountered exception on operation AddBlockOp [path=/tmp/foo/bar,
> penultimateBlock=NULL, lastBlock=blk_1073743386_69322, RpcClientId=,
> RpcCallId=-2]
> java.lang.IllegalArgumentException: reportedBlock is not striped
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
> at
> ....
> 6:42:18.190 AM FATAL EditLogTailer
> Unknown error encountered while tailing edits. Shutting down standby NN.
> java.io.IOException: java.lang.IllegalArgumentException: reportedBlock is not
> striped
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:251)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:150)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:882)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:863)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:293)
> at
> {code}
> We should add coverage for these important edit logs, i.e., set/unset policy,
> enable/remove policies and etc are correctly persisted in edit logs, and test
> the scenarios like:
> * Restart NN
> * Replay edits after checkpoint
> * Apply edits on SNN.
> * and etc.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
