[
https://issues.apache.org/jira/browse/HDFS-16645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17602102#comment-17602102
]
ASF GitHub Bot commented on HDFS-16645:
---------------------------------------
ZanderXu commented on PR #4518:
URL: https://github.com/apache/hadoop/pull/4518#issuecomment-1241475601
@xkrogen Thanks for your comment.
do you want to remove the redundant in-progress segment? This diff is only
fix the exception "Invalid log manifest".
If we plan to remove the redundant in-progress segment, we can use
JournalNodeSyncer asynchronously to fix it, such as copying the finalized
segment from other JNs and replace it.
@jojochuang Master, I'm looking forward your ideas, so that we can push this
bug forward.
> Multi inProgress segments caused "Invalid log manifest"
> -------------------------------------------------------
>
> Key: HDFS-16645
> URL: https://issues.apache.org/jira/browse/HDFS-16645
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: ZanderXu
> Assignee: ZanderXu
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> JournalNode will have a residual in-progress segment if it is shut down
> abnormally. After this JournalNode restarted and Active NameNode try to open
> a new in-progress segment, this journalnode will contains two in-progress
> segment, one is the latest segment and another is the residual segment.
> At this moment, NameNode gets one IllegalStateException when trying to
> getEditLogManifest from this JournalNode, and the exception as bellow:
> {code:java}
> java.lang.IllegalStateException: Invalid log manifest (log [1-?
> (in-progress)] overlaps [6-? (in-progress)])[[6-? (in-progress)], [1-?
> (in-progress)]] CommittedTxId: 0
> at
> org.apache.hadoop.hdfs.server.protocol.RemoteEditLogManifest.checkState(RemoteEditLogManifest.java:62)
> at
> org.apache.hadoop.hdfs.server.protocol.RemoteEditLogManifest.<init>(RemoteEditLogManifest.java:46)
> at
> org.apache.hadoop.hdfs.qjournal.server.Journal.getEditLogManifest(Journal.java:740)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
