JiangHua Zhu created HDFS-17894:
-----------------------------------
Summary: A new tool has been added to synchronize the latest
FsImage
Key: HDFS-17894
URL: https://issues.apache.org/jira/browse/HDFS-17894
Project: Hadoop HDFS
Issue Type: Improvement
Components: namenode
Affects Versions: 3.4.1
Reporter: JiangHua Zhu
The observer namenode stores the following fsimage files:
{code:java}
-rw-rw-r-- 1 hadoop hadoop 12075926817 3月 11 18:20 fsimage_0000000000810972885
-rw-rw-r-- 1 hadoop hadoop 62 3月 11 18:20
fsimage_0000000000810972885.md5
{code}
The standby namenode stores the latest fsimage file:
{code:java}
-rw-rw-r-- 1 hadoop hadoop 12142544499 Mar 20 04:42 fsimage_0000000000816122358
-rw-rw-r-- 1 hadoop hadoop 62 Mar 20 04:42
fsimage_0000000000816122358.md5
{code}
When the observer namenode has been running for a long time and needs to be
restarted, it will prompt that the edit log is missing. The log is as follows:
{code:java}
2026-03-20 09:56:44,259 WARN
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Encountered exception
loading fsimage
java.io.IOException: There appears to be a gap in the edit log. We expected
txid 810972886, but got txid 814100454.
at
org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:95)
at
org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:269)
at
org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:183)
at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:917)
at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:762)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:339)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1275)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:834)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:817)
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:904)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:1158)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:1133)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1906)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1971)
{code}
We can add a tool to synchronize the latest fsimage from the standby namenode.
This tool is mainly used by the administrator.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
