[
https://issues.apache.org/jira/browse/HDFS-17323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17804541#comment-17804541
]
Srinivasu Majeti commented on HDFS-17323:
-----------------------------------------
Hi [~arp] / [~weichu] , Could you please take a look at this Jira?
> Uncontrolled fsimage size due to snapshot diff meta for file deletions
> ----------------------------------------------------------------------
>
> Key: HDFS-17323
> URL: https://issues.apache.org/jira/browse/HDFS-17323
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
> Affects Versions: 3.1.1
> Reporter: Srinivasu Majeti
> Priority: Major
>
> We have seen quite a good number of customer cases w.r.t fsimage size
> increased drastically while storing snapshot meta for fileDiff entries. Here
> is an example fsimage meta storing entire inode info after deleting a file.
> I'm not sure about any restrictions on why the entire inode meta needs to be
> stored in fileDiff entry when there is no change w.r.t actual inode meta and
> it's just a delete file operation.
> The fileDiffEntry for the inode 1860467 seems redundant for a simple file
> delete operation.
> {code:java}
> <snapshot><id>43</id><root><id>1860465</id><type>DIRECTORY</type><name>s3</name><mtime>1704197935903</mtime><permission>hdfs:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></root></snapshot>
> <snapshot><id>44</id><root><id>1860465</id><type>DIRECTORY</type><name>s4</name><mtime>1704197951829</mtime><permission>hdfs:supergroup:0755</permission><nsquota>-1</nsquota><dsquota>-1</dsquota></root></snapshot>
> <inode><id>1860467</id><type>FILE</type><name>file1</name><replication>3</replication><mtime>1704197917315</mtime><atime>1704197917031</atime><preferredBlockSize>134217728</preferredBlockSize><permission>hdfs:supergroup:0644</permission><blocks><block><id>1074008442</id><genstamp>267653</genstamp><numBytes>418</numBytes></block>
> <ref><referredId>1860467</referredId><name>file1</name><dstSnapshotId>0</dstSnapshotId><lastSnapshotId>43</lastSnapshotId></ref>
> <ref><referredId>1860467</referredId><name></name><dstSnapshotId>2147483646</dstSnapshotId><lastSnapshotId>0</lastSnapshotId></ref>
> </dirDiffEntry><fileDiffEntry><inodeId>1860467</inodeId><count>1</count><fileDiff><snapshotId>43</snapshotId><size>418</size><name>file1</name><snapshotCopy><replication>3</replication><mtime>1704197917315</mtime><atime>1704197917031</atime><preferredBlockSize>134217728</preferredBlockSize><permission>hdfs:supergroup:0644</permission><storagePolicyId>0</storagePolicyId></snapshotCopy>
> </fileDiff>
> </fileDiffEntry>{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
