[
https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12921193#action_12921193
]
dhruba borthakur commented on HDFS-1435:
----------------------------------------
I looked at the code, looks good. Have two comments:
1. I would prefer to have the compression start after the header record. For
example, the imgVersion, numFiles, genStamp, defaultReplication, etc not be
compressed. This allows easier debugging, ability to dump the header of a file
to find out its contents (via od -x), etc.
2. The new image version is 25. but the code refers to it as -19
{code}
if (imgVersion <= -19) { // -19: 1st version providing compression option
isCompressed = in.readBoolean();
if (isCompressed) {
String codecClassName = Text.readString(in);
{code}
> Provide an option to store fsimage compressed
> ---------------------------------------------
>
> Key: HDFS-1435
> URL: https://issues.apache.org/jira/browse/HDFS-1435
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Affects Versions: 0.22.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: checkpoint-limitandcompress.patch,
> trunkImageCompress.patch, trunkImageCompress1.patch
>
>
> Our HDFS has fsimage as big as 20G bytes. It consumes a lot of network
> bandwidth when secondary NN uploads a new fsimage to primary NN.
> If we could store fsimage compressed, the problem could be greatly alleviated.
> I plan to provide a new configuration hdfs.image.compressed with a default
> value of false. If it is set to be true, fsimage is stored as compressed.
> The fsimage will have a new layout with a new field "compressed" in its
> header, indicating if the namespace is stored compressed or not.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
