[
https://issues.apache.org/jira/browse/HDFS-6346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993657#comment-13993657
]
Uma Maheswara Rao G commented on HDFS-6346:
-------------------------------------------
Please modify log message something like:
"Xattr " + xattr + " does not exit on the path " + path
Other than this patch looks great to me. +1 on addressing this comment.
I have verified by following operations:
150 setXattr and 150 removeXattr operations with the key as user.a1<number
from 1-150> and value as {0x31, 0x32, 0x33}
Resulted editlog file size with out patch is 343KB and After applying the
patch, the editlog size reduced to 11KB.
Thanks a lot, for taking this and improving.
> Optimize OP_SET_XATTRS by persisting single Xattr entry per
> setXattr/removeXattr api call
> -----------------------------------------------------------------------------------------
>
> Key: HDFS-6346
> URL: https://issues.apache.org/jira/browse/HDFS-6346
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Reporter: Uma Maheswara Rao G
> Assignee: Yi Liu
> Attachments: HDFS-6346.1.patch, HDFS-6346.2.patch, HDFS-6346.patch,
> editsStored
>
>
> When a new xattrs set on an Inode, it may add this with OP_SET_XATTRS and
> let's say [user.name1:value1]
> On a next call if we set another xattrs, then it may store along with older
> existing xattrs again. It may be like [user.name1:value1, user.name2:value2]
> So, on adding more xattrs on same Inode, that list may grow and we keep store
> the entries of already existing name, value fairs.
> Right now we defaulted the max Xattrs on an Inode to 32 and configured. If
> user modified it to much larger value and start setting xattrs, then edits
> loading may create issue like my above example.
> But I didn't refer any usecase of having large number of xattrs, this is just
> the assumption to consider a case. My biggest doubt is whether we will have
> such real usecases to have huge xattrs on a single INode.
> So, here is a thought on having OP_SET_XATTR for each setXAttr operation to
> be logged, When we replay them we need to consolidate. This is some initial
> thought we can think more if others also feel we need to consider this case
> to handle.
> Otherwise we endup storing Xattrs entries in editlog file as n(n+1)/2 where n
> is number xattrs for a file/dir. This may be issue only when we have large
> number configured max xattrs for inode.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
