[ https://issues.apache.org/jira/browse/HDFS-6346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HDFS-6346 started by Yi Liu. > Optimize OP_SET_XATTRS by persisting single Xattr entry per > setXattr/removeXattr api call > ----------------------------------------------------------------------------------------- > > Key: HDFS-6346 > URL: https://issues.apache.org/jira/browse/HDFS-6346 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: namenode > Reporter: Uma Maheswara Rao G > Assignee: Yi Liu > Attachments: HDFS-6346.patch, editsStored > > > When a new xattrs set on an Inode, it may add this with OP_SET_XATTRS and > let's say [user.name1:value1] > On a next call if we set another xattrs, then it may store along with older > existing xattrs again. It may be like [user.name1:value1, user.name2:value2] > So, on adding more xattrs on same Inode, that list may grow and we keep store > the entries of already existing name, value fairs. > Right now we defaulted the max Xattrs on an Inode to 32 and configured. If > user modified it to much larger value and start setting xattrs, then edits > loading may create issue like my above example. > But I didn't refer any usecase of having large number of xattrs, this is just > the assumption to consider a case. My biggest doubt is whether we will have > such real usecases to have huge xattrs on a single INode. > So, here is a thought on having OP_SET_XATTR for each setXAttr operation to > be logged, When we replay them we need to consolidate. This is some initial > thought we can think more if others also feel we need to consider this case > to handle. > Otherwise we endup storing Xattrs entries in editlog file as n(n+1)/2 where n > is number xattrs for a file/dir. This may be issue only when we have large > number configured max xattrs for inode. -- This message was sent by Atlassian JIRA (v6.2#6252)