[
https://issues.apache.org/jira/browse/HDFS-946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hairong Kuang updated HDFS-946:
-------------------------------
Attachment: HdfsFileStatus3.patch
This patch incorporates all review comments except for "2. consider move
isDirectory outside". In addition, it
1. removes the method getPath(), which returns the byte array, from
HdfsFileStatus. This makes the byte array immutable, which makes the change
safer.
2. fixes a few more bugs in Jsp code which misuses getPath().
3. adds comments to INode#name that reminds that if this encoding is changed,
the ClientProtocol is changed and the decoding of HdfsFileStatus#name should
change too.
> NameNode should not return full path name when lisitng a diretory or getting
> the status of a file
> -------------------------------------------------------------------------------------------------
>
> Key: HDFS-946
> URL: https://issues.apache.org/jira/browse/HDFS-946
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: HDFSFileStatus.patch, HDFSFileStatus1.patch,
> HdfsFileStatus3.patch
>
>
> FSDirectory#getListring(String src) has the following code:
> int i = 0;
> for (INode cur : contents) {
> listing[i] = createFileStatus(srcs+cur.getLocalName(), cur);
> i++;
> }
> So listing a directory will return an array of FileStatus. Each FileStatus
> element has the full path name. This increases the return message size and
> adds non-negligible CPU time to the operation.
> FSDirectory#getFileInfo(String) does not need to return the file name either.
> Another optimization is that in the version of FileStatus that's used in the
> wire protocol, the field path does not need to be Path; It could be a String
> or a byte array ideally. This could avoid unnecessary creation of the Path
> objects at NameNode, thus help reduce the GC problem observed when a large
> number of getFileInfo or getListing operations hit NameNode.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
