[jira] [Updated] (HDFS-8900) Compact XAttrs to optimize memory footprint.
[ https://issues.apache.org/jira/browse/HDFS-8900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8900: - Resolution: Fixed Hadoop Flags: Incompatible change,Reviewed (was: Incompatible change) Fix Version/s: 2.8.0 Status: Resolved (was: Patch Available) Compact XAttrs to optimize memory footprint. Key: HDFS-8900 URL: https://issues.apache.org/jira/browse/HDFS-8900 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Reporter: Yi Liu Assignee: Yi Liu Fix For: 2.8.0 Attachments: HDFS-8900.001.patch, HDFS-8900.002.patch, HDFS-8900.003.patch, HDFS-8900.004.patch, HDFS-8900.005.patch {code} private final ImmutableListXAttr xAttrs; {code} Currently we use above in XAttrFeature, it's not efficient from memory point of view, since {{ImmutableList}} and {{XAttr}} have object memory overhead, and each object has memory alignment. We can use a {{byte[]}} in XAttrFeature and do some compact in {{XAttr}}. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8900) Compact XAttrs to optimize memory footprint.
[ https://issues.apache.org/jira/browse/HDFS-8900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8900: - Attachment: HDFS-8900.004.patch Compact XAttrs to optimize memory footprint. Key: HDFS-8900 URL: https://issues.apache.org/jira/browse/HDFS-8900 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Reporter: Yi Liu Assignee: Yi Liu Attachments: HDFS-8900.001.patch, HDFS-8900.002.patch, HDFS-8900.003.patch, HDFS-8900.004.patch {code} private final ImmutableListXAttr xAttrs; {code} Currently we use above in XAttrFeature, it's not efficient from memory point of view, since {{ImmutableList}} and {{XAttr}} have object memory overhead, and each object has memory alignment. We can use a {{byte[]}} in XAttrFeature and do some compact in {{XAttr}}. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8900) Compact XAttrs to optimize memory footprint.
[ https://issues.apache.org/jira/browse/HDFS-8900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8900: - Attachment: HDFS-8900.005.patch Update patch to fix the checkstyle and add more comment in XAttrFeature. Compact XAttrs to optimize memory footprint. Key: HDFS-8900 URL: https://issues.apache.org/jira/browse/HDFS-8900 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Reporter: Yi Liu Assignee: Yi Liu Attachments: HDFS-8900.001.patch, HDFS-8900.002.patch, HDFS-8900.003.patch, HDFS-8900.004.patch, HDFS-8900.005.patch {code} private final ImmutableListXAttr xAttrs; {code} Currently we use above in XAttrFeature, it's not efficient from memory point of view, since {{ImmutableList}} and {{XAttr}} have object memory overhead, and each object has memory alignment. We can use a {{byte[]}} in XAttrFeature and do some compact in {{XAttr}}. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8900) Compact XAttrs to optimize memory footprint.
[ https://issues.apache.org/jira/browse/HDFS-8900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8900: - Attachment: HDFS-8900.003.patch Compact XAttrs to optimize memory footprint. Key: HDFS-8900 URL: https://issues.apache.org/jira/browse/HDFS-8900 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Reporter: Yi Liu Assignee: Yi Liu Attachments: HDFS-8900.001.patch, HDFS-8900.002.patch, HDFS-8900.003.patch {code} private final ImmutableListXAttr xAttrs; {code} Currently we use above in XAttrFeature, it's not efficient from memory point of view, since {{ImmutableList}} and {{XAttr}} have object memory overhead, and each object has memory alignment. We can use a {{byte[]}} in XAttrFeature and do some compact in {{XAttr}}. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8900) Compact XAttrs to optimize memory footprint.
[ https://issues.apache.org/jira/browse/HDFS-8900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8900: - Summary: Compact XAttrs to optimize memory footprint. (was: Optimize XAttr memory footprint.) Compact XAttrs to optimize memory footprint. Key: HDFS-8900 URL: https://issues.apache.org/jira/browse/HDFS-8900 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Reporter: Yi Liu Assignee: Yi Liu Attachments: HDFS-8900.001.patch, HDFS-8900.002.patch {code} private final ImmutableListXAttr xAttrs; {code} Currently we use above in XAttrFeature, it's not efficient from memory point of view, since {{ImmutableList}} and {{XAttr}} have object memory overhead, and each object has memory alignment. We can use a {{byte[]}} in XAttrFeature and do some compact in {{XAttr}}. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HDFS-8900) Compact XAttrs to optimize memory footprint.
[ https://issues.apache.org/jira/browse/HDFS-8900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8900: -- Hadoop Flags: Incompatible change Release Note: The config key dfs.namenode.fs-limits.max-xattr-size can no longer be set to a value of 0 (previously used to indicate unlimited) or a value greater than 32KB. This is a constraint on xattr size similar to many local filesystems. Compact XAttrs to optimize memory footprint. Key: HDFS-8900 URL: https://issues.apache.org/jira/browse/HDFS-8900 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Reporter: Yi Liu Assignee: Yi Liu Attachments: HDFS-8900.001.patch, HDFS-8900.002.patch {code} private final ImmutableListXAttr xAttrs; {code} Currently we use above in XAttrFeature, it's not efficient from memory point of view, since {{ImmutableList}} and {{XAttr}} have object memory overhead, and each object has memory alignment. We can use a {{byte[]}} in XAttrFeature and do some compact in {{XAttr}}. -- This message was sent by Atlassian JIRA (v6.3.4#6332)