[jira] [Updated] (HDFS-5651) Remove dfs.namenode.caching.enabled and improve CRM locking
[ https://issues.apache.org/jira/browse/HDFS-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-5651: --- Resolution: Fixed Status: Resolved (was: Patch Available) this was committed to trunk Remove dfs.namenode.caching.enabled and improve CRM locking --- Key: HDFS-5651 URL: https://issues.apache.org/jira/browse/HDFS-5651 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 3.0.0 Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Fix For: 3.0.0 Attachments: HDFS-5651.001.patch, HDFS-5651.002.patch, HDFS-5651.003.patch, HDFS-5651.004.patch, HDFS-5651.006.patch, HDFS-5651.006.patch, HDFS-5651.008.patch, HDFS-5651.009.patch We can remove dfs.namenode.caching.enabled and simply always enable caching, similar to how we do with snapshots and other features. The main overhead is the size of the cachedBlocks GSet. However, we can simply make the size of this GSet configurable, and people who don't want caching can set it to a very small value. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5651) remove dfs.namenode.caching.enabled and fix HA state transition race
[
https://issues.apache.org/jira/browse/HDFS-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-5651:
---
Attachment: HDFS-5651.009.patch
There was one place in CacheManager where I was calling {{CRM#setNeedsRescan}}
directly, rather than calling {{CacheManager#setNeedsRescan}}. That was
leading to an assert, since the latter needs to be done under the CRM lock. I
fixed this and it should be good to go now
remove dfs.namenode.caching.enabled and fix HA state transition race
Key: HDFS-5651
URL: https://issues.apache.org/jira/browse/HDFS-5651
Project: Hadoop HDFS
Issue Type: Sub-task
Components: namenode
Affects Versions: 3.0.0
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Attachments: HDFS-5651.001.patch, HDFS-5651.002.patch,
HDFS-5651.003.patch, HDFS-5651.004.patch, HDFS-5651.006.patch,
HDFS-5651.006.patch, HDFS-5651.008.patch, HDFS-5651.009.patch
We can remove dfs.namenode.caching.enabled and simply always enable caching,
similar to how we do with snapshots and other features. The main overhead is
the size of the cachedBlocks GSet. However, we can simply make the size of
this GSet configurable, and people who don't want caching can set it to a
very small value.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HDFS-5651) Remove dfs.namenode.caching.enabled and improve CRM locking
[ https://issues.apache.org/jira/browse/HDFS-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-5651: -- Summary: Remove dfs.namenode.caching.enabled and improve CRM locking (was: remove dfs.namenode.caching.enabled and fix HA state transition race) Remove dfs.namenode.caching.enabled and improve CRM locking --- Key: HDFS-5651 URL: https://issues.apache.org/jira/browse/HDFS-5651 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 3.0.0 Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Attachments: HDFS-5651.001.patch, HDFS-5651.002.patch, HDFS-5651.003.patch, HDFS-5651.004.patch, HDFS-5651.006.patch, HDFS-5651.006.patch, HDFS-5651.008.patch, HDFS-5651.009.patch We can remove dfs.namenode.caching.enabled and simply always enable caching, similar to how we do with snapshots and other features. The main overhead is the size of the cachedBlocks GSet. However, we can simply make the size of this GSet configurable, and people who don't want caching can set it to a very small value. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5651) Remove dfs.namenode.caching.enabled and improve CRM locking
[ https://issues.apache.org/jira/browse/HDFS-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-5651: -- Fix Version/s: 3.0.0 Remove dfs.namenode.caching.enabled and improve CRM locking --- Key: HDFS-5651 URL: https://issues.apache.org/jira/browse/HDFS-5651 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 3.0.0 Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Fix For: 3.0.0 Attachments: HDFS-5651.001.patch, HDFS-5651.002.patch, HDFS-5651.003.patch, HDFS-5651.004.patch, HDFS-5651.006.patch, HDFS-5651.006.patch, HDFS-5651.008.patch, HDFS-5651.009.patch We can remove dfs.namenode.caching.enabled and simply always enable caching, similar to how we do with snapshots and other features. The main overhead is the size of the cachedBlocks GSet. However, we can simply make the size of this GSet configurable, and people who don't want caching can set it to a very small value. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5651) remove dfs.namenode.caching.enabled and fix HA state transition race
[ https://issues.apache.org/jira/browse/HDFS-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-5651: --- Attachment: HDFS-5651.008.patch remove dfs.namenode.caching.enabled and fix HA state transition race Key: HDFS-5651 URL: https://issues.apache.org/jira/browse/HDFS-5651 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 3.0.0 Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Attachments: HDFS-5651.001.patch, HDFS-5651.002.patch, HDFS-5651.003.patch, HDFS-5651.004.patch, HDFS-5651.006.patch, HDFS-5651.006.patch, HDFS-5651.008.patch We can remove dfs.namenode.caching.enabled and simply always enable caching, similar to how we do with snapshots and other features. The main overhead is the size of the cachedBlocks GSet. However, we can simply make the size of this GSet configurable, and people who don't want caching can set it to a very small value. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5651) remove dfs.namenode.caching.enabled and fix HA state transition race
[ https://issues.apache.org/jira/browse/HDFS-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-5651: --- Summary: remove dfs.namenode.caching.enabled and fix HA state transition race (was: remove dfs.namenode.caching.enabled) remove dfs.namenode.caching.enabled and fix HA state transition race Key: HDFS-5651 URL: https://issues.apache.org/jira/browse/HDFS-5651 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 3.0.0 Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Attachments: HDFS-5651.001.patch, HDFS-5651.002.patch, HDFS-5651.003.patch, HDFS-5651.004.patch, HDFS-5651.006.patch We can remove dfs.namenode.caching.enabled and simply always enable caching, similar to how we do with snapshots and other features. The main overhead is the size of the cachedBlocks GSet. However, we can simply make the size of this GSet configurable, and people who don't want caching can set it to a very small value. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HDFS-5651) remove dfs.namenode.caching.enabled and fix HA state transition race
[ https://issues.apache.org/jira/browse/HDFS-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-5651: --- Attachment: HDFS-5651.006.patch remove dfs.namenode.caching.enabled and fix HA state transition race Key: HDFS-5651 URL: https://issues.apache.org/jira/browse/HDFS-5651 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 3.0.0 Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Attachments: HDFS-5651.001.patch, HDFS-5651.002.patch, HDFS-5651.003.patch, HDFS-5651.004.patch, HDFS-5651.006.patch, HDFS-5651.006.patch We can remove dfs.namenode.caching.enabled and simply always enable caching, similar to how we do with snapshots and other features. The main overhead is the size of the cachedBlocks GSet. However, we can simply make the size of this GSet configurable, and people who don't want caching can set it to a very small value. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HDFS-5651) remove dfs.namenode.caching.enabled
[ https://issues.apache.org/jira/browse/HDFS-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-5651: --- Attachment: HDFS-5651.006.patch remove dfs.namenode.caching.enabled --- Key: HDFS-5651 URL: https://issues.apache.org/jira/browse/HDFS-5651 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 3.0.0 Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Attachments: HDFS-5651.001.patch, HDFS-5651.002.patch, HDFS-5651.003.patch, HDFS-5651.004.patch, HDFS-5651.006.patch We can remove dfs.namenode.caching.enabled and simply always enable caching, similar to how we do with snapshots and other features. The main overhead is the size of the cachedBlocks GSet. However, we can simply make the size of this GSet configurable, and people who don't want caching can set it to a very small value. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HDFS-5651) remove dfs.namenode.caching.enabled
[ https://issues.apache.org/jira/browse/HDFS-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-5651: --- Attachment: HDFS-5651.004.patch remove dfs.namenode.caching.enabled --- Key: HDFS-5651 URL: https://issues.apache.org/jira/browse/HDFS-5651 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 3.0.0 Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Attachments: HDFS-5651.001.patch, HDFS-5651.002.patch, HDFS-5651.003.patch, HDFS-5651.004.patch We can remove dfs.namenode.caching.enabled and simply always enable caching, similar to how we do with snapshots and other features. The main overhead is the size of the cachedBlocks GSet. However, we can simply make the size of this GSet configurable, and people who don't want caching can set it to a very small value. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HDFS-5651) remove dfs.namenode.caching.enabled
[ https://issues.apache.org/jira/browse/HDFS-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-5651: --- Attachment: HDFS-5651.003.patch remove test of dfs.namenode.caching.enabled (it's not needed, since this config doesn't exist any more) remove dfs.namenode.caching.enabled --- Key: HDFS-5651 URL: https://issues.apache.org/jira/browse/HDFS-5651 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 3.0.0 Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Attachments: HDFS-5651.001.patch, HDFS-5651.002.patch, HDFS-5651.003.patch We can remove dfs.namenode.caching.enabled and simply always enable caching, similar to how we do with snapshots and other features. The main overhead is the size of the cachedBlocks GSet. However, we can simply make the size of this GSet configurable, and people who don't want caching can set it to a very small value. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HDFS-5651) remove dfs.namenode.caching.enabled
[ https://issues.apache.org/jira/browse/HDFS-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-5651: --- Status: Patch Available (was: Open) remove dfs.namenode.caching.enabled --- Key: HDFS-5651 URL: https://issues.apache.org/jira/browse/HDFS-5651 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 3.0.0 Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Attachments: HDFS-5651.001.patch We can remove dfs.namenode.caching.enabled and simply always enable caching, similar to how we do with snapshots and other features. The main overhead is the size of the cachedBlocks GSet. However, we can simply make the size of this GSet configurable, and people who don't want caching can set it to a very small value. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HDFS-5651) remove dfs.namenode.caching.enabled
[ https://issues.apache.org/jira/browse/HDFS-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-5651: --- Attachment: HDFS-5651.001.patch remove dfs.namenode.caching.enabled --- Key: HDFS-5651 URL: https://issues.apache.org/jira/browse/HDFS-5651 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 3.0.0 Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Attachments: HDFS-5651.001.patch We can remove dfs.namenode.caching.enabled and simply always enable caching, similar to how we do with snapshots and other features. The main overhead is the size of the cachedBlocks GSet. However, we can simply make the size of this GSet configurable, and people who don't want caching can set it to a very small value. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HDFS-5651) remove dfs.namenode.caching.enabled
[ https://issues.apache.org/jira/browse/HDFS-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-5651: --- Attachment: HDFS-5651.002.patch also update defaults and docs. remove dfs.namenode.caching.enabled --- Key: HDFS-5651 URL: https://issues.apache.org/jira/browse/HDFS-5651 Project: Hadoop HDFS Issue Type: Sub-task Components: namenode Affects Versions: 3.0.0 Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Attachments: HDFS-5651.001.patch, HDFS-5651.002.patch We can remove dfs.namenode.caching.enabled and simply always enable caching, similar to how we do with snapshots and other features. The main overhead is the size of the cachedBlocks GSet. However, we can simply make the size of this GSet configurable, and people who don't want caching can set it to a very small value. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
