[jira] [Updated] (HDFS-6763) Initialize file system-wide quota once on transitioning to active
[
https://issues.apache.org/jira/browse/HDFS-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HDFS-6763:
-
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.8.0
3.0.0
Status: Resolved (was: Patch Available)
> Initialize file system-wide quota once on transitioning to active
> -
>
> Key: HDFS-6763
> URL: https://issues.apache.org/jira/browse/HDFS-6763
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, namenode
>Reporter: Daryn Sharp
>Assignee: Kihwal Lee
> Labels: BB2015-05-TBR
> Fix For: 3.0.0, 2.8.0
>
> Attachments: HDFS-6763.patch, HDFS-6763.v2.patch, HDFS-6763.v3.patch
>
>
> {{FSImage#loadEdits}} calls {{updateCountForQuota}} to recalculate & verify
> quotas for the entire namespace. A standby NN using shared edits calls this
> method every minute. The standby may appear to "hang" for many seconds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-6763) Initialize file system-wide quota once on transitioning to active
[
https://issues.apache.org/jira/browse/HDFS-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HDFS-6763:
-
Attachment: HDFS-6763.v3.patch
Removed more imports to fix checkstyle warnings. Failed unit tests all pass.
> Initialize file system-wide quota once on transitioning to active
> -
>
> Key: HDFS-6763
> URL: https://issues.apache.org/jira/browse/HDFS-6763
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, namenode
>Reporter: Daryn Sharp
>Assignee: Kihwal Lee
> Labels: BB2015-05-TBR
> Attachments: HDFS-6763.patch, HDFS-6763.v2.patch, HDFS-6763.v3.patch
>
>
> {{FSImage#loadEdits}} calls {{updateCountForQuota}} to recalculate & verify
> quotas for the entire namespace. A standby NN using shared edits calls this
> method every minute. The standby may appear to "hang" for many seconds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-6763) Initialize file system-wide quota once on transitioning to active
[
https://issues.apache.org/jira/browse/HDFS-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HDFS-6763:
-
Status: Patch Available (was: Open)
> Initialize file system-wide quota once on transitioning to active
> -
>
> Key: HDFS-6763
> URL: https://issues.apache.org/jira/browse/HDFS-6763
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, namenode
>Reporter: Daryn Sharp
>Assignee: Kihwal Lee
> Labels: BB2015-05-TBR
> Attachments: HDFS-6763.patch, HDFS-6763.v2.patch
>
>
> {{FSImage#loadEdits}} calls {{updateCountForQuota}} to recalculate & verify
> quotas for the entire namespace. A standby NN using shared edits calls this
> method every minute. The standby may appear to "hang" for many seconds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-6763) Initialize file system-wide quota once on transitioning to active
[
https://issues.apache.org/jira/browse/HDFS-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HDFS-6763:
-
Attachment: HDFS-6763.v2.patch
Forgot to remove three unused imports from {FSImage}. Resubmitting the patch.
> Initialize file system-wide quota once on transitioning to active
> -
>
> Key: HDFS-6763
> URL: https://issues.apache.org/jira/browse/HDFS-6763
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, namenode
>Reporter: Daryn Sharp
>Assignee: Kihwal Lee
> Labels: BB2015-05-TBR
> Attachments: HDFS-6763.patch, HDFS-6763.v2.patch
>
>
> {{FSImage#loadEdits}} calls {{updateCountForQuota}} to recalculate & verify
> quotas for the entire namespace. A standby NN using shared edits calls this
> method every minute. The standby may appear to "hang" for many seconds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-6763) Initialize file system-wide quota once on transitioning to active
[
https://issues.apache.org/jira/browse/HDFS-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HDFS-6763:
-
Attachment: (was: HDFS-6763.v2.patch)
> Initialize file system-wide quota once on transitioning to active
> -
>
> Key: HDFS-6763
> URL: https://issues.apache.org/jira/browse/HDFS-6763
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, namenode
>Reporter: Daryn Sharp
>Assignee: Kihwal Lee
> Labels: BB2015-05-TBR
> Attachments: HDFS-6763.patch
>
>
> {{FSImage#loadEdits}} calls {{updateCountForQuota}} to recalculate & verify
> quotas for the entire namespace. A standby NN using shared edits calls this
> method every minute. The standby may appear to "hang" for many seconds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-6763) Initialize file system-wide quota once on transitioning to active
[
https://issues.apache.org/jira/browse/HDFS-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HDFS-6763:
-
Status: Open (was: Patch Available)
> Initialize file system-wide quota once on transitioning to active
> -
>
> Key: HDFS-6763
> URL: https://issues.apache.org/jira/browse/HDFS-6763
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, namenode
>Reporter: Daryn Sharp
>Assignee: Kihwal Lee
> Labels: BB2015-05-TBR
> Attachments: HDFS-6763.patch, HDFS-6763.v2.patch
>
>
> {{FSImage#loadEdits}} calls {{updateCountForQuota}} to recalculate & verify
> quotas for the entire namespace. A standby NN using shared edits calls this
> method every minute. The standby may appear to "hang" for many seconds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-6763) Initialize file system-wide quota once on transitioning to active
[
https://issues.apache.org/jira/browse/HDFS-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HDFS-6763:
-
Attachment: HDFS-6763.v2.patch
Revised the patch after HDFS-8865. The quota initialization code is moved from
{{FSImage}} to {{FSDirectory}}. This is an operation against a directory tree
after all.
It was wired to {{FSImage#loadImage()}}, so {{EditLogTailer}} was causing quota
initialization every time it checks (default 1 min interval), regardless of
whether it replayed any edits or not. This really only needs to be done when
the NN becomes active.
> Initialize file system-wide quota once on transitioning to active
> -
>
> Key: HDFS-6763
> URL: https://issues.apache.org/jira/browse/HDFS-6763
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, namenode
>Reporter: Daryn Sharp
>Assignee: Kihwal Lee
> Labels: BB2015-05-TBR
> Attachments: HDFS-6763.patch, HDFS-6763.v2.patch
>
>
> {{FSImage#loadEdits}} calls {{updateCountForQuota}} to recalculate & verify
> quotas for the entire namespace. A standby NN using shared edits calls this
> method every minute. The standby may appear to "hang" for many seconds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-6763) Initialize file system-wide quota once on transitioning to active
[
https://issues.apache.org/jira/browse/HDFS-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated HDFS-6763:
---
Labels: BB2015-05-TBR (was: )
> Initialize file system-wide quota once on transitioning to active
> -
>
> Key: HDFS-6763
> URL: https://issues.apache.org/jira/browse/HDFS-6763
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, namenode
>Reporter: Daryn Sharp
>Assignee: Kihwal Lee
> Labels: BB2015-05-TBR
> Attachments: HDFS-6763.patch
>
>
> {{FSImage#loadEdits}} calls {{updateCountForQuota}} to recalculate & verify
> quotas for the entire namespace. A standby NN using shared edits calls this
> method every minute. The standby may appear to "hang" for many seconds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-6763) Initialize file system-wide quota once on transitioning to active
[
https://issues.apache.org/jira/browse/HDFS-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HDFS-6763:
-
Status: Patch Available (was: Open)
> Initialize file system-wide quota once on transitioning to active
> -
>
> Key: HDFS-6763
> URL: https://issues.apache.org/jira/browse/HDFS-6763
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, namenode
>Reporter: Daryn Sharp
>Assignee: Kihwal Lee
> Attachments: HDFS-6763.patch
>
>
> {{FSImage#loadEdits}} calls {{updateCountForQuota}} to recalculate & verify
> quotas for the entire namespace. A standby NN using shared edits calls this
> method every minute. The standby may appear to "hang" for many seconds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-6763) Initialize file system-wide quota once on transitioning to active
[
https://issues.apache.org/jira/browse/HDFS-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HDFS-6763:
-
Summary: Initialize file system-wide quota once on transitioning to active
(was: Perform file system-wide quota initialization once on transitioning to
active)
> Initialize file system-wide quota once on transitioning to active
> -
>
> Key: HDFS-6763
> URL: https://issues.apache.org/jira/browse/HDFS-6763
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, namenode
>Reporter: Daryn Sharp
>Assignee: Kihwal Lee
> Attachments: HDFS-6763.patch
>
>
> {{FSImage#loadEdits}} calls {{updateCountForQuota}} to recalculate & verify
> quotas for the entire namespace. A standby NN using shared edits calls this
> method every minute. The standby may appear to "hang" for many seconds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
