[jira] [Updated] (HDFS-3646) LeaseRenewer can hold reference to inactive DFSClient instances forever
[
https://issues.apache.org/jira/browse/HDFS-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HDFS-3646:
-
Attachment: hdfs-3646-branch-23.patch.txt
The new patch includes the suggested change.
LeaseRenewer can hold reference to inactive DFSClient instances forever
---
Key: HDFS-3646
URL: https://issues.apache.org/jira/browse/HDFS-3646
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.23.3, 2.0.0-alpha
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
Fix For: 3.0.0, 2.2.0-alpha
Attachments: hdfs-3646-branch-23.patch.txt,
hdfs-3646-branch-23.patch.txt, hdfs-3646.patch, hdfs-3646.patch.txt,
hdfs-3646.patch.txt
If {{LeaseRenewer#closeClient()}} is not called, {{LeaseRenewer}} keeps the
reference to a {{DFSClient}} instance in {{dfsclients}} forever. This
prevents {{DFSClient}}, {{LeaseRenewer}}, conf, etc. from being garbage
collected, leading to memory leak.
{{LeaseRenewer}} should remove the reference after some delay, if a
{{DFSClient}} instance no longer has active streams.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3646) LeaseRenewer can hold reference to inactive DFSClient instances forever
[
https://issues.apache.org/jira/browse/HDFS-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daryn Sharp updated HDFS-3646:
--
Target Version/s: 0.23.3, 3.0.0, 2.2.0-alpha
Fix Version/s: (was: 2.1.0-alpha)
(was: 0.23.3)
2.2.0-alpha
LeaseRenewer can hold reference to inactive DFSClient instances forever
---
Key: HDFS-3646
URL: https://issues.apache.org/jira/browse/HDFS-3646
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.23.3, 2.0.0-alpha
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
Fix For: 3.0.0, 2.2.0-alpha
Attachments: hdfs-3646.patch, hdfs-3646.patch.txt, hdfs-3646.patch.txt
If {{LeaseRenewer#closeClient()}} is not called, {{LeaseRenewer}} keeps the
reference to a {{DFSClient}} instance in {{dfsclients}} forever. This
prevents {{DFSClient}}, {{LeaseRenewer}}, conf, etc. from being garbage
collected, leading to memory leak.
{{LeaseRenewer}} should remove the reference after some delay, if a
{{DFSClient}} instance no longer has active streams.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3646) LeaseRenewer can hold reference to inactive DFSClient instances forever
[
https://issues.apache.org/jira/browse/HDFS-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HDFS-3646:
-
Attachment: hdfs-3646-branch-23.patch.txt
Sorry I forgot to post the 0.23 patch. I actually had conflicts due to absence
of HDFS-3641 and HA when I tested it on 0.23, but somehow forgot about that. I
reran all tests in HDFS with the 0.23 patch.
LeaseRenewer can hold reference to inactive DFSClient instances forever
---
Key: HDFS-3646
URL: https://issues.apache.org/jira/browse/HDFS-3646
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.23.3, 2.0.0-alpha
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
Fix For: 3.0.0, 2.2.0-alpha
Attachments: hdfs-3646-branch-23.patch.txt, hdfs-3646.patch,
hdfs-3646.patch.txt, hdfs-3646.patch.txt
If {{LeaseRenewer#closeClient()}} is not called, {{LeaseRenewer}} keeps the
reference to a {{DFSClient}} instance in {{dfsclients}} forever. This
prevents {{DFSClient}}, {{LeaseRenewer}}, conf, etc. from being garbage
collected, leading to memory leak.
{{LeaseRenewer}} should remove the reference after some delay, if a
{{DFSClient}} instance no longer has active streams.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3646) LeaseRenewer can hold reference to inactive DFSClient instances forever
[
https://issues.apache.org/jira/browse/HDFS-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HDFS-3646:
-
Attachment: hdfs-3646.patch.txt
LeaseRenewer can hold reference to inactive DFSClient instances forever
---
Key: HDFS-3646
URL: https://issues.apache.org/jira/browse/HDFS-3646
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.23.3, 2.0.0-alpha
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
Fix For: 0.23.3, 2.1.0-alpha, 3.0.0
Attachments: hdfs-3646.patch, hdfs-3646.patch.txt, hdfs-3646.patch.txt
If {{LeaseRenewer#closeClient()}} is not called, {{LeaseRenewer}} keeps the
reference to a {{DFSClient}} instance in {{dfsclients}} forever. This
prevents {{DFSClient}}, {{LeaseRenewer}}, conf, etc. from being garbage
collected, leading to memory leak.
{{LeaseRenewer}} should remove the reference after some delay, if a
{{DFSClient}} instance no longer has active streams.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3646) LeaseRenewer can hold reference to inactive DFSClient instances forever
[
https://issues.apache.org/jira/browse/HDFS-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HDFS-3646:
-
Attachment: hdfs-3646.patch.txt
Followed Dary's suggestion and simplifed the {{DFSClient}} side. All affected
tests have been updated. The termianl condition of the renewal loop in
{{LeaseRenewer}} has been changed to make it honor the grace period after
{{dfsclients}} becomes empty.
LeaseRenewer can hold reference to inactive DFSClient instances forever
---
Key: HDFS-3646
URL: https://issues.apache.org/jira/browse/HDFS-3646
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.23.3, 2.0.0-alpha
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
Fix For: 0.23.3, 2.1.0-alpha, 3.0.0
Attachments: hdfs-3646.patch, hdfs-3646.patch.txt
If {{LeaseRenewer#closeClient()}} is not called, {{LeaseRenewer}} keeps the
reference to a {{DFSClient}} instance in {{dfsclients}} forever. This
prevents {{DFSClient}}, {{LeaseRenewer}}, conf, etc. from being garbage
collected, leading to memory leak.
{{LeaseRenewer}} should remove the reference after some delay, if a
{{DFSClient}} instance no longer has active streams.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3646) LeaseRenewer can hold reference to inactive DFSClient instances forever
[
https://issues.apache.org/jira/browse/HDFS-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HDFS-3646:
-
Attachment: hdfs-3646.patch
The goal of this patch is to remove unnecessary references from both DFSClient
and LeaseRenewer whenever possible.
LeaseRenewer can hold reference to inactive DFSClient instances forever
---
Key: HDFS-3646
URL: https://issues.apache.org/jira/browse/HDFS-3646
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.23.3, 2.0.0-alpha
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
Fix For: 0.23.3, 2.0.1-alpha, 3.0.0
Attachments: hdfs-3646.patch
If {{LeaseRenewer#closeClient()}} is not called, {{LeaseRenewer}} keeps the
reference to a {{DFSClient}} instance in {{dfsclients}} forever. This
prevents {{DFSClient}}, {{LeaseRenewer}}, conf, etc. from being garbage
collected, leading to memory leak.
{{LeaseRenewer}} should remove the reference after some delay, if a
{{DFSClient}} instance no longer has active streams.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3646) LeaseRenewer can hold reference to inactive DFSClient instances forever
[
https://issues.apache.org/jira/browse/HDFS-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HDFS-3646:
-
Status: Patch Available (was: Open)
LeaseRenewer can hold reference to inactive DFSClient instances forever
---
Key: HDFS-3646
URL: https://issues.apache.org/jira/browse/HDFS-3646
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 2.0.0-alpha, 0.23.3
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
Fix For: 0.23.3, 2.0.1-alpha, 3.0.0
Attachments: hdfs-3646.patch
If {{LeaseRenewer#closeClient()}} is not called, {{LeaseRenewer}} keeps the
reference to a {{DFSClient}} instance in {{dfsclients}} forever. This
prevents {{DFSClient}}, {{LeaseRenewer}}, conf, etc. from being garbage
collected, leading to memory leak.
{{LeaseRenewer}} should remove the reference after some delay, if a
{{DFSClient}} instance no longer has active streams.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3646) LeaseRenewer can hold reference to inactive DFSClient instances forever
[
https://issues.apache.org/jira/browse/HDFS-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HDFS-3646:
-
Summary: LeaseRenewer can hold reference to inactive DFSClient instances
forever (was: LeaseRenewer can hold reference to inactive DFSClient instances)
LeaseRenewer can hold reference to inactive DFSClient instances forever
---
Key: HDFS-3646
URL: https://issues.apache.org/jira/browse/HDFS-3646
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.23.3, 2.0.0-alpha
Reporter: Kihwal Lee
Assignee: Kihwal Lee
Priority: Critical
Fix For: 0.23.3, 2.0.1-alpha, 3.0.0
If {{LeaseRenewer#closeClient()}} is not called, {{LeaseRenewer}} keeps the
reference to a {{DFSClient}} instance in {{dfsclients}} forever. This
prevents {{DFSClient}}, {{LeaseRenewer}}, conf, etc. from being garbage
collected, leading to memory leak.
{{LeaseRenewer}} should remove the reference after some delay, if a
{{DFSClient}} instance no longer has active streams.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
