[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer thread is not stopped when its filesystem is closed
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13464652#comment-13464652 ] Tom White commented on HADOOP-8852: --- Generally looks good. * Rather than always starting a DelegationTokenRenewer thread in the HftpFileSystem/WebHdfsFileSystem constructor (which is a bad practice, and is why findbugs is complaining), you could do what WebHdfsFileSystem does already and only start the thread if security is enabled. See the addRenewAction() method on WebHdfsFileSystem. * The 3000ms join timeout could be made into a constant. Perhaps you could add a shutdown method to DelegationTokenRenewer which encapsulates the interrupt and join. DelegationTokenRenewer thread is not stopped when its filesystem is closed -- Key: HADOOP-8852 URL: https://issues.apache.org/jira/browse/HADOOP-8852 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.0-alpha Reporter: Tom White Assignee: Karthik Kambatla Attachments: hadoop-8852.patch, hadoop-8852.patch HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer thread is not stopped when its filesystem is closed
[
https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13464708#comment-13464708
]
Daryn Sharp commented on HADOOP-8852:
-
Hmm. I think we may have regressed functionality because I thought I Nicholas
and I added lazy start to the renewer when the first token is added. While a
little bit larger in scope, and it's something I've always been meaning to do,
I'd suggest the renewer should:
* be a singleton to avoid multiple threads
* lazy starts when a token is added
* filesystem won't explicitly start the renewer or join with it
* when hftp or webhdfs is closed, it should cancel the token if
{{initDelegationToken}} had to create a token
* the cancel methods should tell the renewer to stop renewing the canceled
token. perhaps the renewer should be the one to cancel the token to avoid race
conditions (fs cancels just before the renewer tries to renew)
* the renewer should shutdown if it runs out of tokens
DelegationTokenRenewer thread is not stopped when its filesystem is closed
--
Key: HADOOP-8852
URL: https://issues.apache.org/jira/browse/HADOOP-8852
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.0-alpha
Reporter: Tom White
Assignee: Karthik Kambatla
Attachments: hadoop-8852.patch, hadoop-8852.patch
HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer
thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer thread is not stopped when its filesystem is closed
[
https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13464846#comment-13464846
]
Karthik Kambatla commented on HADOOP-8852:
--
Thanks Tom and Daryn for your comments.
Daryn, your suggestions make absolute sense. Will work on the patch. While
working on this earlier, I thought of making it a Singleton, but thought there
was a reason why it was not Singleton :)
I will add {{cancelToken()}} and {{shutdown()}} to {{DelegationTokenRenewer}},
and use them in the FileSystems.
DelegationTokenRenewer thread is not stopped when its filesystem is closed
--
Key: HADOOP-8852
URL: https://issues.apache.org/jira/browse/HADOOP-8852
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.0-alpha
Reporter: Tom White
Assignee: Karthik Kambatla
Attachments: hadoop-8852.patch, hadoop-8852.patch
HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer
thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer thread is not stopped when its filesystem is closed
[
https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13464120#comment-13464120
]
Karthik Kambatla commented on HADOOP-8852:
--
{{DelegationTokenRenewer}} fields in both filesystems are static fields. As
stopping the thread corresponding to static field in close() does not seem like
good practice, we should make them non-static.
DelegationTokenRenewer thread is not stopped when its filesystem is closed
--
Key: HADOOP-8852
URL: https://issues.apache.org/jira/browse/HADOOP-8852
Project: Hadoop Common
Issue Type: Bug
Reporter: Tom White
Assignee: Karthik Kambatla
HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer
thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer thread is not stopped when its filesystem is closed
[
https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13464235#comment-13464235
]
Karthik Kambatla commented on HADOOP-8852:
--
By the way, in the little time I worked on Hadoop so far, I noticed several
cases like this where {{close()}} does not stop _all_ the threads it spawns. I
feel we should add more structure to ensure we do not miss any of them.
Thoughts?
DelegationTokenRenewer thread is not stopped when its filesystem is closed
--
Key: HADOOP-8852
URL: https://issues.apache.org/jira/browse/HADOOP-8852
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.0-alpha
Reporter: Tom White
Assignee: Karthik Kambatla
Attachments: hadoop-8852.patch
HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer
thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer thread is not stopped when its filesystem is closed
[
https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13464311#comment-13464311
]
Hadoop QA commented on HADOOP-8852:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12546759/hadoop-8852.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1533//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1533//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1533//console
This message is automatically generated.
DelegationTokenRenewer thread is not stopped when its filesystem is closed
--
Key: HADOOP-8852
URL: https://issues.apache.org/jira/browse/HADOOP-8852
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.0-alpha
Reporter: Tom White
Assignee: Karthik Kambatla
Attachments: hadoop-8852.patch
HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer
thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer thread is not stopped when its filesystem is closed
[ https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13464319#comment-13464319 ] Karthik Kambatla commented on HADOOP-8852: -- When I ran test-patch.sh locally, it didn't complain of any findbugs warnings. DelegationTokenRenewer thread is not stopped when its filesystem is closed -- Key: HADOOP-8852 URL: https://issues.apache.org/jira/browse/HADOOP-8852 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.0-alpha Reporter: Tom White Assignee: Karthik Kambatla Attachments: hadoop-8852.patch HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer thread when they are closed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8852) DelegationTokenRenewer thread is not stopped when its filesystem is closed
[
https://issues.apache.org/jira/browse/HADOOP-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13464423#comment-13464423
]
Hadoop QA commented on HADOOP-8852:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12546790/hadoop-8852.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
org.apache.hadoop.hdfs.TestPersistBlocks
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1535//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1535//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1535//console
This message is automatically generated.
DelegationTokenRenewer thread is not stopped when its filesystem is closed
--
Key: HADOOP-8852
URL: https://issues.apache.org/jira/browse/HADOOP-8852
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.0-alpha
Reporter: Tom White
Assignee: Karthik Kambatla
Attachments: hadoop-8852.patch, hadoop-8852.patch
HftpFileSystem and WebHdfsFileSystem should stop the DelegationTokenRenewer
thread when they are closed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
