[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154204#comment-14154204 ] Hadoop QA commented on HADOOP-11157: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12672199/HADOOP-11157.patch against trunk revision a469833. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.crypto.random.TestOsSecureRandom org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4841//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4841//console This message is automatically generated. > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Gregory Chanan > Attachments: HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182326#comment-14182326 ] Hadoop QA commented on HADOOP-11157: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12676805/HADOOP-11157.patch against trunk revision db45f04. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4944//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4944//console This message is automatically generated. > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Gregory Chanan > Attachments: HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14185623#comment-14185623 ] Gregory Chanan commented on HADOOP-11157: - Test failure looks unrelated, passes locally for me. > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Gregory Chanan > Attachments: HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14185805#comment-14185805 ] Karthik Kambatla commented on HADOOP-11157: --- Thanks for working on this, Greg. Couple of minor comments: Can we wait for a timeout after shutdown? Also, it would be nice to have a unit test. > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Gregory Chanan > Attachments: HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14187567#comment-14187567 ] Gregory Chanan commented on HADOOP-11157: - [~kkambatl] while writing up a test as you requested, I found a number of other issues. This will be kind of scatter-brained, sorry: 1) related to shutdown - a) the ExpiredToken is shut down after the ZKDelegationTokenSecretManager's curator, which causes an exception to be thrown and the process to exit. This can be addressed by shutting down the ExpiredToken thread before the curator. - b) even with a), the ExpiredTokenThread is interrupted by AbstractDelegationTokenSecretManager.closeThreads...if the ExpiredTokenThread is currently rolling the master key or expiring tokens in ZK, the interruption will cause the process to exit. It seems like this can be addressed by holding the noInterruptsLock while the ExpiredTokenThread is not sleeping (should be waiting), but I'm not sure if we want to go that route. Perhaps alternatively we could deal with the interruption by checking if its expected (i.e. if running is false). One issue is that approach is that the ZKDelegationTokenSecretManager functions called from the ExpiredTokenThread don't throw or keep the interrupt flag, they just catch the exceptions and possibly throw them as a runtime exception. I'm not sure if we can just swallow the InterruptedException -- presumably we need the ZK state to be in some reasonable state in case the process restarts? Of course we have no tests of that... 2) not related to shutdown - a) if you run TestZKDelegationTokenSecretManager#testCancelTokenSingleManager in a loop it will fail eventually. It looks like the issue is how we deal with asynchronous ZK updates. Consider the following code: {code} token = createToken cancelToken(token) verifyToken(token){code} cancelToken will delete it from the local cache and delete the znode. But the curator client will get the create child message (in the listener thread) and add the token back. If that happens after cancelToken, the token will be added back until the listener thread gets the cancel message again. (It also just occurred to me that this is happening in two different threads but some of the structures, like the currentToken, aren't thread safe). The usual way to prevent this is to assign versions to the znodes so you can track whether you are getting an update for an old version. I don't know how to deal with it in this case where deletes are a possibility and there doesn't appear to be a master that is responsible for writing (i.e. what is preventing some other SecretManager from recreating the token just after delete -- how would versions help with that?). This may affect the keyCache as well as the tokenCache. > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Gregory Chanan > Attachments: HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14187594#comment-14187594 ] Gregory Chanan commented on HADOOP-11157: - Here's another issue I think could happen, but have no test for: 1) set up two SecretManagers sharing zk 2) get a delegation token from one 3) use on both 4) renew on one around token expiration time Then, both SecretManagers will run the token expiration code and possibly expire the newly renewed token. > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Gregory Chanan > Attachments: HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14189301#comment-14189301 ] Hadoop QA commented on HADOOP-11157: {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12678049/HADOOP-11157.2.patch against trunk revision 3ae84e1. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4984//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4984//console This message is automatically generated. > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Attachments: HADOOP-11157.2.patch, HADOOP-11157.patch, > HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191152#comment-14191152 ] Gregory Chanan commented on HADOOP-11157: - Some notes: {code} private void processTokenAddOrUpdate(ChildData data) throws IOException { Stat stat = null; try { stat = zkClient.checkExists().forPath(data.getPath()); } catch (Exception e) { LOG.warn("Could not get path for Token Add/Update notification.. going to update !!", e); stat = null; } {code} I don't think setting stat to null is necessary and I don't understand the warning -- aren't you not going to do anything in the rest of the function because stat is null anyway? {code} + stat = zkClient.checkExists().forPath(data.getPath()); +} catch (Exception e) { + LOG.warn("Could not get path for Token Delete notification.. going to delete from localcache !!", e); + stat = null; +} {code} Again, setting stat to null doesn't seem necessary. {code} +// Check if Token has already been cancelled.. +if (stat == null) { {code} Here, and in the opposite case where stat == null on the add/remove, don't we want to handle those cases? We aren't guaranteed to see every notification (http://zookeeper.apache.org/doc/trunk/zookeeperProgrammers.html#ch_zkWatches). Should we just have one handle function where you run the logic based on the current state? - It would be nice if there were a test for starting a secret manager after a delegation token on another secret manager has already been created, and verifying it works. Also, the same case but shutting down and restarting a secret manager and verifying the tokens (for itself or for others still works). > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Attachments: HADOOP-11157.2.patch, HADOOP-11157.patch, > HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191185#comment-14191185 ] Arun Suresh commented on HADOOP-11157: -- bq. Here, and in the opposite case where stat == null on the add/remove, don't we want to handle those case No, in the event the node receives an Add / Update event, it checks ZK.. it stores in local cache ONLY if (stat != null).. it (stat == null) it should not do anything, since the node might have been removed by another ZKDTSM. As per the Log message.. I agree i can change it.. I am also considering throwing the {{IOException}} and bailing out if it can't reach ZK. bq. We aren't guaranteed to see every notification (http://zookeeper.apache.org/doc/trunk/zookeeperProgrammers.html#ch_zkWatches) I am using Curator PathCache.. which as per the documentation and some discussion sez it receives all events eventually : https://groups.google.com/forum/#!msg/curator-users/mGkG8w6PG9w/zv8jkiEEE78J bq. It would be nice if there were a test for starting a secret manager after a delegation token on another secret manager has already been sure.. will add some more tests.. > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Attachments: HADOOP-11157.2.patch, HADOOP-11157.patch, > HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191275#comment-14191275 ] Hadoop QA commented on HADOOP-11157: {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12678383/HADOOP-11157.3.patch against trunk revision e1f7d65. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-common-project/hadoop-common. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4992//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4992//console This message is automatically generated. > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14192859#comment-14192859 ] Gregory Chanan commented on HADOOP-11157: - Thanks for the patch, Arun. Some notes: - I haven't looked at the renew logic in depth and will probably skip it as outside the scope of this JIRA. - I haven't looked at the new cancel logic in depth, but I will soon. - The new tests look good Other things: {code} private static final int NUM_ZK_RETRIES = 3; + private static final int ZK_RETRY_WAIT_MS = 200; {code} These don't seem to be used. {code} ++ZK_DTSM_ZK_SHUTDOWN_TIMEOUT_DEFAULT); {code} extra plus? {code}allKeys.put(key.getKeyId(), key);{code} does this need to be synchronized? Why does the currentToken stuff need to be, but this not? {code} LOG.debug("Thread interrupted while performing token counter increment", e); LOG.debug("Thread interrupted while performing keyId increment", e); {code} Set the interrupt flag in both these cases? {code} public ExecutorService getListenerThreadPool() { {code} @VisibleForTesting? {code} Thread.sleep(5000); {code} Maybe make a constant (it's used a couple times) and reduce the time? Seems like a second should be sufficient {code} // Create a new token on ne ZKDTSM {code} ne? {code} protected void verifyDestroy(DelegationTokenManager tm, Configuration conf) {code} rename to destroyAndVerify? Some idents seem to be 2 spaces, some 4. > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14193755#comment-14193755 ] Hadoop QA commented on HADOOP-11157: {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12678774/HADOOP-11157.4.patch against trunk revision 5c0381c. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-common-project/hadoop-common. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5003//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5003//console This message is automatically generated. > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.4.patch, HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14195946#comment-14195946 ] Gregory Chanan commented on HADOOP-11157: - Hmm, I put all the test methods in a loop that ran 100 times and testMultiNodeOperations failed (others might as well, but that one failed first). > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.4.patch, HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14195954#comment-14195954 ] Arun Suresh commented on HADOOP-11157: -- Can you point me to where it failed exactly ? > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.4.patch, HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196508#comment-14196508 ] Gregory Chanan commented on HADOOP-11157: - Looks like it's only the one test (I increased the timeout so all the tests would run): {code} Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 2,628.821 sec <<< FAILURE! - in org.apache.hadoop.security.token.delegation.TestZKDelegationTokenSecretManager testMultiNodeOperations(org.apache.hadoop.security.token.delegation.TestZKDelegationTokenSecretManager) Time elapsed: 7.557 sec <<< FAILURE! java.lang.AssertionError: Expected InvalidToken at org.junit.Assert.fail(Assert.java:88) at org.apache.hadoop.security.token.delegation.TestZKDelegationTokenSecretManager.testMultiNodeOperations(TestZKDelegationTokenSecretManager.java:97) Results : Failed tests: TestZKDelegationTokenSecretManager.testMultiNodeOperations:97 Expected InvalidToken Tests run: 5, Failures: 1, Errors: 0, Skipped: 0 {code} Note, the line number is off because I modified the test to run in loops. It's this line: https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/token/delegation/TestZKDelegationTokenSecretManager.java#L90 > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.4.patch, HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196509#comment-14196509 ] Gregory Chanan commented on HADOOP-11157: - Sorry, permanent link: https://github.com/apache/hadoop/blob/2bb327eb939f57626d3dac10f7016ed634375d94/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/token/delegation/TestZKDelegationTokenSecretManager.java#L90 > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.4.patch, HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196736#comment-14196736 ] Hadoop QA commented on HADOOP-11157: {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12679282/HADOOP-11157.5.patch against trunk revision 1eed102. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-common-project/hadoop-common. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5016//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5016//console This message is automatically generated. > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.4.patch, HADOOP-11157.5.patch, HADOOP-11157.patch, > HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197430#comment-14197430 ] Hadoop QA commented on HADOOP-11157: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12679457/HADOOP-11157.6.patch against trunk revision 0762b4a. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The following test timeouts occurred in hadoop-common-project/hadoop-common: org.apache.hadoop.security.token.delegation.TestZKDelegationTokenSecretManager {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5024//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5024//console This message is automatically generated. > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.4.patch, HADOOP-11157.5.patch, HADOOP-11157.6.patch, > HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201174#comment-14201174 ] Gregory Chanan commented on HADOOP-11157: - patch lgtm. One note, not really related to this JIRA, is it doesn't look like the tokenCache and keyCache actually need to have cacheData=true. Looks like they are just being used for notification. > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.4.patch, HADOOP-11157.5.patch, HADOOP-11157.6.patch, > HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209257#comment-14209257 ] Aaron T. Myers commented on HADOOP-11157: - Latest patch looks pretty good to me. A few comments: # Looks like we still have an inadvertent extra "+" here: "++ ZK_DTSM_ZK_SHUTDOWN_TIMEOUT_DEFAULT);" # In {{stopThreads(...)}}, do we really need to nest all of those try/finally blocks? Given that content of most of the "{{catch}}" blocks is just to log and ignore, seems like they don't need to be nested, and instead we can do away with the finally blocks and just list the {{try/catch}} blocks linearly. # In the case that shutting down the thread pool times out, recommend logging an error before forcefully shutting down the thread pool. # Given that the new tests time out when run on our Jenkins infrastructure, seems like we should lower the default number of {{TEST_RETRIES}} to 1 or 2. I'll be +1 once the above are addressed and Jenkins comes back clean. > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.4.patch, HADOOP-11157.5.patch, HADOOP-11157.6.patch, > HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209352#comment-14209352 ] Hadoop QA commented on HADOOP-11157: {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12681257/HADOOP-11157.7.patch against trunk revision e073b61. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-common-project/hadoop-common. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5076//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5076//console This message is automatically generated. > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.4.patch, HADOOP-11157.5.patch, HADOOP-11157.6.patch, > HADOOP-11157.7.patch, HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14215152#comment-14215152 ] Aaron T. Myers commented on HADOOP-11157: - +1, the latest patch looks good to me. I'm going to commit this momentarily. > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.4.patch, HADOOP-11157.5.patch, HADOOP-11157.6.patch, > HADOOP-11157.7.patch, HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14215186#comment-14215186 ] Hudson commented on HADOOP-11157: - SUCCESS: Integrated in Hadoop-trunk-Commit #6558 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/6558/]) HADOOP-11157. ZKDelegationTokenSecretManager never shuts down listenerThreadPool. Contributed by Arun Suresh. (atm: rev bd8196e85e49d44de57237a59bcd7ceae4332c2e) * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/token/delegation/TestZKDelegationTokenSecretManager.java * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/ZKDelegationTokenSecretManager.java > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Fix For: 2.7.0 > > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.4.patch, HADOOP-11157.5.patch, HADOOP-11157.6.patch, > HADOOP-11157.7.patch, HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14216089#comment-14216089 ] Hudson commented on HADOOP-11157: - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #9 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/9/]) HADOOP-11157. ZKDelegationTokenSecretManager never shuts down listenerThreadPool. Contributed by Arun Suresh. (atm: rev bd8196e85e49d44de57237a59bcd7ceae4332c2e) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/ZKDelegationTokenSecretManager.java * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/token/delegation/TestZKDelegationTokenSecretManager.java > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Fix For: 2.7.0 > > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.4.patch, HADOOP-11157.5.patch, HADOOP-11157.6.patch, > HADOOP-11157.7.patch, HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14216102#comment-14216102 ] Hudson commented on HADOOP-11157: - SUCCESS: Integrated in Hadoop-Yarn-trunk #747 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/747/]) HADOOP-11157. ZKDelegationTokenSecretManager never shuts down listenerThreadPool. Contributed by Arun Suresh. (atm: rev bd8196e85e49d44de57237a59bcd7ceae4332c2e) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/ZKDelegationTokenSecretManager.java * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/token/delegation/TestZKDelegationTokenSecretManager.java > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Fix For: 2.7.0 > > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.4.patch, HADOOP-11157.5.patch, HADOOP-11157.6.patch, > HADOOP-11157.7.patch, HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14216201#comment-14216201 ] Hudson commented on HADOOP-11157: - FAILURE: Integrated in Hadoop-Hdfs-trunk #1937 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1937/]) HADOOP-11157. ZKDelegationTokenSecretManager never shuts down listenerThreadPool. Contributed by Arun Suresh. (atm: rev bd8196e85e49d44de57237a59bcd7ceae4332c2e) * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/token/delegation/TestZKDelegationTokenSecretManager.java * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/ZKDelegationTokenSecretManager.java > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Fix For: 2.7.0 > > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.4.patch, HADOOP-11157.5.patch, HADOOP-11157.6.patch, > HADOOP-11157.7.patch, HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14216215#comment-14216215 ] Hudson commented on HADOOP-11157: - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #9 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/9/]) HADOOP-11157. ZKDelegationTokenSecretManager never shuts down listenerThreadPool. Contributed by Arun Suresh. (atm: rev bd8196e85e49d44de57237a59bcd7ceae4332c2e) * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/token/delegation/TestZKDelegationTokenSecretManager.java * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/ZKDelegationTokenSecretManager.java > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Fix For: 2.7.0 > > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.4.patch, HADOOP-11157.5.patch, HADOOP-11157.6.patch, > HADOOP-11157.7.patch, HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14216275#comment-14216275 ] Hudson commented on HADOOP-11157: - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1961 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1961/]) HADOOP-11157. ZKDelegationTokenSecretManager never shuts down listenerThreadPool. Contributed by Arun Suresh. (atm: rev bd8196e85e49d44de57237a59bcd7ceae4332c2e) * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/ZKDelegationTokenSecretManager.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/token/delegation/TestZKDelegationTokenSecretManager.java > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Fix For: 2.7.0 > > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.4.patch, HADOOP-11157.5.patch, HADOOP-11157.6.patch, > HADOOP-11157.7.patch, HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11157) ZKDelegationTokenSecretManager never shuts down listenerThreadPool
[ https://issues.apache.org/jira/browse/HADOOP-11157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14216296#comment-14216296 ] Hudson commented on HADOOP-11157: - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #9 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/9/]) HADOOP-11157. ZKDelegationTokenSecretManager never shuts down listenerThreadPool. Contributed by Arun Suresh. (atm: rev bd8196e85e49d44de57237a59bcd7ceae4332c2e) * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/token/delegation/TestZKDelegationTokenSecretManager.java * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/ZKDelegationTokenSecretManager.java > ZKDelegationTokenSecretManager never shuts down listenerThreadPool > -- > > Key: HADOOP-11157 > URL: https://issues.apache.org/jira/browse/HADOOP-11157 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.6.0 >Reporter: Gregory Chanan >Assignee: Arun Suresh > Fix For: 2.7.0 > > Attachments: HADOOP-11157.2.patch, HADOOP-11157.3.patch, > HADOOP-11157.4.patch, HADOOP-11157.5.patch, HADOOP-11157.6.patch, > HADOOP-11157.7.patch, HADOOP-11157.patch, HADOOP-11157.patch > > > I'm trying to integrate Solr with the DelegationTokenAuthenticationFilter and > running into this issue. The solr unit tests look for leaked threads and > when I started using the ZKDelegationTokenSecretManager it started reporting > leaks. Shuting down the listenerThreadPool after the objects that use it > resolves the leak threads errors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)