[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-218707783 **[Test build #58467 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58467/consoleFull)** for PR 9875 at commit [`70f610f`](https://github.com/apache/spark/commit/70f610f855c3fd2b3ea9b0a44f6015162f555f87). * This patch **fails R style tests**. * This patch **does not merge cleanly**. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-218707787 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-218707788 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58467/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-218706876 **[Test build #58467 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58467/consoleFull)** for PR 9875 at commit [`70f610f`](https://github.com/apache/spark/commit/70f610f855c3fd2b3ea9b0a44f6015162f555f87). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user djdean commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-160785941 After applying the provided patch things still do not work. I've been doing some debugging, I've found some additional information. When it works, it seems that two tokens are created with the renewal interval being set for the first one using the "getTokenRenewalInterval(stagingDirPath)" function in Client.scala. The second time around (after stopping and restarting the context), however, it prints a message saying 1 token was created, but no renewal interval is set. Finally, it dies saying the token can't be found in the cache. The relevant output is below (ip/hostnames removed): ---Successful run-- 15/11/30 14:18:57 INFO yarn.Client: Credentials file set to: credentials-372be24e-9614-48d4-9f51-4cf275c51f46 15/11/30 14:18:57 INFO yarn.YarnSparkHadoopUtil: delegTokenRenewer: hadoop 15/11/30 14:18:57 INFO yarn.YarnSparkHadoopUtil: getting token for namenode: hdfs://HOSTNAME:9000/user/hadoop/.sparkStaging/application_1446695132208_0114 15/11/30 14:18:57 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 142 for hadoop on xxx.xxx.xxx.xxx 15/11/30 14:18:57 INFO yarn.Client: Renewal Interval set to 86400400 15/11/30 14:18:57 INFO yarn.Client: Preparing resources for our AM container 15/11/30 14:18:57 INFO yarn.YarnSparkHadoopUtil: delegTokenRenewer: rm/HOSTNAME 15/11/30 14:18:57 INFO yarn.YarnSparkHadoopUtil: getting token for namenode: hdfs://HOSTNAME:9000/user/hadoop/.sparkStaging/application_1446695132208_0114 15/11/30 14:18:57 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 143 for hadoop on xxx.xxx.xxx.xxx 15/11/30 14:18:58 INFO yarn.YarnSparkHadoopUtil: Hive class not found java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf 15/11/30 14:18:58 INFO yarn.Client: To enable the AM to login from keytab, credentials are being copied over to the AM via the YARN Secure Distributed Cache. 15/11/30 14:18:58 INFO yarn.Client: Uploading resource file:/etc/security/keytabs/hadoop.keytab -> /user/hadoop/.sparkStaging/application_1446695132208_0114/hadoop.keytab End successful run-- Failed run 15/11/30 14:19:46 INFO yarn.Client: Credentials file set to: credentials-b91660b6-a7c4-49f1-b869-ded70fec1641 15/11/30 14:19:46 INFO yarn.Client: Preparing resources for our AM container 15/11/30 14:19:46 INFO yarn.YarnSparkHadoopUtil: Called with conf: Configuration: core-default.xml, core-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml, hdfs-default.xml, hdfs-site.xml 15/11/30 14:19:46 INFO yarn.YarnSparkHadoopUtil: delegTokenRenewer: rm/HOSTNAME 15/11/30 14:19:46 INFO yarn.YarnSparkHadoopUtil: getting token for namenode: hdfs://HOSTNAME:9000/user/hadoop/.sparkStaging/application_1446695132208_0115 15/11/30 14:19:46 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 144 for hadoop on xxx.xxx.xxx.xxx 15/11/30 14:19:46 INFO yarn.YarnSparkHadoopUtil: Hive class not found java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf 15/11/30 14:19:46 INFO yarn.Client: To enable the AM to login from keytab, credentials are being copied over to the AM via the YARN Secure Distributed Cache. 15/11/30 14:19:46 INFO yarn.Client: Uploading resource file:/etc/security/keytabs/hadoop.keytab -> hdfs://HOSTNAME:9000/user/hadoop/.sparkStaging/application_1446695132208_0115/hadoop.keytab 15/11/30 14:19:46 INFO yarn.Client: Uploading resource file:/var/tmp/spark-1.6.0-SNAPSHOT-bin-patch-8/lib/spark-assembly-1.6.0-SNAPSHOT-hadoop2.7.1.jar -> hdfs://HOSTNAME/user/hadoop/.sparkStaging/application_1446695132208_0115/spark-assembly-1.6.0-SNAPSHOT-hadoop2.7.1.jar 15/11/30 14:19:58 WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): token (HDFS_DELEGATION_TOKEN token 143 for hadoop) can't be found in cache 15/11/30 14:19:58 WARN hdfs.LeaseRenewer: Failed to renew lease for [DFSClient_NONMAPREDUCE_-695293104_13] for 30 seconds. Will retry shortly ... org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): token (HDFS_DELEGATION_TOKEN token 143 for hadoop) can't be found in cache --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-159281467 If we are going to include the fix for the issue @SaintBacchus mentioned then I think the right thing is to only login from the keytab or get the tokens (not both).If keytab is supplied always use that and don't bother with tokens on the driver, otherwise get the tokens. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user steveloughran commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-159241477 Is the situation where the client can only have tokens need to be covered? As users may not have keytabs, only kinit-granted tickets, and they still have the right to submit work with a lifespan <= ticket life. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-159085964 Also, to be clear..it would work fine in Cluster mode. In Client mode, #7394 should have taken care of the long-running app issue (though there was one where @SaintBacchus I think mentioned that the `EventLoggingListener` was not able to write to HDFS anymore, even though the tokens were getting updated. So in either case, I am wondering whether in client mode, we should simply re-login using keytab and not bother with tokens on the driver app at all (so the AM would login, update tokens etc, while the client app always just logs in). So you think that make sense, @tgravescs? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-158988545 Yes, that is correct but this happens even without the tokens expiring. What do you think about doing the relogin in the YarnCLientSchedulerBackend? This is messy with client mode but I don't think we have another option though. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-158958240 @harishreedharan I thought this issue existed even without reusing the jvm (going by other jira and prs that were filed)? For instance if you just have a long running process and had specified a keytab to use. I thought they had said it wasn't relogging in because of the token that it was acquiring. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-158576879 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-158576880 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46466/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-158576831 **[Test build #46466 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46466/consoleFull)** for PR 9875 at commit [`70f610f`](https://github.com/apache/spark/commit/70f610f855c3fd2b3ea9b0a44f6015162f555f87). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-158573490 **[Test build #46466 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46466/consoleFull)** for PR 9875 at commit [`70f610f`](https://github.com/apache/spark/commit/70f610f855c3fd2b3ea9b0a44f6015162f555f87). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-158573009 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-158568192 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-158568193 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46454/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-158568143 **[Test build #46454 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46454/consoleFull)** for PR 9875 at commit [`70f610f`](https://github.com/apache/spark/commit/70f610f855c3fd2b3ea9b0a44f6015162f555f87). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-158567863 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-158567864 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46458/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-158567789 **[Test build #46458 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46458/consoleFull)** for PR 9875 at commit [`70f610f`](https://github.com/apache/spark/commit/70f610f855c3fd2b3ea9b0a44f6015162f555f87). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-158565442 **[Test build #46458 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46458/consoleFull)** for PR 9875 at commit [`70f610f`](https://github.com/apache/spark/commit/70f610f855c3fd2b3ea9b0a44f6015162f555f87). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-158563415 **[Test build #46454 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46454/consoleFull)** for PR 9875 at commit [`70f610f`](https://github.com/apache/spark/commit/70f610f855c3fd2b3ea9b0a44f6015162f555f87). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/9875#issuecomment-158563127 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-11662] [YARN]. In Client mode, make sur...
GitHub user harishreedharan opened a pull request: https://github.com/apache/spark/pull/9875 [SPARK-11662] [YARN]. In Client mode, make sure we re-login before at⦠â¦tempting to create new delegation tokens if a new SparkContext is created within the same application. Since Hadoop gives precedence to the delegation tokens, we must make sure we login as a different user, get new tokens and replace the old ones in the current user's credentials cache to avoid not being able to get new ones. /cc @tedyu @tgravescs You can merge this pull request into a Git repository by running: $ git pull https://github.com/harishreedharan/spark keytab-relogin-restarted-context Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/9875.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #9875 commit 70f610f855c3fd2b3ea9b0a44f6015162f555f87 Author: Hari Shreedharan Date: 2015-11-20T23:42:01Z [SPARK-11662] [YARN]. In Client mode, make sure we re-login before attempting to create new delegation tokens if a new SparkContext is created within the same application. Since Hadoop gives precedence to the delegation tokens, we must make sure we login as a different user, get new tokens and replace the old ones in the current user's credentials cache to avoid not being able to get new ones. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
