Re: Delegation Token renewal in yarn-cluster

On Fri, Nov 4, 2016 at 1:57 AM, Zsolt Tóth wrote: > This was what confused me in the first place. Why does Spark ask for new > tokens based on the renew-interval instead of the max-lifetime? It could be just a harmless bug, since tokens have a "getMaxDate()" method

Re: Delegation Token renewal in yarn-cluster

On 4 Nov 2016, at 01:37, Marcelo Vanzin > wrote: On Thu, Nov 3, 2016 at 3:47 PM, Zsolt Tóth > wrote: What is the purpose of the delegation token renewal (the one that is done automatically

Re: Delegation Token renewal in yarn-cluster

I checked the logs of my tests, and found that the Spark schedules the token refresh based on the renew-interval property, not the max-lifetime. The settings in my tests: dfs.namenode.delegation.key.update-interval=52 dfs.namenode.delegation.token.max-lifetime=102

Re: Delegation Token renewal in yarn-cluster

On Thu, Nov 3, 2016 at 3:47 PM, Zsolt Tóth wrote: > What is the purpose of the delegation token renewal (the one that is done > automatically by Hadoop libraries, after 1 day by default)? It seems that it > always happens (every day) until the token expires, no matter

Re: Delegation Token renewal in yarn-cluster

Thank you for the clarification Marcelo, makes sense. I'm thinking about 2 questions here, somewhat unrelated to the original problem. What is the purpose of the delegation token renewal (the one that is done automatically by Hadoop libraries, after 1 day by default)? It seems that it always

Re: Delegation Token renewal in yarn-cluster

I think you're a little confused about what "renewal" means here, and this might be the fault of the documentation (I haven't read it in a while). The existing delegation tokens will always be "renewed", in the sense that Spark (actually Hadoop code invisible to Spark) will talk to the NN to

Re: Delegation Token renewal in yarn-cluster

Yes, I did change dfs.namenode.delegation.key.update-interval and dfs.namenode.delegation.token.renew-interval to 15 min, the max-lifetime to 30min. In this case the application (without Spark having the keytab) did not fail after 15 min, only after 30 min. Is it possible that the resource manager

Re: Delegation Token renewal in yarn-cluster

Sounds like your test was set up incorrectly. The default TTL for tokens is 7 days. Did you change that in the HDFS config? The issue definitely exists and people definitely have run into it. So if you're not hitting it, it's most definitely an issue with your test configuration. On Thu, Nov 3,

Re: Delegation Token renewal in yarn-cluster

Any ideas about this one? Am I missing something here? 2016-11-03 15:22 GMT+01:00 Zsolt Tóth : > Hi, > > I ran some tests regarding Spark's Delegation Token renewal mechanism. As > I see, the concept here is simple: if I give my keytab file and client > principal to

Delegation Token renewal in yarn-cluster

Hi, I ran some tests regarding Spark's Delegation Token renewal mechanism. As I see, the concept here is simple: if I give my keytab file and client principal to Spark, it starts a token renewal thread, and renews the namenode delegation tokens after some time. This works fine. Then I tried to