Hi, Nick. Do you mean that you manually execute "kinit -R" to renew the ticket cache? If that is the case, Flink already sets the "renewTGT" to true. If everything is ok, you do not need to do it yourself. However, it seems this mechanism has a bug and this bug is not fixed in all JDK versions. Please refer to [1].
If you mean that you generate a new ticket cache in the same place(by default /tmp/krb5cc_uid), I'm not sure will Krb5LoginModule re-login with your new ticket cache. I'll try to do a deeper investigation. [1] https://bugs.openjdk.java.net/browse/JDK-8058290. Best, Yangze Guo On Sat, May 30, 2020 at 3:07 AM Nick Bendtner <buggi...@gmail.com> wrote: > > Hi Guo, > Thanks again for your inputs. If I periodically renew the kerberos cache > using an external process(kinit) on all flink nodes in standalone mode, will > the cluster still be short lived or will the new ticket in the cache be used > and the cluster can live till the end of the new expiry ? > > Best, > Nick. > > On Sun, May 24, 2020 at 9:15 PM Yangze Guo <karma...@gmail.com> wrote: >> >> Yes, you can use kinit. But AFAIK, if you deploy Flink on Kubernetes >> or Mesos, Flink will not ship the ticket cache. If you deploy Flink on >> Yarn, Flink will acquire delegation tokens with your ticket cache and >> set tokens for job manager and task executor. As the document said, >> the main drawback is that the cluster is necessarily short-lived since >> the generated delegation tokens will expire (typically within a week). >> >> Best, >> Yangze Guo >> >> On Sat, May 23, 2020 at 1:23 AM Nick Bendtner <buggi...@gmail.com> wrote: >> > >> > Hi Guo, >> > Even for HDFS I don't really need to set >> > "security.kerberos.login.contexts" . As long as there is the right ticket >> > in the ticket cache before starting the flink cluster it seems to work >> > fine. I think even [4] from your reference seems to do the same thing. I >> > have defined own ticket cache specifically for flink cluster by setting >> > this environment variable. Before starting the cluster I create a ticket >> > by using kinit. >> > This is how I make flink read this cache. >> > export KRB5CCNAME=/home/was/Jaas/krb5cc . I think even flink tries to find >> > the location of ticket cache using this variable [1]. >> > Do you see any problems in setting up hadoop security module this way ? >> > And thanks a lot for your help. >> > >> > [1] >> > https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/security/KerberosUtils.java >> > >> > Best, >> > Nick >> > >> > >> > >> > On Thu, May 21, 2020 at 9:54 PM Yangze Guo <karma...@gmail.com> wrote: >> >> >> >> Hi, Nick, >> >> >> >> From my understanding, if you configure the >> >> "security.kerberos.login.keytab", Flink will add the >> >> AppConfigurationEntry of this keytab to all the apps defined in >> >> "security.kerberos.login.contexts". If you define >> >> "java.security.auth.login.config" at the same time, Flink will also >> >> keep the configuration in it. For more details, see [1][2]. >> >> >> >> If you want to use this keytab to interact with HDFS, HBase and Yarn, >> >> you need to set "security.kerberos.login.contexts". See [3][4]. >> >> >> >> [1] >> >> https://ci.apache.org/projects/flink/flink-docs-master/ops/security-kerberos.html#jaas-security-module >> >> [2] >> >> https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/security/modules/JaasModule.java >> >> [3] >> >> https://ci.apache.org/projects/flink/flink-docs-master/ops/security-kerberos.html#hadoop-security-module >> >> [4] >> >> https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/security/modules/HadoopModule.java >> >> >> >> Best, >> >> Yangze Guo >> >> >> >> On Thu, May 21, 2020 at 11:06 PM Nick Bendtner <buggi...@gmail.com> wrote: >> >> > >> >> > Hi guys, >> >> > Is there any difference in providing kerberos config to the flink jvm >> >> > using this method in the flink configuration? >> >> > >> >> > env.java.opts: -Dconfig.resource=qa.conf >> >> > -Djava.library.path=/usr/mware/flink-1.7.2/simpleapi/lib/ >> >> > -Djava.security.auth.login.config=/usr/mware/flink-1.7.2/Jaas/kafka-jaas.conf >> >> > -Djava.security.krb5.conf=/usr/mware/flink-1.7.2/Jaas/krb5.conf >> >> > >> >> > Is there any difference in doing it this way vs providing it from >> >> > security.kerberos.login.keytab . >> >> > >> >> > Best, >> >> > >> >> > Nick.
