You could set config option "kubernetes.namespace" to your flink-conf ConfigMap. And then KubernetesHAService will use it to create/watch the ConfigMap. Please note the default service account has enough permission.
Of course, you could also set the config option "kubernetes.service-account" to another service account with enough permission. Best, Yang Ashish Nigam <ashnigamt...@gmail.com> 于2021年1月22日周五 上午3:16写道： > It works now. Job manager is able to start. But now, I have run into > another issue. > > It seems job manager is trying to create configmap in default namespace > and namespace/service account where I run job manager does not have access > to configmap > > GET at: > https://X.X.X.X/api/v1/namespaces/default/configmaps/dev-cluster-restserver-leader. > Message: Forbidden!Configured service account doesn't have access. Service > account may have been revoked. configmaps "dev-cluster-restserver-leader" > is forbidden: User "system:serviceaccount:apps-dev:apps" cannot get > resource "configmaps" in API group "" in the namespace "default". > > Is there a workaround apart from adding permission? Can leader configmap > be created in the same namespace as Job Manager? > > On Wed, Jan 20, 2021 at 7:42 PM Yang Wang <danrtsey...@gmail.com> wrote: > >> You do not need to put flink-kubernetes_2.12-1.12.1.jar under the plugin >> directory. Only the S3 fs jar >> needs to be put there. >> >> I think I found the root cause. It seems your flink-dist is still 1.11. >> Do you want to use the image 1.11 and enable the K8s HA at the same time? >> I think it could not work. Because the HA related codes have been >> refactored from release 1.12. >> Even though you copy the flink-kubernetes_2.12-1.12.1.jar to >> /opt/flink/lib and make the KubernetesHaServicesFactory class >> could be resolved. It will encounter other issues after then. >> >> Could you please try with the latest Flink 1.12 image? >> docker pull apache/flink:1.12.1 >> >> Best, >> Yang >> >> Ashish Nigam <ashnigamt...@gmail.com> 于2021年1月21日周四 上午8:05写道： >> >>> I did move to 1.12.1 version and also ensured that docker has kubernetes >>> jar file in the right location, i.e. >>> /opt/flink/plugins/s3-fs-presto/flink-kubernetes_2.12-1.12.1.jar >>> >>> But job manager is still not able to find the class >>> >>> 2021-01-21 00:00:49,376 ERROR >>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Could not >>> start cluster entrypoint StandaloneApplicationClusterEntryPoint. >>> org.apache.flink.runtime.entrypoint.ClusterEntrypointException: Failed >>> to initialize the cluster entrypoint StandaloneApplicationClusterEntryPoint. >>> at >>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:190) >>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>> at >>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:520) >>> [flink-dist_2.12-1.11.2.jar:1.11.2] >>> at >>> org.apache.flink.container.entrypoint.StandaloneApplicationClusterEntryPoint.main(StandaloneApplicationClusterEntryPoint.java:94) >>> [flink-dist_2.12-1.11.2.jar:1.11.2] >>> Caused by: org.apache.flink.util.FlinkException: Could not instantiate >>> class >>> 'org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory' >>> of type >>> 'org.apache.flink.runtime.highavailability.HighAvailabilityServicesFactory'. >>> Please make sure that this class is on your class path. >>> at >>> org.apache.flink.util.InstantiationUtil.instantiate(InstantiationUtil.java:352) >>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>> at >>> org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.loadCustomHighAvailabilityServicesFactory(HighAvailabilityServicesUtils.java:263) >>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>> at >>> org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createCustomHAServices(HighAvailabilityServicesUtils.java:246) >>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>> at >>> org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createHighAvailabilityServices(HighAvailabilityServicesUtils.java:126) >>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>> at >>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.createHaServices(ClusterEntrypoint.java:306) >>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>> at >>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.initializeServices(ClusterEntrypoint.java:269) >>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>> at >>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:211) >>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>> at >>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:172) >>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>> at >>> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30) >>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>> at >>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:171) >>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>> ... 2 more >>> Caused by: java.lang.ClassNotFoundException: >>> org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory >>> at jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source) >>> ~[?:?] >>> at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown >>> Source) ~[?:?] >>> at java.lang.ClassLoader.loadClass(Unknown Source) ~[?:?] >>> >>> On Mon, Jan 18, 2021 at 7:52 PM Yang Wang <danrtsey...@gmail.com> wrote: >>> >>>> Class >>>> "org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory" >>>> is introduced from 1.12.0 version. >>>> You could try with the latest version 1.12.1[1]. >>>> >>>> Will that jar file need to be copied under >>>>> /opt/flink/plugins/s3-fs-presto folder? >>>> >>>> Yes, you need to copy the s3 fs implementation to plugin directory. >>>> An alternative is you could set the environment to enable the plugin[1]. >>>> >>>> [1]. >>>> https://hub.docker.com/r/apache/flink/tags?page=1&ordering=last_updated&name=1.12.1 >>>> [2]. >>>> https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/docker.html#using-filesystem-plugins >>>> >>>> Best, >>>> Yang >>>> >>>> Ashish Nigam <ashnigamt...@gmail.com> 于2021年1月18日周一 下午11:15写道： >>>> >>>>> Hi, >>>>> I am not able to identify maven artifact that will have >>>>> implementation for this class >>>>> >>>>> org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory >>>>> >>>>> >>>>> I am using info in this link to test out HA implementation >>>>> >>>>> https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/ha/kubernetes_ha.html >>>>> >>>>> Please point me to the right artifact. Also, I plan to use S3 bucket >>>>> as storageDir. So, will that jar file need to be copied under >>>>> /opt/flink/plugins/s3-fs-presto folder? >>>>> >>>>> Thanks >>>>> Ashish >>>>> >>>>
