Yang, It worked finally... after adding kubernetes related config in configmap.
On Thu, Jan 21, 2021 at 7:13 PM Yang Wang <danrtsey...@gmail.com> wrote: > You could set config option "kubernetes.namespace" to your flink-conf > ConfigMap. And then > KubernetesHAService will use it to create/watch the ConfigMap. Please note > the default service > account has enough permission. > > Of course, you could also set the config option > "kubernetes.service-account" to another service account > with enough permission. > > Best, > Yang > > Ashish Nigam <ashnigamt...@gmail.com> 于2021年1月22日周五 上午3:16写道： > >> It works now. Job manager is able to start. But now, I have run into >> another issue. >> >> It seems job manager is trying to create configmap in default namespace >> and namespace/service account where I run job manager does not have access >> to configmap >> >> GET at: >> https://X.X.X.X/api/v1/namespaces/default/configmaps/dev-cluster-restserver-leader. >> Message: Forbidden!Configured service account doesn't have access. Service >> account may have been revoked. configmaps "dev-cluster-restserver-leader" >> is forbidden: User "system:serviceaccount:apps-dev:apps" cannot get >> resource "configmaps" in API group "" in the namespace "default". >> >> Is there a workaround apart from adding permission? Can leader configmap >> be created in the same namespace as Job Manager? >> >> On Wed, Jan 20, 2021 at 7:42 PM Yang Wang <danrtsey...@gmail.com> wrote: >> >>> You do not need to put flink-kubernetes_2.12-1.12.1.jar under the plugin >>> directory. Only the S3 fs jar >>> needs to be put there. >>> >>> I think I found the root cause. It seems your flink-dist is still 1.11. >>> Do you want to use the image 1.11 and enable the K8s HA at the same time? >>> I think it could not work. Because the HA related codes have been >>> refactored from release 1.12. >>> Even though you copy the flink-kubernetes_2.12-1.12.1.jar to >>> /opt/flink/lib and make the KubernetesHaServicesFactory class >>> could be resolved. It will encounter other issues after then. >>> >>> Could you please try with the latest Flink 1.12 image? >>> docker pull apache/flink:1.12.1 >>> >>> Best, >>> Yang >>> >>> Ashish Nigam <ashnigamt...@gmail.com> 于2021年1月21日周四 上午8:05写道： >>> >>>> I did move to 1.12.1 version and also ensured that docker has >>>> kubernetes jar file in the right location, i.e. >>>> /opt/flink/plugins/s3-fs-presto/flink-kubernetes_2.12-1.12.1.jar >>>> >>>> But job manager is still not able to find the class >>>> >>>> 2021-01-21 00:00:49,376 ERROR >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Could not >>>> start cluster entrypoint StandaloneApplicationClusterEntryPoint. >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypointException: Failed >>>> to initialize the cluster entrypoint >>>> StandaloneApplicationClusterEntryPoint. >>>> at >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:190) >>>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>>> at >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:520) >>>> [flink-dist_2.12-1.11.2.jar:1.11.2] >>>> at >>>> org.apache.flink.container.entrypoint.StandaloneApplicationClusterEntryPoint.main(StandaloneApplicationClusterEntryPoint.java:94) >>>> [flink-dist_2.12-1.11.2.jar:1.11.2] >>>> Caused by: org.apache.flink.util.FlinkException: Could not instantiate >>>> class >>>> 'org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory' >>>> of type >>>> 'org.apache.flink.runtime.highavailability.HighAvailabilityServicesFactory'. >>>> Please make sure that this class is on your class path. >>>> at >>>> org.apache.flink.util.InstantiationUtil.instantiate(InstantiationUtil.java:352) >>>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>>> at >>>> org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.loadCustomHighAvailabilityServicesFactory(HighAvailabilityServicesUtils.java:263) >>>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>>> at >>>> org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createCustomHAServices(HighAvailabilityServicesUtils.java:246) >>>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>>> at >>>> org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createHighAvailabilityServices(HighAvailabilityServicesUtils.java:126) >>>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>>> at >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.createHaServices(ClusterEntrypoint.java:306) >>>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>>> at >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.initializeServices(ClusterEntrypoint.java:269) >>>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>>> at >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:211) >>>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>>> at >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:172) >>>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>>> at >>>> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30) >>>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>>> at >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:171) >>>> ~[flink-dist_2.12-1.11.2.jar:1.11.2] >>>> ... 2 more >>>> Caused by: java.lang.ClassNotFoundException: >>>> org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory >>>> at jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source) >>>> ~[?:?] >>>> at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown >>>> Source) ~[?:?] >>>> at java.lang.ClassLoader.loadClass(Unknown Source) ~[?:?] >>>> >>>> On Mon, Jan 18, 2021 at 7:52 PM Yang Wang <danrtsey...@gmail.com> >>>> wrote: >>>> >>>>> Class >>>>> "org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory" >>>>> is introduced from 1.12.0 version. >>>>> You could try with the latest version 1.12.1[1]. >>>>> >>>>> Will that jar file need to be copied under >>>>>> /opt/flink/plugins/s3-fs-presto folder? >>>>> >>>>> Yes, you need to copy the s3 fs implementation to plugin directory. >>>>> An alternative is you could set the environment to enable the >>>>> plugin[1]. >>>>> >>>>> [1]. >>>>> https://hub.docker.com/r/apache/flink/tags?page=1&ordering=last_updated&name=1.12.1 >>>>> [2]. >>>>> https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/docker.html#using-filesystem-plugins >>>>> >>>>> Best, >>>>> Yang >>>>> >>>>> Ashish Nigam <ashnigamt...@gmail.com> 于2021年1月18日周一 下午11:15写道： >>>>> >>>>>> Hi, >>>>>> I am not able to identify maven artifact that will have >>>>>> implementation for this class >>>>>> >>>>>> org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory >>>>>> >>>>>> >>>>>> I am using info in this link to test out HA implementation >>>>>> >>>>>> https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/ha/kubernetes_ha.html >>>>>> >>>>>> Please point me to the right artifact. Also, I plan to use S3 bucket >>>>>> as storageDir. So, will that jar file need to be copied under >>>>>> /opt/flink/plugins/s3-fs-presto folder? >>>>>> >>>>>> Thanks >>>>>> Ashish >>>>>> >>>>>
