Yang, "... This could be done by starting a pod as the Flink client."
Do you have an example yaml you could share? On Sun, Mar 7, 2021 at 9:57 PM Yang Wang <danrtsey...@gmail.com> wrote: > I think you want to submit a Flink python job to the existing session > cluster. > Please ensure the session cluster is created with proper service exposed > type[1]. > * LoadBalancer for the cloud environment > * NodePort for self managed K8s cluster > * ClusterIP for the K8s internal submission, which means you should submit > the Flink application in the K8s cluster. This could be done by starting a > pod as the Flink client. > > [1]. > https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/native_kubernetes/#accessing-flinks-web-ui > > Best, > Yang > > Robert Cullen <cinquate...@gmail.com> 于2021年3月6日周六 上午12:43写道： > >> Trying to spin up a Python Flink instance in my Kubernetes cluster with >> this configuration ... >> >> sudo ./bin/flink run \ >> --target kubernetes-session \ >> -Dkubernetes.cluster-id=flink-python \ >> -Dkubernetes.namespace=cmdaa \ >> -Dkubernetes.container.image=cmdaa/pyflink:0.0.1 \ >> --pyModule word_count \ >> --pyFiles /opt/flink-1.12.2/examples/python/table/batch/word_count.py >> >> ... But getting this error: >> >> Traceback (most recent call last): >> File "/Users/admin/.pyenv/versions/3.6.10/lib/python3.6/runpy.py", line >> 193, in _run_module_as_main >> "__main__", mod_spec) >> File "/Users/admin/.pyenv/versions/3.6.10/lib/python3.6/runpy.py", line >> 85, in _run_code >> exec(code, run_globals) >> File >> "/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/pyflink/074c581c-5b39-4b07-ac84-55a86c46f9eb/affe3559-5364-43f1-93ef-b7f6cd9a2f77/word_count.py", >> line 80, in <module> >> word_count() >> File >> "/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/pyflink/074c581c-5b39-4b07-ac84-55a86c46f9eb/affe3559-5364-43f1-93ef-b7f6cd9a2f77/word_count.py", >> line 74, in word_count >> t_env.execute("word_count") >> File >> "/opt/flink-1.12.2/opt/python/pyflink.zip/pyflink/table/table_environment.py", >> line 1276, in execute >> File >> "/opt/flink-1.12.2/opt/python/py4j-0.10.8.1-src.zip/py4j/java_gateway.py", >> line 1286, in __call__ >> File >> "/opt/flink-1.12.2/opt/python/pyflink.zip/pyflink/util/exceptions.py", line >> 147, in deco >> File >> "/opt/flink-1.12.2/opt/python/py4j-0.10.8.1-src.zip/py4j/protocol.py", line >> 328, in get_return_value >> py4j.protocol.Py4JJavaError: An error occurred while calling o2.execute. >> : java.lang.RuntimeException: >> org.apache.flink.client.deployment.ClusterRetrieveException: Could not get >> the rest endpoint of flink-python >> at >> org.apache.flink.kubernetes.KubernetesClusterDescriptor.lambda$createClusterClientProvider$0(KubernetesClusterDescriptor.java:102) >> at >> org.apache.flink.kubernetes.KubernetesClusterDescriptor.retrieve(KubernetesClusterDescriptor.java:145) >> at >> org.apache.flink.kubernetes.KubernetesClusterDescriptor.retrieve(KubernetesClusterDescriptor.java:66) >> at >> org.apache.flink.client.deployment.executors.AbstractSessionClusterExecutor.execute(AbstractSessionClusterExecutor.java:75) >> at >> org.apache.flink.table.api.internal.BatchTableEnvImpl.executePipeline(BatchTableEnvImpl.scala:347) >> at >> org.apache.flink.table.api.internal.BatchTableEnvImpl.execute(BatchTableEnvImpl.scala:317) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:498) >> at >> org.apache.flink.api.python.shaded.py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) >> at >> org.apache.flink.api.python.shaded.py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) >> at >> org.apache.flink.api.python.shaded.py4j.Gateway.invoke(Gateway.java:282) >> at >> org.apache.flink.api.python.shaded.py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) >> at >> org.apache.flink.api.python.shaded.py4j.commands.CallCommand.execute(CallCommand.java:79) >> at >> org.apache.flink.api.python.shaded.py4j.GatewayConnection.run(GatewayConnection.java:238) >> at java.lang.Thread.run(Thread.java:748) >> Caused by: org.apache.flink.client.deployment.ClusterRetrieveException: >> Could not get the rest endpoint of flink-python >> ... 17 more >> >> org.apache.flink.client.program.ProgramAbortException >> at >> org.apache.flink.client.python.PythonDriver.main(PythonDriver.java:124) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:498) >> at >> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:349) >> at >> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:219) >> at >> org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114) >> at >> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:812) >> at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:246) >> at >> org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1054) >> at >> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132) >> at >> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28) >> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132) >> >> -- >> Robert Cullen >> 240-475-4490 >> > -- Robert Cullen 240-475-4490
