Thanks Zhilong for taking a look! Primarily I am looking for ways to start it through a REST api [1] . For Java, I pass along entry-class pointing to a main class in the jar which constructs the job graph and triggers the execute(). How do we accomplish this for pyflink jobs? The closest I encountered is PythonDriver [2]
[1] https://nightlies.apache.org/flink/flink-docs-master/docs/ops/rest_api/#jars-jarid-run [2] https://github.com/apache/flink/blob/release-1.13/flink-python/src/main/java/org/apache/flink/client/python/PythonDriver.java On Sat, Mar 5, 2022 at 10:37 AM Zhilong Hong <zhlongh...@gmail.com> wrote: > Hi, Aryan: > > You could refer to the official docs [1] for how to submit PyFlink jobs. > > $ ./bin/flink run \ > --target yarn-per-job > --python examples/python/table/word_count.py > > With this command you can submit a per-job application to YARN. The docs > [2] and [3] describe how to submit jobs to the YARN session and the > Kubernetes session. > > $ ./bin/flink run -t yarn-session \ > -Dyarn.application.id=application_XXXX_YY \ > --python examples/python/table/word_count.py > > $ ./bin/flink run -t kubernetes-session \ > -Dkubernetes.cluster-id=my-first-flink-cluster \ > --python examples/python/table/word_count.py > > Best, > Zhilong > > [1] > https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/cli/#submitting-pyflink-jobs > [2] > https://nightlies.apache.org/flink/flink-docs-master/zh/docs/deployment/resource-providers/yarn/#session-mode > [3] > https://nightlies.apache.org/flink/flink-docs-master/zh/docs/deployment/resource-providers/native_kubernetes/#session-mode > > On Sun, Mar 6, 2022 at 2:08 AM aryan m <maryan8...@gmail.com> wrote: > >> Hi ! >> In a session cluster, what is the recommended way to submit a pyFlink >> job via REST ? I am on Flink 1.13 and my job code is available at >> web.upload.dir >> <https://ci.apache.org/projects/flink/flink-docs-stable/ops/config.html#web-upload-dir> >> . >> >> Appreciate the help! >> >> >>
