Hi Fabian, thanks for starting this discussion. In general I would be a bit hesitant to build upon Flink's web UI submission because it suffers from a couple of drawbacks.
1) The web UI submission only supports single job applications. 2) The JobGraph is generated from within the web UI Netty thread. Hence, if the user code blocks, then this can make the web UI unresponsive. 3) Uploaded jars are not persisted. Hence, if a JobManager failover occurs between uploading and running the job, then you might have lost the uploaded jars. The reason for some of these problems is that the feature was actually implemented for some conference and almost remained untouched ever since. Building more functionality on top of it will mean that it will be harder to remove in the future. Cheers, Till On Tue, Dec 8, 2020 at 12:00 PM Fabian Paul <fabianp...@data-artisans.com> wrote: > Hi all, > > Currently, the most convenient way of programmatically submitting a job to > a running session cluster is using Flink’s RestClusterClient. > Unfortunately, it is only supported, as of now, to submit a job graph.[1] > To construct a job graph from a jar file, additional Flink dependencies are > required, which is not ideal. > > It is also possible to directly use the Flink rest API and first upload > the jar file via /jars/upload[2] and then run it via /jar/:jarId/run[3]. It > has the downside that it is impossible to set a Flink execution > configuration, and it will always take the underlying session cluster > configuration. > > I know changing the ClusterClient has already been discussed. It would > involve a lot of effort, so what do you think of making the jar execution > more prominent via the rest endpoint by adding the option to pass an > execution configuration? > > Best, > Fabian > > [1] > https://github.com/apache/flink/blob/65cd385d7de504a946b17193aceea73b3c8e78a8/flink-clients/src/main/java/org/apache/flink/client/program/ClusterClient.java#L95 > [2] > https://github.com/apache/flink/blob/c2972b6e336cc3b3a6cbd22c69a6710dab5246e6/flink-container/src/main/java/org/apache/flink/container/entrypoint/StandaloneApplicationClusterConfigurationParserFactory.java#L56 > < > https://github.com/apache/flink/blob/c2972b6e336cc3b3a6cbd22c69a6710dab5246e6/flink-container/src/main/java/org/apache/flink/container/entrypoint/StandaloneApplicationClusterConfigurationParserFactory.java#L56 > > > [3] > https://github.com/apache/flink/blob/c2972b6e336cc3b3a6cbd22c69a6710dab5246e6/flink-container/src/main/java/org/apache/flink/container/entrypoint/StandaloneApplicationClusterConfigurationParserFactory.java#L56
