Yes, agreed. This could be better streamlined. If you wanna help with this, then feel free to open a JIRA issue for it.
Cheers, Till On Wed, Feb 17, 2021 at 11:37 AM Manas Kale <manaskal...@gmail.com> wrote: > Hi Till, > Oh I see... I managed to do what you said using a bunch of docker exec > commands. However, I think this solution is quite hacky and could be > improved by providing some simple command to submit jobs using the Flink > runtime within the docker images. I believe this will achieve full > containerization - the host system is not at all expected to have the Flink > runtime, everything is within Docker images. > > Thanks a lot! > > On Tue, Feb 16, 2021 at 6:08 PM Till Rohrmann <trohrm...@apache.org> > wrote: > >> Hi Manas, >> >> I think the documentation assumes that you first start a session cluster >> and then submit jobs from outside the Docker images. If your jobs are >> included in the Docker image, then you could log into the master process >> and start the jobs from within the Docker image. >> >> Cheers, >> Till >> >> On Tue, Feb 16, 2021 at 1:00 PM Manas Kale <manaskal...@gmail.com> wrote: >> >>> Hi, >>> >>> I have a project that is a set of 6 jobs out of which 4 are written in >>> Java and 2 are written in pyFlink. I want to dockerize these so that all 6 >>> can be run in a single Flink session cluster. >>> >>> I have been able to successfully set up the JobManager and TaskManager >>> containers as per [1] after creating a custom Docker image that has Python. >>> For the last step, the guide asks us to submit the job using a local >>> distribution of Flink: >>> >>> $ ./bin/flink run ./examples/streaming/TopSpeedWindowing.jar >>> >>> I am probably missing something here because I have the following >>> questions: >>> Why do I need to use a local distribution to submit a job? >>> Why can't I use the Flink distribution that already exists within the >>> images? >>> How do I submit a job using the Docker image's distribution? >>> >>> >>> [1] >>> https://ci.apache.org/projects/flink/flink-docs-stable/deployment/resource-providers/standalone/docker.html#starting-a-session-cluster-on-docker >>> >>>
