You should be able to provide extra_args to the expansion service: ``` --defaultEnvironmentType=PROCESS --defaultEnvironmentConfig={"command":"/opt/apache/beam_java/boot"}
``` I am also running the Xlang pipeline in flink k8s cluster. After setting the defaultEnvironmentType to PROCESS, I don't need to use DinD or DooD at all. You can also find my full settings in: https://stackoverflow.com/a/74035035/19259600 Sincerely, Lydian Lee On Tue, Oct 11, 2022 at 11:34 AM Xiao Ma <xia...@geotab.com> wrote: > The worker pool means `starting a java or python sdk`, to accept the java > or python pipeline running. For example, to execute python pipeline, we > have to start a python worker pool with `--worker_pool` arguments. For the > Java code, besides the docker mode (default one), do we have other better > ways to start a java worker pool? > > For now, our flink cluster is running on the k8s. If we choose the default > sdk harness mode (docker), we will have the docker (java sdk harness) in > docker (flink-taskmanager). So, what we are doing is to call > org.apache.beam.fn.harness.ExternalWorkerService class with pipeline > options as environment variables and fixed two small issues in the > FnHarness class to make sure the java sdk harness can run smoothly. > > Thank you. > *Mark Ma* > > > On Tue, Oct 11, 2022 at 12:46 PM Alexey Romanenko < > aromanenko....@gmail.com> wrote: > >> I’m not sure that I get it correctly. What do you mean by “worker pool” >> in your case? >> >> — >> Alexey >> >> On 8 Oct 2022, at 03:24, Xiao Ma <xia...@geotab.com> wrote: >> >> Hello, >> >> I would like to run a pipeline with Java as the main language and python >> transformation embedded. The beam pipeline is running on the flink cluster. >> Currently, I can run it with a taskmanager + java worker pool and a python >> worker pool. Could I ask if there is a way to run the java code on the task >> manager directly and keep the python worker pool? >> >> Current: taskmanager + java worker pool + python worker pool >> Desired: taskmanager + python worker pool >> >> Thank you very much. >> >> *Mark Ma* >> >> >>