You should be able to provide extra_args to the expansion service:
```
--defaultEnvironmentType=PROCESS
--defaultEnvironmentConfig={"command":"/opt/apache/beam_java/boot"}
```
I am also running the Xlang pipeline in flink k8s cluster. After setting
the defaultEnvironmentType to PROCESS, I don't need to use DinD or DooD at
all.
You can also find my full settings in:
https://stackoverflow.com/a/74035035/19259600
Sincerely,
Lydian Lee
On Tue, Oct 11, 2022 at 11:34 AM Xiao Ma <xia...@geotab.com> wrote:
> The worker pool means `starting a java or python sdk`, to accept the java
> or python pipeline running. For example, to execute python pipeline, we
> have to start a python worker pool with `--worker_pool` arguments. For the
> Java code, besides the docker mode (default one), do we have other better
> ways to start a java worker pool?
>
> For now, our flink cluster is running on the k8s. If we choose the default
> sdk harness mode (docker), we will have the docker (java sdk harness) in
> docker (flink-taskmanager). So, what we are doing is to call
> org.apache.beam.fn.harness.ExternalWorkerService class with pipeline
> options as environment variables and fixed two small issues in the
> FnHarness class to make sure the java sdk harness can run smoothly.
>
> Thank you.
> *Mark Ma*
>
>
> On Tue, Oct 11, 2022 at 12:46 PM Alexey Romanenko <
> aromanenko....@gmail.com> wrote:
>
>> I’m not sure that I get it correctly. What do you mean by “worker pool”
>> in your case?
>>
>> —
>> Alexey
>>
>> On 8 Oct 2022, at 03:24, Xiao Ma <xia...@geotab.com> wrote:
>>
>> Hello,
>>
>> I would like to run a pipeline with Java as the main language and python
>> transformation embedded. The beam pipeline is running on the flink cluster.
>> Currently, I can run it with a taskmanager + java worker pool and a python
>> worker pool. Could I ask if there is a way to run the java code on the task
>> manager directly and keep the python worker pool?
>>
>> Current: taskmanager + java worker pool + python worker pool
>> Desired: taskmanager + python worker pool
>>
>> Thank you very much.
>>
>> *Mark Ma*
>>
>>
>>
