Hi all,
I am trying to run *Python* beam pipeline on a Spark cluster. Since workers are running on separate nodes, I am using "
PROCESS
" for "evironment_type" in pipeline options, but I couldn't find any documentation on what "command" I should pass to "environment_config"
to run on the worker, so executor can be able to communicate with.Can someone help me on that?