Hello, Can I run Beam Python pipelines with Flink deployed on Kubernetes?
I have successfully managed to run a Beam Python pipeline using the Portable runner and the job service pointing to a local Flink server running as Docker containers. I was able to achieve that mounting the Docker socket in my Flink container and running Flink as root, so the class DockerEnvironmentFactory can create the Python harness container. Unfortunately, I can't use the same solution when Flink is running in Kubernetes. Moreover, I don't want to create the Python harness container using the Docker command from my pods. I noticed there is an implementation called ExternalEnvironmentFactory, but I am not sure how to use it. Is there a way to deploy a side container and use a different factory to run the Python harness process? What is the correct approach? Thanks, Andrea [10x]<https://www.10xbanking.com/> 10x Banking Please consider the environment before printing this e-mail ********************************************************************************************************************************************************** This e-mail, including any attachments, is confidential. If you are not the intended recipient, you may not copy, forward, disclose or use any part of it. If you have received this e-mail in error, please notify the sender immediately by return e-mail and delete this message and any copies and attachments. 10x Future Technologies Services Limited is registered in England & Wales Registered Number: 11262659 Registered Address: 5 Howick Place, Victoria, London, SW1P 1WG
