Hello I created an example of running python code on Apache Beam + Spark + Kubernetes at https://github.com/cometta/python-apache-beam-spark because I unable to find any step-by-step guide on how to do this for the past few weeks. I hope it will be useful for other that also looking for similar implementation. Have a good day. Contributors are welcome to further improve the code.
Thank you