Re: Specifying dataflow template location with Apache beam Python SDK

Thanks all. Yes I was under a misunderstanding that we can directly use one of these templates as a base without creating a custom template. Thanks for clarifying it for me. Regards, Sumit Desai On Mon, 18 Dec 2023, 10:34 pm Bruno Volpato via user, wrote: > Right, there's some misunderstanding

Re: Specifying dataflow template location with Apache beam Python SDK

Right, there's some misunderstanding here, so Bartosz and XQ's inputs are correct. Just want to add that the template_location parameter is the GCS path that you want to store your template on, and not the image reference of the base image. The GCR path that you are trying to use is used in the

Re: Specifying dataflow template location with Apache beam Python SDK

https://github.com/google/dataflow-ml-starter/tree/main?tab=readme-ov-file#run-the-beam-pipeline-with-dataflow-flex-templates has a full example about how to create your own flex template. FYI. On Mon, Dec 18, 2023 at 5:01 AM Bartosz Zabłocki via user < user@beam.apache.org> wrote: > Hi Sumit, >

Re: Specifying dataflow template location with Apache beam Python SDK

Hi Sumit, could you elaborate a little bit more on what you are trying to achieve with the templates? As far as I know, these base Docker images serve as base images for your own custom templates. If you want to use an existing template, you can use one of these:

Specifying dataflow template location with Apache beam Python SDK

I am creating an Apache beam pipeline using Python SDK.I want to use some standard template of dataflow (this one ). But when I am specifying it using 'template_location' key while