Deployment of beam pipelines on flink cluster

Hi all, We are currently using beam to create a few pipelines, and then deploy them on our on-prem Flink cluster. we have a few questions regarding the automation of the pipelines deployment: We have beam running as a k8s pod which starts a java process for each pipeline and has the flinkrun

Typehint a doFn with multiple outputs in python

Hi, I can't seem to find in the documentation how to add type hints to a doFn doing multiple tagged output. My function looks like this: values = beam.FlatMap( parse_sessions.sessions_unnest ).with_outputs( OUTPU1, OUTPU2, OUTPU3 ) I'm looking for a way to have a correct typehint later w

[Question] How to provide S3 endpoint-url for --artifacts_dir="s3://..."

Hello, I’m new to Beam and trying to submit python WordCount job to a Standalone Flink Cluster. I submit the job from the same node where my Flink JobManager is running. python -m apache_beam.examples.wordcount --input /input.txt --output count --runner FlinkRunner --flink_master=localhost:808