Re: KafkaIO committing semantics

Thank you very much for your explanation! commitOffsetsInFinalize() -> although checkpointing depends on the runner is it not configurable in a connector implementation? Basically, I want to understand how this can be done with a new IO connector implementation, esp. with the new SDF API. If I

Problems with BigQuery Sinks - Error encountered during execution. Retrying may solve the problem

Hi all, We've been using Cloud Dataflow (currently on Apache Beam Python 3.7 SDK 2.22.0) since about January and are running into a problem recently with BQ Sinks. Essentially what seems to be happening is our (batch) dataflow pipelines fail because of a failure in one of the load jobs that bring

Ability to link to "latest" of python docs

Hey folks - I'm wondering if there's a way to link to the latest SDK version of the Python documentation. I see that if I go here , it lists all the available documented SDK versions. But it'd be really nice to go to a link like " https://beam.apache.org/re

Re: Ability to link to "latest" of python docs

+dev Lynn, Seems totally doable. If others don't speak up with a good way to do this (or in opposition), I'm sure we can sort something out to accomplish this (will dig into intersphinx mapping tomorrow). Cheers, Austin On Tue, Sep 8, 2020, 5:19 PM Lynn Root wrote: > Hey folks - > > I'm

FlinkRunner Graphite metrics

Hi, I have a streaming pipeline that reads messages from Kafka (KafkaIO.read), transforms, and then writes to Kafka (KafkaIO.write). It is using Apache Beam 2.23.0 and Flink 1.10.1. The pipeline is working as expected and I am trying to collect some metrics through Graphite so I did the following