Re: Checkpoints timing out upgrading from Beam version 2.29 with Flink 1.12 to Beam 2.38 and Flink 1.14

2022-07-26 Thread Kenneth Knowles
Bumping this and adding +John Casey who knows about KafkaIO and unbounded sources, though probably less about the FlinkRunner. It seems you have isolated it to the Flink translation logic. I'm not sure who would be the best expert to evaluate if that logic is still OK. Kenn On Wed, Jun 29, 2022

Re: [CDAP IO] SparkReceiverIO integration testing

2022-07-26 Thread Chamikara Jayalath via dev
On Tue, Jul 26, 2022 at 11:24 AM Elizaveta Lomteva < elizaveta.lomt...@akvelon.com> wrote: > Hi, community! > Our team has prepared SparkReceiverIO Read via SDF PR [1]. We have started > working on integration tests for the SparkReceiverIO connector which will > allow to read data from Custom Spar

Re: BigTable reader for Python?

2022-07-26 Thread Sachin Agarwal via dev
On Tue, Jul 26, 2022 at 6:12 PM Chamikara Jayalath via dev < dev@beam.apache.org> wrote: > > > On Mon, Jul 25, 2022 at 12:53 PM Lina Mårtensson via dev < > dev@beam.apache.org> wrote: > >> Hi dev, >> >> We're starting to incorporate BigTable in our stack and I've delighted >> my co-workers with ho

Re: BigTable reader for Python?

2022-07-26 Thread Chamikara Jayalath via dev
On Mon, Jul 25, 2022 at 12:53 PM Lina Mårtensson via dev < dev@beam.apache.org> wrote: > Hi dev, > > We're starting to incorporate BigTable in our stack and I've delighted > my co-workers with how easy it was to create some BigTables with > Beam... but there doesn't appear to be a reader for BigTa

[CDAP IO] SparkReceiverIO integration testing

2022-07-26 Thread Elizaveta Lomteva
Hi, community! Our team has prepared SparkReceiverIO Read via SDF PR [1]. We have started working on integration tests for the SparkReceiverIO connector which will allow to read data from Custom Spark Receivers in Apache Beam pipeline. A general Apache Beam recommendation is to implement “ write

Beam High Priority Issue Report

2022-07-26 Thread beamactions
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/22440 [Bug]: Python Batch