Re: [design] A streaming Fn API runner for Python

Thanks for picking this up again Pablo, I wrote some small comments concerning the TestStream. On Tue, Oct 15, 2019 at 4:42 PM Robert Bradshaw wrote: > Very excited to see this! I've added some comments to the doc. > > On Tue, Oct 15, 2019 at 3:43 PM Pablo Estrada wrote: > >> I've just been

Re: [design] A streaming Fn API runner for Python

Very excited to see this! I've added some comments to the doc. On Tue, Oct 15, 2019 at 3:43 PM Pablo Estrada wrote: > I've just been informed that access wasn't open. I've since opened access > to it. > Thanks > -P. > > On Tue, Oct 15, 2019 at 2:10 PM Pablo Estrada wrote: > >> Hello all, >> I

Re: [design] A streaming Fn API runner for Python

I've just been informed that access wasn't open. I've since opened access to it. Thanks -P. On Tue, Oct 15, 2019 at 2:10 PM Pablo Estrada wrote: > Hello all, > I am planning to work on removing the old BundleBasedDirectRunner, and > expand the FnApiRunner to work on streaming as well as batch.

Re: Outreachy Applicant

Hi - sorry for the delay - I would definitely be interested in working on that! On Tue, Oct 15, 2019, 11:32 PM Rui Wang wrote: > Hi Omshi, > > Do you want to take https://jira.apache.org/jira/browse/BEAM-8406? If you > are interested in, we can discuss the details in that JIRA. > > > -Rui > >

Re: [PROPOSAL] Preparing for Beam 2.17.0 release

Thanks Mikhail! +1 On Tue, Oct 15, 2019 at 11:38 AM Mikhail Gryzykhin wrote: > Hi all, > > Beam 2.17 release branch cut is scheduled on Oct 23 according to the > release calendar [1]. I would like to volunteer myself to do this release. > The plan is to cut the branch on that date, and

[PROPOSAL] Preparing for Beam 2.17.0 release

Hi all, Beam 2.17 release branch cut is scheduled on Oct 23 according to the release calendar [1]. I would like to volunteer myself to do this release. The plan is to cut the branch on that date, and cherrypick release-blocking fixes afterwards if any. If you have release blocking issues for

Re: Gauge Metrics

There are cases where we may want to update a Gauge frequently. Especially if it's from a single worker. The example I always think of is progress in a Kafka partition. This is consumed by a single worker at a time, and updated by a single worker at a time - and frequently updated too. Max, do

Re: Outreachy Applicant

Hi Omshi, Do you want to take https://jira.apache.org/jira/browse/BEAM-8406? If you are interested in, we can discuss the details in that JIRA. -Rui On Tue, Oct 15, 2019 at 10:39 AM Omshi Samal wrote: > Hi Rui, > > No problem and thanks! > > Best, > Omshi > > On Tue, Oct 15, 2019 at 10:16 PM

Re: Outreachy Applicant

Hi Rui, No problem and thanks! Best, Omshi On Tue, Oct 15, 2019 at 10:16 PM Rui Wang wrote: > Sorry Omshi. I waited a bit for someone to add your JIRA id. But we can > start without it. I will find a starter task and send it to you. > > > -Rui > > On Tue, Oct 15, 2019 at 8:54 AM Abhishek

Re: Gauge Metrics

Would you elaborate on what you are expecting the behaviour to look like? Ideally your runner would export gauges at a periodic interval. The design of gauge is inherently unable to handle multiple updates to it around the same time. Consider the case of multiple machines reporting the gauge at

Gauge Metrics

Hi, While adding metrics for the Python state cache [1], I was wondering about the story of Gauges in Beam. It seems like we only keep a value at a time and use a combiner [2] that replaces an old, possibly not reported gauge result, with a newer gauge result based on their timestamps. This

Re: Outreachy Applicant

Sorry Omshi. I waited a bit for someone to add your JIRA id. But we can start without it. I will find a starter task and send it to you. -Rui On Tue, Oct 15, 2019 at 8:54 AM Abhishek Uniyal wrote: > Did you get any response? > > On 2019/10/11 20:53:25, Omshi Samal wrote: > > Hi Rui, > > >

Re: Outreachy Applicant

Did you get any response? On 2019/10/11 20:53:25, Omshi Samal wrote: > Hi Rui, > > It would be great if I could get some advice on where to get started - I'm > a bit new to this. > > Thank you! :) > > Best, > Omshi > > > > > On Wed, Oct 9, 2019, 11:24 PM Omshi Samal wrote: > > > Hello!

[PROPOSAL] KeyedPCollection design draft

Hi, as a follow-up from a discussion [1] about reiterations over result from a GBK I have put together a very basic draft of a design document [2] which could be a solution to the problem (and maybe several others). I'd be vary happy to hear about what do people think about the basic