Re: [PROPOSAL] Preparing for Beam 2.19 release

Comment: thank you for keeping on the schedule! And comment for owners of bugs: if you have a bug targeting this release and it is not already in fixed or critical urgency, please remove the Fix Version. Here is a search to help identify: https://issues.apache.org/jira/browse/BEAM-8989?jql=project

Re: [DISCUSS] BIP reloaded

It does seem that the community would find this useful. I agree with Robert that it has downsides and it is not appropriate all the time. We added https://beam.apache.org/roadmap/ a little while ago. I think that the granularity of a BIP is about the same as the granularity of what we would want t

Re: Dropping late data in DirectRunner

That is true. And as I have written at the beginning, I came across this behavior when I was comparing behavior of ordered (@RequiresTimeSortedInput) stateful DoFn to unordered. I think it might be fine that both of these have different dropping semantics (ordered case simply has to drop every

Re: [PROPOSAL] Preparing for Beam 2.19 release

Thanks Boyuan for taking care of 2.19 release! -Rui On Wed, Jan 8, 2020 at 7:18 PM Boyuan Zhang wrote: > Hi all, > > The next (2.19) release branch cut is scheduled forJan 15th, according to > the calendar >

[PROPOSAL] Preparing for Beam 2.19 release

Hi all, The next (2.19) release branch cut is scheduled forJan 15th, according to the calendar . I would like to volunteer myself to do this release. The plan is to cut the branch on that date,

Re: [PROPOSAL] gRPC Vendor Release

Hi all, The PR of [1] which bump the version of gRPC to 1.26.0 has been merged. Would be great to push forward the gRPC Vendor Release process now. And I am appreciate if someone of committer could be the release manager to help with the release process. Best, Jincheng [1] https://github.com/a

Re: beam_PostCommit_Java_Nexmark_Flink are failing continuously

git bisect on commits. this issue is fixed by https://github.com/apache/beam/commit/4af507f567605723af940c2af3b2912d0948adf5 On Thu, Oct 17, 2019 at 3:14 AM Maximilian Michels wrote: > Definitely not expected :) Thanks for investigating Kai. > > On 16.10.19 20:29, Kai Jiang wrote: > > I checked

Re: Dropping late data in DirectRunner

But there's no ordering inside a window. A stateful DoFn can see the input elements inside of a window in any order at all. This is another reason it's best to think of time spatially - as another data dimension - rather than like normal processing time. On Wed, Jan 8, 2020 at 2:26 AM Jan Lukavský

Re: Request for review of PR [Beam-8564]

Apologies for a late reply. The PR has been updated now. Would love to get your thoughts/suggestions. On Tue, Jan 7, 2020 at 12:36 AM Luke Cwik wrote: > Have you had a chance to update the PR? > > On Mon, Dec 30, 2019 at 5:00 AM Amogh Tiwari wrote: > >> Hi Luke, >> >> We have gone through sheve

Re: Custom window invariants and

Hmm. I've seen this manifest in some other tweaked versions of Sessions. Your invariants are right. In fact, the Nexmark queries have auctions that truncate in a similar way. This prompted https://issues.apache.org/jira/browse/BEAM-654. I think we have not really nailed down the right spec for mer

Re: Please comment on draft comms strategy by Oct 16

Hi everyone, I'm writing to send an update about the communication strategy for Beam. In a nutshell, I have 3 proposed changes (copied from the md file here: https://github.com/macruzbar/beam/blob/master/Communication-strategy-DRAFT.md ). While all the channels are connected to a specific function

Re: Becoming a Contributor

Welcome, someone has already added you as a contributor to the Beam JIRA project. On Wed, Jan 8, 2020 at 10:59 AM Dave Hughes wrote: > Hey, > > My name is Dave. I plan on working on some issues on Beam. Can I be added > as a contributor in the Beam issue tracker? My jira username is dwhu. > > Th

[Design Proposal] DataStore SQL Connector

Hello everyone! I have written up a proposal [1] for a DataStore SQL connector. I would love to hear comments and suggestions from the Beam dev community! A quick summary: DataStore [2] is a NoSQL database with a dynamic schema, where entities (documents) are stored in Kinds (databases). Each ent

Becoming a Contributor

Hey, My name is Dave. I plan on working on some issues on Beam. Can I be added as a contributor in the Beam issue tracker? My jira username is dwhu. Thanks, Dave

Re: [DISCUSS] Python static type checkers

I am fine with adding this as a linter. I would not want to block either (let alone both) until we have some experience with them. Hopefully, if our code is clean and correctly typed, it should pass both. Where it doesn't, I'm hopeful that the looseness provided by gradual typing will allow us to h

Re: Unset / delete Timers

Can you explain the use case? You can already overwrite a timer with a new time. Do you have a specific use case where you want to delete, not overwrite, a timer? I don't think this would be terribly hard to implement, but I'm curious about the use case. Reuven On Tue, Jan 7, 2020 at 8:11 PM Rez

Re: Jenkins jobs not running for my PR 10438

Is there anything specific I need to do? The tests won't start in https://github.com/apache/beam/pull/9903 Thanks, Max On 08.01.20 14:38, Rehman Murad Ali wrote: Thank you Ismaël and Alexey. * * * * * * *Thanks & Regards* *Rehman Murad Ali* Software Engineer Mobile: +92 3452076766 S

Re: Unset / delete Timers

The model already requires functionality for deleting timers. If a timer is set more than once, the last invocation must override all the other. In Flink deleting timers is as expensive as setting them. -Max On 08.01.20 05:10, Reza Rokni wrote: Hi, Was exploring the ability to add unset / re

Re: Jenkins jobs not running for my PR 10438

Thank you Ismaël and Alexey. *Thanks & Regards* *Rehman Murad Ali* Software Engineer Mobile: +92 3452076766 Skype: rehman.muradali On Wed, Jan 8, 2020 at 6:03 PM Ismaël Mejía wrote: > It worked for me, it is running now, not sure if it was me or just a > delayed queue I noticed in other

Re: Jenkins jobs not running for my PR 10438

It worked for me, it is running now, not sure if it was me or just a delayed queue I noticed in other PRs that the jobs take longer to start than before. On Wed, Jan 8, 2020 at 1:51 PM Alexey Romanenko wrote: > I tried to retest it but seems that even for committers it doesn’t always > work... >

Re: Jenkins jobs not running for my PR 10438

I tried to retest it but seems that even for committers it doesn’t always work... > On 8 Jan 2020, at 12:48, Rehman Murad Ali > wrote: > > Hi Committer, > > > I appreciate if you could trigger jobs for > https://github.com/apache/beam/pull/9677 >

Re: Jenkins jobs not running for my PR 10438

Hi Committer, I appreciate if you could trigger jobs for https://github.com/apache/beam/pull/9677 *Thanks & Regards* *Rehman Murad Ali* Software Engineer Mobile: +92 3452076766 Skype: rehman.muradali On Wed, Jan 8, 2020 at 2:27 PM Michał Walenia wrote: > Hi, I also think that a whitelist

Re: Dropping late data in DirectRunner

Hi Luke and Kenn, I agree, my mental model fits this as well. But still, even in the presence of simultaneuos existence of all windows at once - GBK and stateful DoFns differ in the way they handle time *inside* each window (and I'm as well leaving merging windows outside, partly because they

Re: Jenkins jobs not running for my PR 10438

Hi, I also think that a whitelist would be a good solution, in terms of security and trust it would be between no restrictions and committers only being able to run jobs. How would we proceed with this? Does it need to be voted on? On Wed, Jan 8, 2020 at 12:50 AM Mark Liu wrote: > Thank you Kenn