[RESULT] [VOTE] Beam's Mascot will be the Firefly (Lampyridae)

I am happy to announce that this vote has passed, with 20 approving +1 votes, 5 of which are binding PMC votes. Beam's Mascot is the Firefly! Kenn On Fri, Jan 3, 2020 at 9:31 PM Kenneth Knowles wrote: > +1 (binding) > > On Tue, Dec 17, 2019 at 12:30 PM Leonardo Miguel < >

Re: [VOTE] Beam's Mascot will be the Firefly (Lampyridae)

+1 (binding) On Tue, Dec 17, 2019 at 12:30 PM Leonardo Miguel < leonardo.mig...@arquivei.com.br> wrote: > +1 > > Em sex., 13 de dez. de 2019 às 01:58, Kenneth Knowles > escreveu: > >> Please vote on the proposal for Beam's mascot to be the Firefly. This >> encompasses the Lampyridae family of

Re: Dropping late data in DirectRunner

I do agree that the direct runner doesn't drop late data arriving at a stateful DoFn (I just tested as well). However, I believe this is consistent with other runners. I'm fairly certain (at least last time I checked) that at least Dataflow will also only drop late data at GBK operations, and

Re: Dropping late data in DirectRunner

I did write a test that tested if data is dropped in a plain stateful DoFn. I did this as part of validating that PR [1] didn't drop more data when using @RequiresTimeSortedInput than it would without this annotation. This test failed and I didn't commit it, yet. The test was basically as

Re: Dropping late data in DirectRunner

Did you write such a @Category(ValidatesRunner.class) test? I believe the Java direct runner does drop late data, for both GBK and stateful ParDo. Stateful ParDo is implemented on top of GBK:

Re: Edit access to Wiki

Thank you! On Fri, Jan 3, 2020 at 10:39 AM Luke Cwik wrote: > I have added you. Happy editing. > > On Fri, Jan 3, 2020 at 10:31 AM Kirill Kozlov > wrote: > >> Hello everyone! >> >> I was hoping to add a design doc for SQL push-down [1] to the Wiki page >> [2], but I need edit access. >> What

Re: Dropping late data in DirectRunner

Yes, the non-reliability of late data dropping in distributed runner is understood. But this is even where DirectRunner can play its role, because only there it is actually possible to emulate and test specific watermark conditions. Question regarding this for the java DirectRunner - should we

Re: Dropping late data in DirectRunner

I agree, in fact we just recently enabled late data dropping to the direct runner in Python to be able to develop better tests for Dataflow. It should be noted, however, that in a distributed runner (absent the quiessence of TestStream) that one can't *count* on late data being dropped at a

Re: Edit access to Wiki

I have added you. Happy editing. On Fri, Jan 3, 2020 at 10:31 AM Kirill Kozlov wrote: > Hello everyone! > > I was hoping to add a design doc for SQL push-down [1] to the Wiki page > [2], but I need edit access. > What is the process for obtaining edit access? > My wiki username is: Kirill

Edit access to Wiki

Hello everyone! I was hoping to add a design doc for SQL push-down [1] to the Wiki page [2], but I need edit access. What is the process for obtaining edit access? My wiki username is: Kirill Kozlov [1] https://docs.google.com/document/d/1-ysD7U7qF3MAmSfkbXZO_5PLJBevAL9bktlLCerd_jE/edit [2]

Re: Dropping late data in DirectRunner

I agree that the DirectRunner should drop late data. Late data dropping is optional but the DirectRunner is used by many for testing and we should have the same behaviour they would get on other runners or users may be surprised. On Fri, Jan 3, 2020 at 3:33 AM Jan Lukavský wrote: > Hi, > > I

Re: Contributor permission for Beam Jira tickets

Done, welcome! On Fri, Jan 3, 2020 at 1:56 AM Xia Bingfeng wrote: > Hi Ismaël, > > My JIRA id is xiabingfeng > > > On Thu, Jan 2, 2020 at 4:37 PM Ismaël Mejía wrote: > >> Hello, What is your JIRA id? >> >> >> On Fri, Jan 3, 2020 at 12:38 AM Xia Bingfeng >> wrote: >> >>> Hi, >>> >>> Can

Dropping late data in DirectRunner

Hi, I just found out that DirectRunner is apparently not using LateDataDroppingDoFnRunner, which means that it doesn't drop late data in cases where there is no GBK operation involved (dropping in GBK seems to be correct). There is apparently no @Category(ValidatesRunner) test for that

Re: [ANNOUNCE] New committer: Kasia Kucharczyk

Congrats Kasia, good job! On Fri, Jan 3, 2020 at 8:22 AM Michał Walenia wrote: > Congratulations, Kasia! > > On Thu, Jan 2, 2020 at 6:52 PM Valentyn Tymofieiev > wrote: > >> Congratulations, Kasia! >> >> On Thu, Jan 2, 2020 at 1:23 AM Katarzyna Kucharczyk < >> ka.kucharc...@gmail.com> wrote:

Re: Jenkins jobs not running for my PR 10438

Hi, I'm also affected by this - I touched my PRs opened before the holiday break and no jobs were triggered. Do we know what breaks Jenkins/fixes it when stuff like this happens? Happy new year, Michal On Fri, Jan 3, 2020 at 1:42 AM Kai Jiang wrote: > Thanks Alan for checking this out! I closed