Re: Jenkins jobs not running for my PR 10438

2020-05-11 Thread rahul patwari
Hi, Can you please trigger pre-commit checks for https://github.com/apache/beam/pull/11581 Thanks, Rahul On Tue, May 12, 2020 at 7:12 AM Ahmet Altay wrote: > Done for both Yoshiki and Tomo's PRs. > > On Mon, May 11, 2020 at 6:33 PM Tomo Suzuki wrote: > >> Hi Beam committers, >> >> Would you

Re: Apache Beam application to Season of Docs 2020

2020-05-11 Thread Pablo Estrada
I'll be happy to mentor for improvements to the Capability Matrix. There had been some discussions about that earlier. +Kenneth Knowles +Robert Bradshaw +Thomas Weise I may set a quick meeting with each of you to get your thoughts about Capability Matrix improvements. (also happy to welcome an

Re: Jenkins jobs not running for my PR 10438

2020-05-11 Thread Ahmet Altay
Done for both Yoshiki and Tomo's PRs. On Mon, May 11, 2020 at 6:33 PM Tomo Suzuki wrote: > Hi Beam committers, > > Would you run the basic precommit checks for > https://github.com/apache/beam/pull/11674 ? > > Regards, > Tomo >

Re: Jenkins jobs not running for my PR 10438

2020-05-11 Thread Tomo Suzuki
Hi Beam committers, Would you run the basic precommit checks for https://github.com/apache/beam/pull/11674 ? Regards, Tomo

Re: Apache Beam application to Season of Docs 2020

2020-05-11 Thread Kyle Weaver
Cool! I signed up as a mentor. On Mon, May 11, 2020 at 4:56 PM Aizhamal Nurmamat kyzy wrote: > PS: I am registered as a mentor too, happy to onboard the tech writers to > The Apache Way and Beam community processes when time comes. > > On Mon, May 11, 2020 at 1:52 PM Aizhamal Nurmamat kyzy < >

Re: [DISCUSS] finishBundle once per window

2020-05-11 Thread Robert Bradshaw
StartBundle pre-dated setUp, which makes it less useful than before. With DoFn re-use, however, startBundle can be used to ensure the DoFn is instantiated to a clean state. On Thu, May 7, 2020 at 2:03 PM Reuven Lax wrote: > I think startBundle is useful for convenience and performance, but not

Re: JIRA priorities explaination

2020-05-11 Thread Kenneth Knowles
This is filed as https://issues.apache.org/jira/browse/INFRA-20231 which should take place now. On Fri, May 1, 2020 at 3:53 PM Ahmet Altay wrote: > +1 sounds good to me. Oftentimes I confused the relative priorities of > critical/blocker/major. > > On Fri, May 1, 2020 at 3:05 PM Tyson Hamilton

Re: [Proposal] Apache Beam Fn API - GCP IO Debuggability Metrics

2020-05-11 Thread Alex Amato
Thanks for the great feedback so far :). I've included many new ideas, and made some revisions. Both docs have changed a fair bit since the initial mail out. https://s.apache.org/beam-gcp-debuggability https://s.apache.org/beam-histogram-metrics PTAL and let me know what you think, and hopefully

Re: Apache Beam application to Season of Docs 2020

2020-05-11 Thread Aizhamal Nurmamat kyzy
PS: I am registered as a mentor too, happy to onboard the tech writers to The Apache Way and Beam community processes when time comes. On Mon, May 11, 2020 at 1:52 PM Aizhamal Nurmamat kyzy wrote: > Apache Beam got accepted into Season of Docs, yay! [1] > > @Kyle/Pablo, could you please fill

Re: Apache Beam application to Season of Docs 2020

2020-05-11 Thread Aizhamal Nurmamat kyzy
Apache Beam got accepted into Season of Docs, yay! [1] @Kyle/Pablo, could you please fill out the mentor registration form by tomorrow morning [2]? Is there anyone else interested in becoming a mentor for either of the 2 projects [3]? The program requires at least two open source mentors for

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-11 Thread Nam Bui
Updates for today: - Thanks Brian & Ahmet for your reviews. I left my comments for some of the questions and also adapted new changes to the reviews [1]. - I see that the new blog post was merged yesterday, so I added it to the PR as well. I briefly tried the script from Robert with the input of

Re: [RESULT][VOTE] Accept the Firefly design donation as Beam Mascot - Deadline Mon April 6

2020-05-11 Thread Aizhamal Nurmamat kyzy
@Ismael, this is in my work items for as soon as we complete the migration of the website. @Kyle, thanks for filing Jira, I assigned it to myself. On Mon, May 11, 2020 at 10:03 AM Kyle Weaver wrote: > > Now that the vote has passed maybe we should add the images somewhere > > in the website so

Re: [DISCUSS] How many Python 3.x minor versions should Beam Python SDK aim to support concurrently?

2020-05-11 Thread Valentyn Tymofieiev
I agree with the point echoed earlier that the lowest and the highest of supported versions will probably give the most useful test signal for possible breakages. So 3.5. and 3.7 as high-priority versions SGTM. This can change later once Beam drops 3.5 support. On Mon, May 11, 2020 at 10:05 AM

Re: [DISCUSS] How many Python 3.x minor versions should Beam Python SDK aim to support concurrently?

2020-05-11 Thread Kyle Weaver
We've since moved our official Docker images here: https://hub.docker.com/search?q=apache%2Fbeam_python=image But Docker downloads are not as representative of actual usage as PyPI. On Mon, May 11, 2020 at 1:05 PM Yoshiki Obata wrote: > Hello again, > > Test infrastructure update is ongoing

Re: [DISCUSS] How many Python 3.x minor versions should Beam Python SDK aim to support concurrently?

2020-05-11 Thread Yoshiki Obata
Hello again, Test infrastructure update is ongoing and then we should determine which Python versions are high-priority. According to Pypi downloads stats[1], download proportion of Python 3.5 is almost always greater than one of 3.6 and 3.7. This situation has not changed since Robert told us

Re: Greetings from Borzoo

2020-05-11 Thread Borzoo Esmailloo
Thank you :) On Mon, May 11, 2020 at 6:30 PM Luke Cwik wrote: > Welcome, I have added you as a JIRA contributor. > > The Apache Beam contribution guide[1] is a good starting point and/or the > starter JIRAs[2]. > > 1: https://beam.apache.org/contribute/ > 2:

Re: [RESULT][VOTE] Accept the Firefly design donation as Beam Mascot - Deadline Mon April 6

2020-05-11 Thread Kyle Weaver
> Now that the vote has passed maybe we should add the images somewhere > in the website so people can easily find the Firefly to use it +1 Maybe something to revisit after the website overhaul is complete. I filed https://jira.apache.org/jira/browse/BEAM-9948 if anyone wants to take it. On Mon,

Re: [RESULT][VOTE] Accept the Firefly design donation as Beam Mascot - Deadline Mon April 6

2020-05-11 Thread Ismaël Mejía
Now that the vote has passed maybe we should add the images somewhere in the website so people can easily find the Firefly to use it. Something like what we do with our logos https://beam.apache.org/community/logos/ WDYT? any taker? On Tue, Apr 28, 2020 at 7:43 PM Pablo Estrada wrote: > > I'll

Re: Support for AWS SDK v2 and enhanced fanout in KinesisIO

2020-05-11 Thread Alexey Romanenko
Thanks Ismaël for recalling of this thread, I think we should start to take some efforts to deprecate the AWS SDK V1 IOs that are already implemeneted using V2 (if there are no other objections). In this case, it would make sense to abstract some common code ONLY if ,for some reasons, we

Re: Greetings from Borzoo

2020-05-11 Thread Luke Cwik
Welcome, I have added you as a JIRA contributor. The Apache Beam contribution guide[1] is a good starting point and/or the starter JIRAs[2]. 1: https://beam.apache.org/contribute/ 2: https://s.apache.org/beam-starter-tasks On Sat, May 9, 2020 at 7:55 AM Borzoo Esmailloo wrote: > Hello, > > I

Re: [DISCUSS] Dealing with @Ignored tests

2020-05-11 Thread Luke Cwik
Deleting ignored tests does lead us to losing the reason as to why the test case was around so I would rather keep it around. I think it would be more valuable to generate a report that goes on the website/wiki showing stability of the modules (num tests, num passed, num skipped, num failed

Re: Beam 2.21 release update

2020-05-11 Thread Luke Cwik
Thanks for the fix Max. On Mon, May 11, 2020 at 5:46 AM Maximilian Michels wrote: > FYI I've created this issue and marked it as a blocker: > https://jira.apache.org/jira/browse/BEAM-9947 > > Essentially, the timer encoding is broken for all non-standard key > coders. The fix can be found here:

RE: Unit-testing BEAM pipelines with PROCESSING_TIME timers

2020-05-11 Thread Robert.Butcher
Many thanks, Darshan! I’ve made equivalent changes to my test case and it’s working fine. Kind regards, Rob From: Darshan Jani [mailto:darshanjani...@gmail.com] Sent: 11 May 2020 15:08 To: dev@beam.apache.org Subject: Re: Unit-testing BEAM pipelines with PROCESSING_TIME timers

Re: Unit-testing BEAM pipelines with PROCESSING_TIME timers

2020-05-11 Thread Darshan Jani
Hi Robert, I found this sample test with Timer on processing time. >From the error, I assume there may be is a problem what are you asserting in your PAssert. https://github.com/apache/beam/blob/master/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/ParDoTest.java#L3633-L3665 I ran

Re: Jenkins jobs not running for my PR 10438

2020-05-11 Thread Yoshiki Obata
Thank you Hannah! And could anyone trigger these checks again in https://github.com/apache/beam/pull/11656 ? Run Portable_Python PreCommit Run Python 3.5 PostCommit Run Python 3.6 PostCommit Run Python 3.7 PostCommit Regards yoshiki 2020年5月11日(月) 12:54 Hannah Jiang : > > It is done. Some more

Re: Beam 2.21 release update

2020-05-11 Thread Maximilian Michels
FYI I've created this issue and marked it as a blocker: https://jira.apache.org/jira/browse/BEAM-9947 Essentially, the timer encoding is broken for all non-standard key coders. The fix can be found here: https://github.com/apache/beam/pull/11658 -Max On 08.05.20 18:53, Udi Meiri wrote: > +Chad

Beam Dependency Check Report (2020-05-11)

2020-05-11 Thread Apache Jenkins Server
ERROR: File 'src/build/dependencyUpdates/beam-dependency-check-report.html' does not exist

Unit-testing BEAM pipelines with PROCESSING_TIME timers

2020-05-11 Thread Robert.Butcher
I have a BEAM DoFn that I'm attempting to unit test. It involves using a timer based on processing time and I've not managed to get it to fire. The relevant code excerpts are as follows: @TimerId("timer") private final TimerSpec timer = TimerSpecs.timer(TimeDomain.PROCESSING_TIME);

Re: [DISCUSS] Dealing with @Ignored tests

2020-05-11 Thread Jan Lukavský
I think that we do have Jira issues for ignored test, there should be no problem with that. The questionable point is that when test gets Ignored, people might consider the problem as "less painful" and postpone the correct solution until ... forever. I'd just like to discuss if people see

Re: [REVIEW][please pause website changes] Migrated the Beam website to Hugo

2020-05-11 Thread Nam Bui
Hi, @Ahmet: Yeah, it's all clear to me. :) @Robert: Thanks for your ideas and also the script. It really helps me to serve my works. Best regard! On Sat, May 9, 2020 at 2:10 AM Ahmet Altay wrote: > This sounds reasonable to me. Thank you. Nam, does it make sense to you? > > On Fri, May 8,