Flaky test issue report

2021-04-06 Thread Beam Jira Bot
This is your daily summary of Beam's current flaky tests. These are P1 issues because they have a major negative impact on the community and make it hard to determine the quality of the software. BEAM-12096: Flake: test_progress_in_HTML_JS_when_in_notebook (https://issues.apache.org/jira/br

P1 issues report

2021-04-06 Thread Beam Jira Bot
This is your daily summary of Beam's current P1 issues, not including flaky tests. See https://beam.apache.org/contribute/jira-priorities/#p1-critical for the meaning and expectations around P1 issues. BEAM-12104: Java WordCountIT on Dataflow failing (GHA) (https://issues.apache.org/jira/b

Re: Long term support versions of Beam Java

2021-04-06 Thread Robert Bradshaw
We actually do try to maintain state (and pipeline shape) compatibility between Beam versions (e.g. this is why we have multiple distinct sketch implementations rather than "fix" the one). It's true that this is easier said than done, and some people are more vigilant about that than others). State

Re: Long term support versions of Beam Java

2021-04-06 Thread Jan Lukavský
Hi, do we know what is the reason users stay on an older version of Beam? My guess would be that it is not related to API changes, but more likely to state incompatibility. Maybe if we could figure out a way which would enable a smooth migration of state (and timers) between Beam versions, tha

Re: Flink runner configuration for closure cleaner

2021-04-06 Thread Kyle Weaver
I don't think this will require Beam to have its own configuration option. You should be able to set the property "pipeline.closure-cleaner-level" in your flink.conf and then pass it to Beam using Beam's "--flink-conf-dir" pipeline option. On Tue, Apr 6, 2021 at 2:28 PM Raman Gupta wrote: > Hell

Flink runner configuration for closure cleaner

2021-04-06 Thread Raman Gupta
Hello all: I created https://issues.apache.org/jira/browse/BEAM-12055 because I'm having an issue with using the Flink runner locally, due to https://issues.apache.org/jira/browse/FLINK-15773. Does anyone see any reason why Beam's Flink runner should not provide a configuration option that can dis

Re: Long term support versions of Beam Java

2021-04-06 Thread Robert Bradshaw
I do think there's value in having an LTS release, if there's sufficient interest to fund it (specifically, figuring out who would be backporting fixes and cutting the new releases). On Mon, Apr 5, 2021 at 1:14 PM Elliotte Rusty Harold wrote: > Hi, > > I'd like to return to the discussion around

Re: [DISCUSS] Include inherited members in Python API Docs?

2021-04-06 Thread Brian Hulette
Sure, I can try cutting out PTransform. We could also look into reducing noise by: - removing undoc-members from the config [1] (this would make it so only objects with a docstring are added to the generated docs) - adding :meta private:` to docstrings for objects we don't want publicly visible [

Re: [DISCUSS] Include inherited members in Python API Docs?

2021-04-06 Thread Robert Bradshaw
Way too many things are inherited from PTransform, can we at least cut that out? On Tue, Apr 6, 2021 at 1:09 PM Brian Hulette wrote: > Just wanted to bump this - does anyone have concerns with the way the API > docs look when inherited members are included? > > On Wed, Mar 31, 2021 at 5:23 PM Br

Re: [DISCUSS] Include inherited members in Python API Docs?

2021-04-06 Thread Brian Hulette
Just wanted to bump this - does anyone have concerns with the way the API docs look when inherited members are included? On Wed, Mar 31, 2021 at 5:23 PM Brian Hulette wrote: > I staged my current working copy built from head here [1], see > CombinePerKey here [2]. Note it also has a few other ch

One day left: Last call for Beam College!

2021-04-06 Thread Mara Ruvalcaba
* * ** *** Hi Beam Community, Don’t miss out on Beam College - book your spot today. It’s packed with tons of content, designed to help improve your data processing skills from industry’s top experts. And it’s FREE! The first day of Beam College starts tomorrow, April 7th. If you wa

Re: Null checking in Beam

2021-04-06 Thread Jan Lukavský
I agree that there are _some_ added annotations at _some_, that are useful - most notably @NonNull on method arguments, possibly return values. Adding @NonNull into exception type being thrown seems awkward. The @UnknownKeyFor probably should not be there, as it brings no value. Did we raise th