Re: Flaky tests in Beam

2021-08-17 Thread Jan Lukavský
If changes to core are causing Dataflow precommits to fail but not local precommits, that suggests we lack test coverage? What is the difference between "Dataflow precommit" and "local precommit" (besides that the latter can be run without GCP)? If "local precommit" should catch _all_ regression

P0 (outage) report

2021-08-17 Thread Beam Jira Bot
This is your daily summary of Beam's current outages. See https://beam.apache.org/contribute/jira-priorities/#p0-outage for the meaning and expectations around P0 issues. BEAM-12766: Already Exists: Dataset apache-beam-testing:python_bq_file_loads_NNN (https://issues.apache.org/jira/browse

P1 issues report (40)

2021-08-17 Thread Beam Jira Bot
This is your daily summary of Beam's current P1 issues, not including flaky tests (https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20statusCategory%20!%3D%20Done%20AND%20priority%20%3D%20P1%20AND%20(labels%20is%20EMPTY%20OR%20labels%20!%3D%20flake). See https://beam.apache.

Flaky test issue report (32)

2021-08-17 Thread Beam Jira Bot
This is your daily summary of Beam's current flaky tests (https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20statusCategory%20!%3D%20Done%20AND%20labels%20%3D%20flake) These are P1 issues because they have a major negative impact on the community and make it hard to determin

Re: MapState API

2021-08-17 Thread Kenneth Knowles
Can the change be observed in that case? I think the semantics should be that the timing of the actual write cannot be observed. Kenn On Tue, Aug 17, 2021 at 10:19 AM Luke Cwik wrote: > Yeah, the "or state is committed" means that we should resolve it before > any additional writes but that onl

Re: MapState API

2021-08-17 Thread Kenneth Knowles
100% agree with your examples. On Tue, Aug 17, 2021 at 1:21 PM Luke Cwik wrote: > I believe that all our ReadableState calls are point in time snapshots of > what the underlying state is plus any changes that happened within the > bundle already (e.g. you can see your own writes/appends/clears a

Re: A simpler way to define and use Java cross-language transforms

2021-08-17 Thread Jan Lukavský
On 8/17/21 10:40 PM, Luke Cwik wrote: On Tue, Aug 17, 2021 at 1:28 PM Chamikara Jayalath mailto:chamik...@google.com>> wrote: On Tue, Aug 17, 2021 at 1:01 PM Luke Cwik mailto:lc...@google.com>> wrote: I see the language differences but still feel as though there is a