Re: [PROPOSAL] Prepare Beam 2.9.0 release

Update: All runners except Dataflow passed quickstart validation on the release branch. Dataflow validation is blocked due to a Google internal issue that prevents building Dataflow containers based on latest code. Created blocker https://issues.apache.org/jira/browse/BEAM-6173 since release is e

2019 Beam Events

Hi Beam Community, I started curating industry conferences, meetups and events that are relevant for Beam, this initial list I came up with . *I'd love your help adding others that I might have overlook

beam9 failing most of the python tests

Hi, I see that beam9 is failing significantly more number of python related builds [1]. This also results in more failure of beam_PreCommit_Portable_Python_Commit [2] on beam9. Can someone with access to beam9 take a look? Thanks, Ankur [1] https://builds.apache.org/computer/beam9/builds [2] ht

Re: Graceful shutdown of long-running Beam pipeline on Flink

As noted, there is currently no support for Flink savepoints through the Beam API. However, it is now possible to restore from a savepoint with a Flink runner specific pipeline option: https://issues.apache.org/jira/browse/BEAM-5396 https://github.com/apache/beam/pull/7169#issuecomment-443283332

Re: Reviews for a few changes to the Go SDK

Hi Andrew, +Robert Burke (assignee for the both JIRAs) would be a better reviewer but he is out of office this week. I was helping him with a few reviews recently and I would be happy to review your changes too in his absence. Since you are not in a rush, I will try to review your changes befor

Re: [DISCUSS] Structuring Java based DSLs

To be honest, I don't think there's much worth doing right now. I think more self-contained is better for Beam SQL, generally. Two things I have on my mind are (1) SQL as an inline transform in every SDK and (2) supporting pure SQL like the CLI and JDBC driver, where the underlying language is an i

Re: [DISCUSS] Structuring Java based DSLs

Hi Robert, currently there is no actual proposal, I was just trying to gather feedback from the community. But my original thoughts would be [1]. I actually don't see much need for restructuring the code by nesting directories. If the community sees that it would make sense to structure the d

Re: [DISCUSS] Structuring Java based DSLs

Taking a step back, what exactly is the proposal. Looking at the original message, I see (1) Letting SQL take a dependency on Euphoria, sharing more code and taking advantage of the logical nesting of levels of abstraction. This makes sense to me. (2) Nesting the directories (but not the gradle ta

Re: Graceful shutdown of long-running Beam pipeline on Flink

There are propoosals for pipeline drain[1] and also for snapshot and update[2] for Apache Beam. We would love contributions in this space. 1: https://docs.google.com/document/d/1NExwHlj-2q2WUGhSO4jTu8XGhDPmm3cllSN8IMmWci8 2: https://docs.google.com/document/d/1UWhnYPgui0gUYOsuGcCjLuoOUlGA4QaY91n8p

Re: [PROPOSAL] Prepare Beam 2.9.0 release

I've been running tests on the existing branch. Will try to cut a candidate today. Seems like this is a small fix that fixes a flake. So feel free to send a cherry-pick. Thanks, Cham On Mon, Dec 3, 2018 at 4:28 AM Maximilian Michels wrote: > How far are we with the release? If the release bran

Re: Nexmark Phrase Triggering

(sorry, I missed this somehow) The documentation for Nexmark was updated on Confluence: https://cwiki.apache.org/confluence/x/sZCzBQ For Performance tests to behave the same way, we need to add separate jobs and simply use a different dataset for storing the results (same as in Nexmark). Besides

Re: [PROPOSAL] Prepare Beam 2.9.0 release

How far are we with the release? If the release branch hasn't been frozen, I'd like to cherry-pick https://github.com/apache/beam/pull/7171/files Thanks, Max On 30.11.18 04:17, Lukasz Cwik wrote: I got to thank Steve Niemitz for double checking my work and pointing out an error which helped n

Beam Dependency Check Report (2018-12-03)

High Priority Dependency Updates Of Beam Python SDK: Dependency Name Current Version Latest Version Release Date Of the Current Used Version Release Date Of The Latest Release JIRA Issue future 0.16.0 0.17.1 2016-10-27

Re: contributor in the Beam

Can you please fix the conflict in the PR ? Thanks Regards JB On 03/12/2018 08:52, Chaim Turkel wrote: > it looks like there was a failure that is not due to the code, how can > i continue the process? > https://github.com/apache/beam/pull/7162 > > On Thu, Nov 29, 2018 at 9:15 PM Chaim Turkel w

Re: contributor in the Beam

I just triggered a rerun of the test. Hint: each check is followed by the trigger phrase that will rerun that check. On Sun, Dec 2, 2018 at 11:53 PM Chaim Turkel wrote: > it looks like there was a failure that is not due to the code, how can > i continue the process? > https://github.com/apache