Re: Release 0.6.0

Thanks Ahmet ! Regards JB On 03/02/2017 07:42 AM, Ahmet Altay wrote: Sure, I can wait. To be clear, Thursday night in which time zone? Thank you, Ahmet On Wed, Mar 1, 2017 at 10:38 PM, Jean-Baptiste Onofré wrote: Hi Ahmet, Can you wait up to Thursday night ? Trying to merge BEAM-649. Tha

Re: Release 0.6.0

Pacific time is fine. Regards JB On 03/02/2017 07:42 AM, Ahmet Altay wrote: Sure, I can wait. To be clear, Thursday night in which time zone? Thank you, Ahmet On Wed, Mar 1, 2017 at 10:38 PM, Jean-Baptiste Onofré wrote: Hi Ahmet, Can you wait up to Thursday night ? Trying to merge BEAM-64

Re: Release 0.6.0

Sure, I can wait. To be clear, Thursday night in which time zone? Thank you, Ahmet On Wed, Mar 1, 2017 at 10:38 PM, Jean-Baptiste Onofré wrote: > Hi Ahmet, > > Can you wait up to Thursday night ? Trying to merge BEAM-649. > > Thanks ! > Regards > JB > > > On 03/01/2017 07:23 PM, Ahmet Altay wro

Re: First stable release: version designation?

Hi Davor, For a Beam community perspective, 1.0.0 would make more sense. We have a fair number of people starting with Beam (without knowing Dataflow). However, as Dataflow SDK (origins of Beam) was in 1.0.0, in order to avoid confusion with users coming to Beam from Dataflow, 2.0.0 could he

Re: Release 0.6.0

Hi Ahmet, Can you wait up to Thursday night ? Trying to merge BEAM-649. Thanks ! Regards JB On 03/01/2017 07:23 PM, Ahmet Altay wrote: Thank you. I will start working on it. Ahmet On Wed, Mar 1, 2017 at 9:03 AM, Aljoscha Krettek wrote: I just closed the last blocking issue, we should be g

Re: Pipeline termination in the unified Beam model

+1 I think it's a fair claim that a PCollection is "done" when it's watermark reaches positive infinity, and then it's easy to claim that a Pipeline is "done" when all of its PCollections are done. Completion is an especially reasonable claim if we consider positive infinity to be an actual infini

Pipeline termination in the unified Beam model

Raising this onto the mailing list from https://issues.apache.org/jira/browse/BEAM-849 The issue came up: what does it mean for a pipeline to finish, in the Beam model? Note that I am deliberately not talking about "batch" and "streaming" pipelines, because this distinction does not exist in the

Re: Beam File System in the Python SDK

Great! Thanks Sourabh. - Cham On Wed, Mar 1, 2017 at 3:58 PM Robert Bradshaw wrote: > Much needed! Added a couple of comments. > > On Wed, Mar 1, 2017 at 3:08 PM, Sourabh Bajaj < > sourabhba...@google.com.invalid> wrote: > > > Hi, > > > > BEAM-1441

Re: First stable release: version designation?

The following explanation for adopting 2.0 version should be put in release notes for the stable release. Cheers On Wed, Mar 1, 2017 at 2:03 PM, Dan Halperin wrote: > A large set of Beam users will be coming from the pre-Apache technologies > (aka Google Cloud Dataflow, Scio). Because Dataflow

Re: Beam File System in the Python SDK

Much needed! Added a couple of comments. On Wed, Mar 1, 2017 at 3:08 PM, Sourabh Bajaj < sourabhba...@google.com.invalid> wrote: > Hi, > > BEAM-1441 is a ticket > for > implementing the Beam File System in the Python SDK similar to the one > intro

Beam File System in the Python SDK

Hi, BEAM-1441 is a ticket for implementing the Beam File System in the Python SDK similar to the one introduced in BEAM-59 . I tried to take a pass on the implementation in #2136

Re: First stable release: version designation?

A large set of Beam users will be coming from the pre-Apache technologies (aka Google Cloud Dataflow, Scio). Because Dataflow was 1.0 before Beam started, there is a lot of pre-existing documentation, Stack Overflow, etc. that refers to version 1.0 to mean what is now a year-and-a-half old release.

Re: First stable release: version designation?

+1 to what Jesse and Amit said. On Wed, Mar 1, 2017 at 12:32 PM, Amit Sela wrote: > I think 1.0.0 for a couple of reasons: > > * It makes sense coming after 0.X (+1 Jesse). > * It is the FIRST stable release as a project, regardless of its roots. > * while the SDK is definitely a 2.0.0, Beam is

Apache Beam (virtual) contributor meeting @ Tue Mar 7, 2017

Hi everyone, Based on the high demand [1], let's try to organize a virtual contributor meeting on Tuesday, March 7, 2017 at 15:00 UTC. For convenience, calendar link [2] and an .ics file are attached. I tried to accommodate as many time zones as possible, but I know it might be hard for some of us

Re: First stable release: version designation?

I think 1.0.0 for a couple of reasons: * It makes sense coming after 0.X (+1 Jesse). * It is the FIRST stable release as a project, regardless of its roots. * while the SDK is definitely a 2.0.0, Beam is not made only of the SDK, and I hope we'll have more milage with users running all sorts of ru

Re: First stable release: version designation?

I think 1.0 makes the most sense. On Wed, Mar 1, 2017, 10:57 AM Davor Bonaci wrote: > The first stable release is our next major project-wide goal; see > discussion in [1]. I've been referring to it as "the first stable release" > for a long time, not "1.0.0" or "2.0.0" or "2017" or something el

Re: Let's make Beam transforms comply with PTransform Style Guide

Hey all, First couple rounds of fixes are in. Thanks Aviem Zur for contributing TextIO fixes and Dan Halperin for reviewing! One more fix by Reuven in progress (https://github.com/apache/beam/pull/1927). Follow https://issues.apache.org/jira/browse/BEAM-1353 and sub-issues for the status. Many of

Re: Merge HadoopInputFormatIO and HDFSIO in a single module

I wanted to follow up on this thread since I see some potential blocking questions arising, and I'm trying to help dipti along with her PR. Dipti's PR[1] is currently written to put files into: io/hadoop/inputformat The recent changes to create hadoop-common created: io/hadoop-common This means

First stable release: version designation?

The first stable release is our next major project-wide goal; see discussion in [1]. I've been referring to it as "the first stable release" for a long time, not "1.0.0" or "2.0.0" or "2017" or something else, to make sure we have an unbiased discussion and a consensus-based decision on this matter

Re: Release 0.6.0

Thank you. I will start working on it. Ahmet On Wed, Mar 1, 2017 at 9:03 AM, Aljoscha Krettek wrote: > I just closed the last blocking issue, we should be good to go now. > > Sorry again for the hold-up. > > On Tue, 28 Feb 2017 at 18:38 Ahmet Altay wrote: > > Thank you all. I will wait for rel

Re: Enforcer Rule- JDK1.7 for Beam

I would rather not turn off "all compiler warnings are errors" in a module unless we absolutely have to, since I think compiler warnings give us useful information. I wanted to talk about a structure that might work for what Dan is suggesting and anticipate some questions that Dipti might have for

Re: Next major milestone: first stable release

Just added the two I mentioned in my previous message.Thanks Davor. On Wed, Mar 1, 2017 at 6:27 PM, Aljoscha Krettek wrote: > On it! > > On Wed, 1 Mar 2017 at 18:17 Davor Bonaci wrote: > > > We've now moved the discussion into the content of the first stable > > release. > > > > I've created a

Re: Next major milestone: first stable release

On it! On Wed, 1 Mar 2017 at 18:17 Davor Bonaci wrote: > We've now moved the discussion into the content of the first stable > release. > > I've created a version in JIRA called "First stable release". I'd like to > invite everyone to triage JIRA issues you care about, and assign "Fix > Versions

Re: Next major milestone: first stable release

We've now moved the discussion into the content of the first stable release. I've created a version in JIRA called "First stable release". I'd like to invite everyone to triage JIRA issues you care about, and assign "Fix Versions" field to "First stable release" to mark the issue blocking for the

Re: Travis retest-this-please magic

Use your best judgement. Travis right now provides multi-JDK, multi-platform coverage not available in Jenkins. If the change is not sensitive to that, it is probably reasonable to proceed. On Wed, Mar 1, 2017 at 9:01 AM, Amit Sela wrote: > +1 > Can we merge PRs without waiting for Travis as lon

Re: Release 0.6.0

I just closed the last blocking issue, we should be good to go now. Sorry again for the hold-up. On Tue, 28 Feb 2017 at 18:38 Ahmet Altay wrote: Thank you all. I will wait for release blocking issues to be closed. Sergio, thank you for the information. I will document the friction points durin

Re: Travis retest-this-please magic

+1 Can we merge PRs without waiting for Travis as long as it's not working ? On Wed, Mar 1, 2017 at 6:52 PM Davor Bonaci wrote: > It cannot be done at this time. > > We should really move all Travis coverage into Jenkins and completely > deprecate Travis. I know Jason is looking into that ;-) >

Re: Travis retest-this-please magic

It cannot be done at this time. We should really move all Travis coverage into Jenkins and completely deprecate Travis. I know Jason is looking into that ;-) On Wed, Mar 1, 2017 at 3:51 AM, Amit Sela wrote: > Hi all, > > Recently I've encountered PRs where everything was green in Jenkins but >

Re: Performance Testing Next Steps

Thanks for writing this and taking care of this, Jason! I'm afraid I also cannot add anything except that I'm excited to see some results from this. On Wed, 1 Mar 2017 at 03:28 Kenneth Knowles wrote: Just got a chance to look this over. I don't have anything to add, but I'm pretty excited to fo

Travis retest-this-please magic

Hi all, Recently I've encountered PRs where everything was green in Jenkins but Travis was stuck and didn't execute. I couldn't (as the committer/reviewer) to do the same "retest this please" magic we apply to Jenkins, and I don't know of the possibility to do this in Travis. I know that on "my" T

Re: Next major milestone: first stable release

Yes, fully agree. As far as I understood/know, BEAM-59 is targeted for Beam 1.0 (it's what we discussed with Pei and Davor). Regards JB On 03/01/2017 11:39 AM, Ismaël Mejía wrote: Also joining a bit late, I agree with Amit, HDFS improvements are a really good thing to have before the stable

Re: Next major milestone: first stable release

Also joining a bit late, I agree with Amit, HDFS improvements are a really good thing to have before the stable release. I will also add the IOChannelFactory refactorings to support things like Read.from(“hdfs://”) aka BEAM-59. In the worse case particular IOs can still be marked as experimental t