Re: [DISCUSS] Current ongoing work on runners

+1. For me it's one of the most important point for the new website. We should give a clear and exhaustive list of what we have, both for runners and IOs (with supported features). Regards JB ⁣​ On Oct 24, 2016, 21:52, at 21:52, "Ismaël Mejía" wrote: >Hello, > >I am really happy to see new ru

Re: [VOTE] Release 0.3.0-incubating, release candidate #1

+1 (binding) I did a quick review and tests. It looks good to me. Regards JB ⁣​ On Oct 24, 2016, 23:30, at 23:30, Aljoscha Krettek wrote: >Hi Team! > >Please review and vote at your leisure on release candidate #1 for >version >0.3.0-incubating, as follows: >[ ] +1, Approve the release >[ ] -1

Re: Podling Report Reminder - November 2016

Hi guys @James do you want to prepare this podling report ? I will review. Let me know if you need any help. Thanks Regards JB ⁣​ On Oct 25, 2016, 02:02, at 02:02, johndam...@apache.org wrote: >Dear podling, > >This email was sent by an automated system on behalf of the Apache >Incubator PMC.

Re: [DISCUSS] Using Verbs for Transforms

Sounds good to me. ⁣​ On Oct 24, 2016, 19:11, at 19:11, je...@smokinghand.com wrote: >I prefer MakeDistinct if we have to make it a verb.

Re: [DISCUSS] Current ongoing work on runners

On Mon, Oct 24, 2016 at 12:52 PM, Ismaël Mejía wrote: > Hello, > > I am really happy to see new runners been contributed to our community > (e.g. GearPump and Apex recently). We have not discussed a lot about the > current capabilities of both runners. > > Following the recent discussion about ma

Re: [DISCUSS] Current ongoing work on runners

I think it would be worth publishing a compatibility matrix, if not on the main site, as part of the branch itself. Even better would be if the compatibility matrix was automatically deduced based on a suite of tests that each runner could (attempt to) pass. On Mon, Oct 24, 2016 at 12:52 PM, Isma

Re: [DISCUSS] Using Verbs for Transforms

On Mon, Oct 24, 2016 at 8:52 PM, Robert Bradshaw wrote: > On Mon, Oct 24, 2016 at 10:12 AM, Kenneth Knowles > wrote: >> The precedent that we use verbs has many exceptions. We have >> ApproximateQuantiles, Values, Keys, WithTimestamps, and I would even >> include Sum (at least when I read it). >

Re: [DISCUSS] Using Verbs for Transforms

On Mon, Oct 24, 2016 at 10:12 AM, Kenneth Knowles wrote: > The precedent that we use verbs has many exceptions. We have > ApproximateQuantiles, Values, Keys, WithTimestamps, and I would even > include Sum (at least when I read it). True. > Historical note: the predilection towards verbs is from

Podling Report Reminder - November 2016

Dear podling, This email was sent by an automated system on behalf of the Apache Incubator PMC. It is an initial reminder to give you plenty of time to prepare your quarterly board report. The board meeting is scheduled for Wed, 16 November 2016, 10:30 am PDT. The report for your podling will for

The Availability of PipelineOptions

Hey everyone, I've been working on a declaration of intent for how we want to use PipelineOptions and an API change to be consistent with that intent. This is generally part of the move to the Runner API, specifically the desire to be able to reuse Pipelines and the ability to choose runner at the

Re: Maven Release Plugin Does Not Update Version of Archetypes

Sounds that the quoted commit is the right behavior; reverting it now would make the archetype fragile. I guess we should revert it just before declaring the first stable version, until then we'll have to do a manual actual during each release. On Mon, Oct 24, 2016 at 12:56 PM, Lukasz Cwik wrote

Re: [DISCUSS] Using Verbs for Transforms

That's how the mainframe programmers I've dealt with refer to it. I agree with Dan. We should either not change the name or change it to Distinct. It's just not worth the effort otherwise. On Mon, Oct 24, 2016, 3:10 PM Eugene Kirpichov wrote: > $0.02: Deduplicate? (lends to extensions like Dedup

Re: [DISCUSS] Using Verbs for Transforms

$0.02: Deduplicate? (lends to extensions like Deduplicate.by(some key extractor function)) On Mon, Oct 24, 2016 at 1:22 PM Dan Halperin wrote: > I find "MakeDistinct" more confusing. My votes in decreasing preference: > > 1. Keep `RemoveDuplicates` name, ensure that important keywords are in the

[VOTE] Release 0.3.0-incubating, release candidate #1

Hi Team! Please review and vote at your leisure on release candidate #1 for version 0.3.0-incubating, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA releas

Re: Tracking backward-incompatible changes for Beam

I don't think we have it right now. We should, of course, but this is something that needs to be defined/discussed first. On Mon, Oct 24, 2016 at 1:20 PM, Neelesh Salian wrote: > +1 for the labels and also a need for tests. > Do we document any rules for backward-compatibility? Be good to have a

Re: [PROPOSAL] New Beam website design?

Abdullah, welcome! I think it's rather clear we've been struggling with the website, so any help is very welcome. It is a little bit messy right now -- there are a few outstanding pull requests and forked branches. I'm trying to get all this into one place, so anybody can contribute and make progr

Re: [DISCUSS] Using Verbs for Transforms

I find "MakeDistinct" more confusing. My votes in decreasing preference: 1. Keep `RemoveDuplicates` name, ensure that important keywords are in the Javadoc. This reduces churn on our users and is honestly pretty dang descriptive. 2. Rename to `Distinct`, which is clear if you're a SQL user and li

Re: Tracking backward-incompatible changes for Beam

+1 for the labels and also a need for tests. Do we document any rules for backward-compatibility? Be good to have a checklist-like list. On Mon, Oct 24, 2016 at 1:02 PM, Davor Bonaci wrote: > It would be awesome to have that! At least a good portion of > backward-incompatible changes could be

Re: Tracking backward-incompatible changes for Beam

It would be awesome to have that! At least a good portion of backward-incompatible changes could be automatically caught. We should also think about defining backward-compatibility more precisely. This would be good in its own right, but also necessary to configure the tool. Historically, we have

Re: Maven Release Plugin Does Not Update Version of Archetypes

Archetypes using SNAPSHOT makes the development process consistent and has us maintain the archetype code as the SDK code changes. I feel as though its important to maintain the archetypes in sync with the SDK code as we introduce backwards incompatible changes. Also, because the SDK is changing, i

[DISCUSS] Current ongoing work on runners

Hello, I am really happy to see new runners been contributed to our community (e.g. GearPump and Apex recently). We have not discussed a lot about the current capabilities of both runners. Following the recent discussion about making ongoing work more explicit in the mailing list, I would like to

Re: Maven Release Plugin Does Not Update Version of Archetypes

Hi, to unblock the release I'm changing the version manually now, yes. Would be good to fix though. Cheers, Aljoscha On Mon, 24 Oct 2016 at 20:30 Dan Halperin wrote: > Hmm, this is new in 0.3.0, looks caused by > > https://github.com/apache/incubator-beam/commit/1f30255edcdd9c1e445b69248191c855

Re: Maven Release Plugin Does Not Update Version of Archetypes

Hmm, this is new in 0.3.0, looks caused by https://github.com/apache/incubator-beam/commit/1f30255edcdd9c1e445b69248191c8552724f086#diff-4795b1d27449c01332aad192348eL111

Re: Maven Release Plugin Does Not Update Version of Archetypes

Would it unblock the release to manually configure the version in the 0.3.0-release branch? On Mon, Oct 24, 2016 at 11:09 AM, Dan Halperin wrote: > Correct issue link: https://issues.apache.org/jira/browse/BEAM-806 > > No answers, but looking around. > > On Mon, Oct 24, 2016 at 10:10 AM, Aljosch

Re: Maven Release Plugin Does Not Update Version of Archetypes

Correct issue link: https://issues.apache.org/jira/browse/BEAM-806 No answers, but looking around. On Mon, Oct 24, 2016 at 10:10 AM, Aljoscha Krettek wrote: > Hi, > are there any Maven mavens who happen to know how > https://issues.apache.org/jira/browse/BEAM-108 can be fixed? By the way, > the

Re: A CI Server for Beam?

Hi Sergio - as I mentioned in my email about integration testing sources[1], we'll also need a cluster hosting mechanism for that and I'm checking out some options. In addition to mesos+marathon, I'm checking out kubernetes and docker swarm. I started out excited about mesos, but so far mesos has

Re: [DISCUSS] Using Verbs for Transforms

The precedent that we use verbs has many exceptions. We have ApproximateQuantiles, Values, Keys, WithTimestamps, and I would even include Sum (at least when I read it). Historical note: the predilection towards verbs is from the Google Style Guide for Java method names

[DISCUSS] Using Verbs for Transforms

I prefer MakeDistinct if we have to make it a verb.

Maven Release Plugin Does Not Update Version of Archetypes

Hi, are there any Maven mavens who happen to know how https://issues.apache.org/jira/browse/BEAM-108 can be fixed? By the way, the release plugin does also not update the version of the archetypes when setting the next SNAPSHOT version. IMHO, it's a bit of a release blocker so I'm hoping we can ge

Re: [DISCUSS] Using Verbs for Transforms

Thanks JB and Jesse. Would something like "MakeDistinct" or "AvoidDuplicate" sound better? I can do the collective changes of the name and the javadoc at one go. Having it documented can be super helpful. On Mon, Oct 24, 2016 at 9:58 AM, Jean-Baptiste Onofré wrote: > It could make sense. Howeve

Re: [DISCUSS] Using Verbs for Transforms

It could make sense. However we should keep it informative. I mean that distinct could appear confusing for some users compare to AvoidDuplicate or whatever. Regards JB ⁣​ On Oct 24, 2016, 18:46, at 18:46, Jesse Anderson wrote: >My original thought for this change was that Crunch uses the cla

[DISCUSS] Using Verbs for Transforms

Hello, First of all, thank you to Daniel, Robert and Jesse for their review on this: https://issues.apache.org/jira/browse/BEAM-239 A point that came up was using verbs explicitly for Transforms. Here is the PR: https://github.com/apache/incubator-beam/pull/1164 Posting it to help understand if

Re: [DISCUSS] Using Verbs for Transforms

My original thought for this change was that Crunch uses the class name Distinct. SQL also uses the keyword distinct. Maybe the rule should be changed to adjectives or verbs depending on the context. Using a verb to describe this class really doesn't connote what the class does as succinctly as t

Re: [DISCUSS] Using Verbs for Transforms

Hi Transforms are already expressed mostly with verbs. I agree with your proposal or at least fully documented semantic. By the way, we have to be careful when changing as it could break the API. Regards JB ⁣​ On Oct 24, 2016, 18:40, at 18:40, Neelesh Salian wrote: >Hello, > >First of all, t

Re: [ANNOUNCEMENT] New committers!

Now I understand the purpose of your e-mail. It wasn't obvious to me. And it's because I know you that I was surprised  by the e-mail. I think your e-mail was aim to be informative, and I try to provide an informative answer. Regarding Beam, it's large project with a production grade background

Re: Start of release 0.3.0-incubating

Thanks JB! (et al.) Excellent suggestions. Thanks, Dan On Thu, Oct 20, 2016 at 9:32 PM, Jean-Baptiste Onofré wrote: > Hi Dan, > > No problem, MQTT and other IOs will be in the next release.. > > IMHO, it would be great to have: > 1. A release reminder couple of days before a release. Just to as

Re: [ANNOUNCEMENT] New committers!

I suggest a seven feats model for becoming a committer https://en.m.wikipedia.org/wiki/Labours_of_Hercules. On Mon, Oct 24, 2016, 7:10 AM Christian Schneider wrote: > I was just a bit concerned that new committers might have to do > comparable contributions like the current three new committers

Re: [ANNOUNCEMENT] New committers!

I was just a bit concerned that new committers might have to do comparable contributions like the current three new committers .. which might be quite a high bar :-) Christian On 24.10.2016 16:05, Jean-Baptiste Onofré wrote: My point is that the bar should not be high for committer-ship but n

Re: [ANNOUNCEMENT] New committers!

My point is that the bar should not be high for committer-ship but not too low as the key thing is valuable contribution. Open mind/discussion, documentation, code, ... are valuable. On the other hand, IMHO, it's also human consideration and behavior. So it's discussion. Not sure I understand t

Re: [ANNOUNCEMENT] New committers!

I know how apache projects work. You should know me long enough to know. For people who are less familiar with the process.. One pmc member proposes someone as new committer and the pmc members vote on him/her. Every project of course has a different idea of how high that bar for entry should

Re: [ANNOUNCEMENT] New committers!

Hi Christian, Your e-mail sounds "weird" to me: like other Apache projects, if your contribution is valuable and sustained, then you will deserve committer-ship. It's meritocracy based. Working part time is not a problem, it would make little bit more time to get into. So, just a timing point,

Re: [DISCUSS] Deferring (pre) combine for merging windows.

@Amit: Yes, Flink is more "what you write is what you get". For example, in Flink we have a Fold function for windows which cannot be efficiently computed with merging windows (it would require using a "group by" window and then folding the iterable). We just don't allow this. For Beam, I think it

Re: [ANNOUNCEMENT] New committers!

Congratulations to all three new committers. These are very substantial contributions and all three definitely have earned the committership. As I am only starting to try to help out with beam I wonder how hard it is to become a committer. If I see that Thomas Groh is the number 1 contributor f

Re: [ANNOUNCEMENT] New committers!

Congrats and a warm welcome! -Max On Sun, Oct 23, 2016 at 6:02 AM, Robert Bradshaw wrote: > Congrats and welcome to all three of you! > > On Sat, Oct 22, 2016 at 9:02 AM, Thomas Weise wrote: >> Thanks everyone! >> >> >> On Sat, Oct 22, 2016 at 12:59 AM, Aljoscha Krettek >> wrote: >> >>> Welco