Re: Possibility for GSoC 2016

Hi Manu, as Davor said, the Runner API will required some weeks to stabilize. So, we may expect the API ready during summer. The PCollection & PTransform should not change that much, but it will be probably impacted by the Runner API. Anyway you can start with this part. Regards JB On 03/2

Re: [HEADS UP] Renaming/polishing

Not Central, Apache SNAPSHOT repo. Regards JB On 03/21/2016 11:12 PM, Andreas Veithen wrote: On Mon, Mar 21, 2016 at 11:54 AM, Jean-Baptiste Onofré wrote: Thinking about this, I would prefer 0.1.0-incubating-SNAPSHOT, as it will also indicate that the SNAPSHOTs (on Central) are from incubati

Re: Possibility for GSoC 2016

Hi JB, I've started working on Gearpump-runner (BEAM-79 ). Do you have a time line for when the new Runner API will get stabilized ? Besides Runner API, will there be any changes to PCollection and PTransform APIs ? If not, I think I can work on the t

[PROPOSAL] Writing More Expressive Beam Tests

Hey everyone, I've been working on a proposal to expand the capabilities of our testing API, mostly around writing deterministic tests for pipelines that have interesting triggering behavior, especially speculative and late triggers. I've shared a doc here

Re: [HEADS UP] Renaming/polishing

On Mon, Mar 21, 2016 at 11:54 AM, Jean-Baptiste Onofré wrote: > Thinking about this, I would prefer 0.1.0-incubating-SNAPSHOT, as it will > also indicate that the SNAPSHOTs (on Central) are from incubating. > There are no snapshots on Central... So, I would propose 0.1.0-incubating-SNAPSHOT (an

Re: Renaming process: first step Maven coordonates

Hi Ben, it works fine with Maven >= 3.2.x (current version is 3.3.9). Most of incubator projects use x.x.x-incubating-SNAPSHOT: https://git1-us-west.apache.org/repos/asf?p=incubator-batchee.git;a=blob_plain;f=pom.xml;hb=HEAD https://git1-us-west.apache.org/repos/asf?p=incubator-apex-core.git;a

Re: Renaming process: first step Maven coordonates

On Mon, Mar 21, 2016 at 6:22 PM, Ben Chambers wrote: > I don't think Maven will recognize 0.1.0-incubating-SNAPSHOT as a snapshot. > It will recognize it as 0.1.0 with the "incubating-SNAPSHOT" qualifier. > > For instance, looking at the code for parsing qualifiers, it only handles > the string "

Re: Renaming process: first step Maven coordonates

Many issues have been raised here and I cannot tell the direction people are working now on the PR. So here is my current thinking, which may be a +1 to some things or may be different. 1. Versions - This seems odd: 0.1.0-incubating < 0.1.0-SNAPSHOT It mostly shouldn't matter but seems better

Re: Renaming process: first step Maven coordonates

I would be in favor of one group id. For the developer, hierarchies are really important. They are visible in the directory layout of the Maven project and in the dependency tree. For the user, it shouldn't matter how the project is structured. He pulls in artifacts simply from the "org.apache.beam

Re: Committer workflow

+1 for knowledge-sharing and getting to know other pieces of our code base. I would definitely be interested in getting to know Beam better, apart from the parts a runner developer deals with. On Mon, Mar 21, 2016, 20:19 Kenneth Knowles wrote: > +1 for emphasizing the knowledge-sharing aspect of

Re: Renaming process: first step Maven coordonates

I don't think Maven will recognize 0.1.0-incubating-SNAPSHOT as a snapshot. It will recognize it as 0.1.0 with the "incubating-SNAPSHOT" qualifier. For instance, looking at the code for parsing qualifiers, it only handles the string "SNAPSHOT" specially, not "incubating-SNAPSHOT". http://maven.apa

Re: Committer workflow

+1 for emphasizing the knowledge-sharing aspect of review. I think it is the most important for project health, and the most fun too! I love the chance to learn about a new piece of code (or learn how I messed up in my own code :-)​

Re: Renaming process: first step Maven coordonates

Hi Ben, 1. True for Python, but it can go in a folder in sdk (sdk/python) anyway. I think the DSLs (Java based) and other languages that we might introduce (Scala, ...) can be the same. 2. The incubating has to be in the released filenames. So it can be in the version or name. Anyway, my pro

Re: Renaming process: first step Maven coordonates

1. Regarding "java" as a module -- are we sure that other languages will be packaged using Maven as well? For instance, Python has its own ecosystem which likely doesn't play well with Python. 2. Using the literal "SNAPSHOT" as the qualifier has special meaning Maven -- it is newer than all other

Re: [PROPOSAL] Create new Jira IO component

We'll like have some other extensions (statistical libraries, etc). But I agree that it's confusing people as it is. +1 to creating java-sdk-io and renaming java-sdk-gcp to java-sdk-io-gcp. On Mon, Mar 21, 2016 at 10:10 AM, Jean-Baptiste Onofré wrote: > Good point Dan: if sdk-java-extension comp

Re: [PROPOSAL] Create new Jira IO component

Good point Dan: if sdk-java-extension component actually contains only IO related, why not rename it (it could be easier for users ;)). Regards JB On 03/21/2016 06:04 PM, Dan Halperin wrote: I currently only own "sdk-java-gcp", which does not give me great visibility into the bugs that are not

Re: [PROPOSAL] Create new Jira IO component

I currently only own "sdk-java-gcp", which does not give me great visibility into the bugs that are not Google Cloud-specific. I would like to see more of the IO-related issues earlier, so in that sense it makes sense to me to have a broader IO component. I looked at "sdk-java-extensions" and ever

Re: [PROPOSAL] Create new Jira IO component

Yes, it's what I used for now (sdk-java-extensions). And you are right, it's enough for now. We will create a dedicated component when the number of IOs (and different extensions) will come. Regards JB On 03/21/2016 05:57 PM, Frances Perry wrote: The original plan was that IOs would just be

Re: Committer workflow

I'd like to suggest a slightly stronger review process. In the draft guidelines it currently says all PRs should get reviewed, regardless of author. The only real exception to that is something that is an emergency f

Re: Committer workflow

+1 The key part is the fact that a reviewer (different from the author) has to put a comment LGTM (Looks Good To Me) in the PR. With this, the reviewer or the author can push the change. Thanks Frances ! Regards JB On 03/21/2016 05:53 PM, Frances Perry wrote: I'd like to suggest a slightly

Re: [PROPOSAL] Create new Jira IO component

The original plan was that IOs would just be in the library extensions (e.g. sdk-java-extensions). It'd fine to subdivide that further if needed, but maybe we should wait until it gets a bit bigger? Dan, what do you think, as component owner? On Mon, Mar 21, 2016 at 2:33 AM, Jean-Baptiste Onofré

Re: Capability Matrix

Thanks, all! On Fri, Mar 18, 2016 at 6:46 AM Amit Sela wrote: > Looks great! > I think it's the best way to give a clear picture of capabilities for users > and runner developers. > And as always, Love the colours ;) > > > On Fri, Mar 18, 2016 at 3:33 PM Kostas Kloudas < > k.klou...@data-artisan

Re: Renaming process: first step Maven coordonates

Hi Lukasz, both are possible. Some projects use different groupId. It's the case for Karaf or Camel for instance: http://repo.maven.apache.org/maven2/org/apache/karaf/ You can see there the different groupId, containing the different artifacts. On the other hand, other projects use an uniqu

Re: Renaming process: first step Maven coordonates

I like the single groupId since it makes it simpler to find all related components for a project. Is there a common practice in maven for multi-module vs inheritance projects for choosing the groupId? On Mon, Mar 21, 2016 at 7:32 AM, Jean-Baptiste Onofré wrote: > Hi beamers, > > I updated the P

Re: Possibility for GSoC 2016

Hi JB, I'm really happy to work with Beam project for my this year's GSoC, I hope I can bear the effort of refactoring the runner to fit the new runner API. And May be I can help with the "new" runner API also. According to the timeline, the date for students to begin coding for their GSoC projec

Re: Renaming process: first step Maven coordonates

Hi beamers, I updated the PR according to your comments. I have couple of points I want to discuss: 1. All modules use the same groupId (org.apache.beam). In order to have a cleaner structure on the Maven repo, I wonder if it's not better to have different groupId depending of the artifacts.

Re: Renaming process: first step Maven coordonates

Hi Davor, thank you so much for your comments. I'm updating the PR according to your PR (and will provide explanation to some changes). Thanks dude ! Regards JB On 03/21/2016 06:29 AM, Davor Bonaci wrote: I left a few comments on PR #46. Thanks JB for doing this; a clear improvement. On M

Re: Possibility for GSoC 2016

Hi beamers, I fully agree with Davor: I think it's better to wait the "new" runner API before starting new runners. So, I'm afraid it will be tricky (in term of timing) for GSOC. Anyway, if you want to start on the runner (and accept the effort to upgrade/change to the Runner API), then I wo

Re: [HEADS UP] Renaming/polishing

Thinking about this, I would prefer 0.1.0-incubating-SNAPSHOT, as it will also indicate that the SNAPSHOTs (on Central) are from incubating. So, I would propose 0.1.0-incubating-SNAPSHOT (and 0.1.0-incubating for the release). Regards JB On 03/21/2016 12:32 PM, Maximilian Michels wrote: If

Re: [HEADS UP] Renaming/polishing

Hi Max, We need the incubating classifier only for releases: it's a requirement to notify that the project release is not yet a TLP project but an incubator/podling release. Regarding this, I'm +1 for 0.1.0-SNAPSHOT (and so 0.1.0-incubating for the first release). Regards JB On 03/21/2016

Re: [HEADS UP] Renaming/polishing

If we can leave out the "incubating" qualifier for development, I would very much appreciate that. I like Davor's proposal to append it only once we release. Apart from the improved Maven version semantics, it would incorporate the fact that incubating projects are only required to include the "inc

Re: Committer workflow

Hi Max, fully agree, my previous e-mail was more my own mea culpa as I pushed a change without PR (of course it was minor change as it concerned the legal NOTICE and DISCLAINER files ;)). So, my intend was just a global reminder, including myself ;) Regards JB On 03/21/2016 10:50 AM, Maximi

Re: Committer workflow

Hi JB, If I remember correctly from the past discussions, we agreed that we want a PR-review process for all code changes which are important/major or break the API in some way. I wholeheartedly agree with this process. In addition, committers preserve the right to provide small fixes which do no

[PROPOSAL] Create new Jira IO component

Hi beamers ! I started to work on the directories re-organization, and especially, I'm moving the different IOs in their own folder. As we can bet on contribution on IO, maybe it would make sense to create the IO component in Jira. Thoughts ? Regards JB -- Jean-Baptiste Onofré jbono...@apa