Re: [VOTE] groupId/artifactId naming & layout

2016-06-03 Thread Jean-Baptiste Onofré
But yes, you are right, I should have used "[DISCUSS]" in the subject instead of "[VOTE]". Sorry about that. Regards JB On 06/03/2016 07:44 PM, Amit Sela wrote: I think that the [VOTE] tagging was a bit confusing, at least for me. I thought it was a formal vote.. On Fri, Jun 3, 2016, 20:29 J

Re: [VOTE] groupId/artifactId naming & layout

2016-06-03 Thread Jean-Baptiste Onofré
It's a formal vote to choose between the two options proposed. If you see another option, you can add it in the vote thread. Regards JB On 06/03/2016 07:44 PM, Amit Sela wrote: I think that the [VOTE] tagging was a bit confusing, at least for me. I thought it was a formal vote.. On Fri, Jun 3

Re: [VOTE] groupId/artifactId naming & layout

2016-06-03 Thread Amit Sela
I think that the [VOTE] tagging was a bit confusing, at least for me. I thought it was a formal vote.. On Fri, Jun 3, 2016, 20:29 Jean-Baptiste Onofré wrote: > Absolutely. > > The vote/discussion can "extended" to other options (even if I don't see > obvious right now). No worries at all. > > Re

Re: [VOTE] groupId/artifactId naming & layout

2016-06-03 Thread Jean-Baptiste Onofré
Absolutely. The vote/discussion can "extended" to other options (even if I don't see obvious right now). No worries at all. Regards JB On 06/03/2016 07:25 PM, Frances Perry wrote: Totally agree on discussing this ;-) I think Davor was just suggesting we lay out all options and understand the

Re: [VOTE] groupId/artifactId naming & layout

2016-06-03 Thread Frances Perry
Totally agree on discussing this ;-) I think Davor was just suggesting we lay out all options and understand them before calling for a vote between them. On Fri, Jun 3, 2016 at 10:19 AM, Jean-Baptiste Onofré wrote: > The purpose of the vote is to get a consensus actually. > > We have two options

Re: [VOTE] groupId/artifactId naming & layout

2016-06-03 Thread Jean-Baptiste Onofré
The purpose of the vote is to get a consensus actually. We have two options expressed on the mailing list: the current "layout" is good IMHO but all don't agree. So, let's put things on the table and move forward. The vote is a way of discussing. It's not a vote for the release, it's a vote/di

Re: Apache Beam for Python

2016-06-03 Thread Amit Sela
Welcome Python people ;) I know a few people who've been waiting for this one! On Fri, Jun 3, 2016, 19:53 Davor Bonaci wrote: > Welcome Python SDK, as well as Silviu, Charles, Ahmet and Chamikara! > > On Fri, Jun 3, 2016 at 7:07 AM, Jean-Baptiste Onofré > wrote: > > > Absolutely ;) > > > > > >

Re: 0.1.0-incubating release

2016-06-03 Thread Thomas Weise
Another consideration for potential future packaging/distribution solutions is how the artifacts line up as files in a flat directory. For that it may be good to have a common prefix in the artifactId and unique artifactId. The name for the source archive (when relying on ASF parent POM) can also

Re: Apache Beam for Python

2016-06-03 Thread Davor Bonaci
Welcome Python SDK, as well as Silviu, Charles, Ahmet and Chamikara! On Fri, Jun 3, 2016 at 7:07 AM, Jean-Baptiste Onofré wrote: > Absolutely ;) > > > On 06/03/2016 03:51 PM, James Malone wrote: > >> Hey Silviu! >> >> I think JB is proposing we create a python directory in the sdks directory >>

Re: [VOTE] groupId/artifactId naming & layout

2016-06-03 Thread Davor Bonaci
This is not a great vote proposal for several reasons: * "Use the current layout" is ambiguous, because it is inconsistent (it is now partly flat and party hierarchical). * Getting the outcome won't move us much closer to the resolution, given that there are several sub-variants in each option. * W

Re: 0.1.0-incubating release

2016-06-03 Thread Davor Bonaci
BEAM-315 is definitely important. Normally, I'd always advocate for holding the release to pick that fix. For the very first release, however, I'd prefer to proceed to get something out there and test the process. As you said, we can address this rather quickly once we have the fix merged in. In t

Re: Apache Beam for Python

2016-06-03 Thread Silviu Calinoiu
Hi James, Yes we fit right into sdks/python. I will have out a doc/proposal about where things go so people can comment. It will follow closely the Beam repository guidelines. Thanks, Silviu On Fri, Jun 3, 2016 at 6:51 AM, James Malone wrote: > Hey Silviu! > > I think JB is proposing we create a

Re: Apache Beam for Python

2016-06-03 Thread Jean-Baptiste Onofré
Absolutely ;) On 06/03/2016 03:51 PM, James Malone wrote: Hey Silviu! I think JB is proposing we create a python directory in the sdks directory in the root repository (and modify the configuration files accordingly): https://github.com/apache/incubator-beam/tree/master/sdks This Beam doc

Re: Apache Beam for Python

2016-06-03 Thread Jean-Baptiste Onofré
Awesome ! Thanks ! Regards JB On 06/03/2016 04:02 PM, Silviu Calinoiu wrote: Hi James, Yes we fit right into sdks/python. I will have out a doc/proposal about where things go so people can comment. It will follow closely the Beam repository guidelines. Thanks, Silviu On Fri, Jun 3, 2016 at 6:5

Re: Apache Beam for Python

2016-06-03 Thread Jean-Baptiste Onofré
I'm more proposing just a folder containing Pythong SDK, not necessary part of the Maven reactor. Regards JB On 06/03/2016 03:34 PM, Silviu Calinoiu wrote: Hi JB, Thanks for the welcome! I come from the Python land so I am not quite familiar with Maven. What do you mean by a Maven module? You

Re: Apache Beam for Python

2016-06-03 Thread James Malone
Hey Silviu! I think JB is proposing we create a python directory in the sdks directory in the root repository (and modify the configuration files accordingly): https://github.com/apache/incubator-beam/tree/master/sdks This Beam document here titled "Apache Beam (Incubating): Repository Struct

Re: Apache Beam for Python

2016-06-03 Thread Silviu Calinoiu
Hi JB, Thanks for the welcome! I come from the Python land so I am not quite familiar with Maven. What do you mean by a Maven module? You mean an artifact so you can install things? In Python, people are used to packages downloaded from PyPI (pypi.python.org -- which is sort of Maven for Python).

Re: Apache Beam for Python

2016-06-03 Thread Jean-Baptiste Onofré
Hi Silviu, thanks for detailed update and great work ! I would advice to create a: sdks/python Maven module to store the Python SDK. WDYT ? By the way, welcome aboard and great to have you all guys in the team ! Regards JB On 06/03/2016 03:13 PM, Silviu Calinoiu wrote: Hi all, My name is

Apache Beam for Python

2016-06-03 Thread Silviu Calinoiu
Hi all, My name is Silviu Calinoiu and I am a member of the Cloud Dataflow team working on the Python SDK. As the original Beam proposal ( https://wiki.apache.org/incubator/BeamProposal) mentioned, we have been planning to merge the Python SDK into Beam. The Python SDK is in an early stage of dev

Re: [VOTE] groupId/artifactId naming & layout

2016-06-03 Thread Amit Sela
+1 for Option2 On Fri, Jun 3, 2016 at 2:09 PM Jean-Baptiste Onofré wrote: > As said in my previous e-mail, just proposed PR #416. > > Let's start a vote for groupId and artifactId naming. > > [ ] Option1: use the current layout (multiple groupId, artifactId > relative to groupId) > [ ] Option2:

[VOTE] groupId/artifactId naming & layout

2016-06-03 Thread Jean-Baptiste Onofré
As said in my previous e-mail, just proposed PR #416. Let's start a vote for groupId and artifactId naming. [ ] Option1: use the current layout (multiple groupId, artifactId relative to groupId) [ ] Option2: use unique org.apache.beam groupId and rename artifactId with a prefix (beam-parent/ap

Re: 0.1.0-incubating release

2016-06-03 Thread Jean-Baptiste Onofré
Hi Max, I discussed with Davor yesterday. Basically, I proposed: 1. To rename all parent with a prefix (beam-parent, flink-runner-parent, spark-runner-parent, etc). 2. For the groupId, I prefer to use different groupId, it's clearer to me, and it's exactly the usage of the groupId. Some projec

Re: 0.1.0-incubating release

2016-06-03 Thread Amit Sela
+1 on Max's comment on naming. I prefer spark-runner and beam-parent as well. On Fri, Jun 3, 2016, 12:03 Maximilian Michels wrote: > Thanks for getting us ready for the first release, Davor! We would > like to fix BEAM-315 next week. Is there already a timeline for the > first release? If so, we

Re: 0.1.0-incubating release

2016-06-03 Thread Maximilian Michels
Thanks for getting us ready for the first release, Davor! We would like to fix BEAM-315 next week. Is there already a timeline for the first release? If so, we could also address this in a minor release. Releasing often will give us some experience with our release process :) I would like everyone

Re: Serialization for org.apache.beam.sdk.util.WindowedValue$*

2016-06-03 Thread Thomas Weise
Amit, Thanks for this pointer as well, CoderHelpers helps indeed! Thomas On Thu, Jun 2, 2016 at 12:51 PM, Amit Sela wrote: > Oh sorry, of course I meant Thomas Groh in my previous email.. But @Thomas > Weise this example > < > https://github.com/apache/incubator-beam/blob/master/runners/spark/

Re: Serialization for org.apache.beam.sdk.util.WindowedValue$*

2016-06-03 Thread Thomas Weise
Thanks, works like a charm! For such hidden gems there should be a Beam runner newbie guide ;-) Thomas On Thu, Jun 2, 2016 at 11:59 AM, Thomas Groh wrote: > The Beam Model ensures that all PCollections have a Coder; the PCollection > Coder is the standard way to materialize the elements of a >