Re: [ANNOUCEMENT] New Foundation members!

[Robert Burke]

Congratulations!

On Sat, 31 Mar 2018 at 11:53 Ekrem Aksoy  wrote:

> Congrats!
>
> On Sat, Mar 31, 2018 at 2:08 AM, Davor Bonaci  wrote:
>
>> Now that this is public... please join me in welcoming three newly
>> elected members of the Apache Software Foundation with ties to this
>> community, who were elected during the most recent Members' Meeting.
>>
>> * Ismaël Mejía (Beam PMC)
>>
>> * Josh Wills (Crunch Chair; Beam, DataFu PMC)
>>
>> * Holden Karau (Spark, SystemML PMC; Mahout, Subversion committer; Beam
>> contributor)
>>
>> These individuals demonstrated merit in Foundation's growth, evolution,
>> and progress. They were recognized, nominated, and elected by existing
>> membership for their significant impact to the Foundation as a whole, such
>> as the roots of project-related and cross-project activities.
>>
>> As members, they now become legal owners and shareholders of the
>> Foundation. They can vote for the Board, incubate new projects, nominate
>> new members, participate in any PMC-private discussions, and contribute to
>> any project.
>>
>> (For the Beam community, this election nearly doubles the number of
>> Foundation members. The new members are joining Jean-Baptiste Onofré,
>> Stephan Ewen, Romain Manni-Bucau and myself in this role.)
>>
>> I'm happy to be able to call all three of you my fellow members.
>> Congratulations!
>>
>> Davor
>>
>
>

Re: [ANNOUCEMENT] New Foundation members!

[Ekrem Aksoy]

Congrats!

On Sat, Mar 31, 2018 at 2:08 AM, Davor Bonaci  wrote:

> Now that this is public... please join me in welcoming three newly elected
> members of the Apache Software Foundation with ties to this community, who
> were elected during the most recent Members' Meeting.
>
> * Ismaël Mejía (Beam PMC)
>
> * Josh Wills (Crunch Chair; Beam, DataFu PMC)
>
> * Holden Karau (Spark, SystemML PMC; Mahout, Subversion committer; Beam
> contributor)
>
> These individuals demonstrated merit in Foundation's growth, evolution,
> and progress. They were recognized, nominated, and elected by existing
> membership for their significant impact to the Foundation as a whole, such
> as the roots of project-related and cross-project activities.
>
> As members, they now become legal owners and shareholders of the
> Foundation. They can vote for the Board, incubate new projects, nominate
> new members, participate in any PMC-private discussions, and contribute to
> any project.
>
> (For the Beam community, this election nearly doubles the number of
> Foundation members. The new members are joining Jean-Baptiste Onofré,
> Stephan Ewen, Romain Manni-Bucau and myself in this role.)
>
> I'm happy to be able to call all three of you my fellow members.
> Congratulations!
>
> Davor
>

Re: [ANNOUCEMENT] New Foundation members!

[Łukasz Gajowy]

Congratulations! :)

Best regards,
Łukasz

2018-03-31 7:54 GMT+02:00 Raghunandana Jayarama Reddy <
rjayarama...@cs.stonybrook.edu>:

> Congratulations!
>
> Best,
> Raghu
>
> On Sat, Mar 31, 2018 at 1:17 AM Jean-Baptiste Onofré 
> wrote:
>
>> Congrats !
>>
>> Regards
>> JB
>> Le 31 mars 2018, à 01:09, Davor Bonaci  a écrit:
>>>
>>> Now that this is public... please join me in welcoming three newly
>>> elected members of the Apache Software Foundation with ties to this
>>> community, who were elected during the most recent Members' Meeting.
>>>
>>> * Ismaël Mejía (Beam PMC)
>>>
>>> * Josh Wills (Crunch Chair; Beam, DataFu PMC)
>>>
>>> * Holden Karau (Spark, SystemML PMC; Mahout, Subversion committer; Beam
>>> contributor)
>>>
>>> These individuals demonstrated merit in Foundation's growth, evolution,
>>> and progress. They were recognized, nominated, and elected by existing
>>> membership for their significant impact to the Foundation as a whole, such
>>> as the roots of project-related and cross-project activities.
>>>
>>> As members, they now become legal owners and shareholders of the
>>> Foundation. They can vote for the Board, incubate new projects, nominate
>>> new members, participate in any PMC-private discussions, and contribute to
>>> any project.
>>>
>>> (For the Beam community, this election nearly doubles the number of
>>> Foundation members. The new members are joining Jean-Baptiste Onofré,
>>> Stephan Ewen, Romain Manni-Bucau and myself in this role.)
>>>
>>> I'm happy to be able to call all three of you my fellow members.
>>> Congratulations!
>>>
>>> Davor
>>>
>>

Re: Gradle migration fixit: April 3

[Romain Manni-Bucau]

Thanks for the doc Lukasz

So we do 3 groups?

1. Jenkins
2. Release
3. Outputs/behavior (shades, jars, javadoc, ensure concurrency is not
hardcoded, ...)

?

Happy to be in 3 or 2 (by preference) if you agree.

Le 31 mars 2018 10:54, "Aljoscha Krettek"  a écrit :

> Thanks Luke, that was helpful!
>
> I've been playing around with gradle a bit and now have a question: By
> default the shadow plugin will include dependencies that are "compile" or
> "runtime" (compile dependencies are by default also runtime dependencies)
> into the shaded jar. Is that correct? The reason our build doesn't do that
> is that we have custom include/exclude rules that prevent that from
> happening. I just noticed that when I change the Flink dependencies from
> shadow to compile in the Flink Runner build file the shaded jar will not
> contain the Flink classes.
>
> Btw, I'm not saying that I want to include the Flink classes, I was just
> messing around with things to get a feeling for how things work.
>
> On 31. Mar 2018, at 00:15, Lukasz Cwik  wrote:
>
> I have started a doc[1] containing a Gradle primer to help people be more
> productive during the fixit day. Feel free to add/update comments and
> content.
>
> 1: https://docs.google.com/document/d/1EiTwEMD8FNhU4Ok6jthASpmK3-
> 1hiAYzVTrdl8qBLrs/edit?usp=sharing
>
> On Fri, Mar 30, 2018 at 8:33 AM Reuven Lax  wrote:
>
>> Here is the Kanban board tracking all the current tasks. It looks like 7
>> of them have already been closed over the past two days, so we're down to
>> 19!
>>
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=242
>>
>> On Thu, Mar 29, 2018 at 11:21 PM Romain Manni-Bucau <
>> rmannibu...@gmail.com> wrote:
>>
>>> Yep - sorry if it was unclear. I know linux distro often do it (never
>>> understood why though).
>>>
>>>
>>> Romain Manni-Bucau
>>> @rmannibucau  |  Blog
>>>  | Old Blog
>>>  | Github
>>>  | LinkedIn
>>>  | Book
>>> 
>>>
>>> 2018-03-30 7:54 GMT+02:00 Reuven Lax :
>>>


 On Thu, Mar 29, 2018 at 10:28 PM Romain Manni-Bucau <
 rmannibu...@gmail.com> wrote:

> It was more about the drop of poms (same case as you for dataflow).
>

 Ah - you're worried that some external users are building directly from
 the poms rather than using the published artifact.

 I think this is a valid concern, and I agree we should announce on
 users@ before (probably some time before) deleting the poms  .


> On that there is a missing but highly important task: gradle to mvn
> descriptors. All the one I saw were corrupted poms so we must take care of
> that as part of the release work (I can work on it on the 3rd if you 
> want).
>
> Le 29 mars 2018 23:36, "Reuven Lax"  a écrit :
>
>> I don't mind notifying users@, but this does seem more interesting
>> for dev@. We will continue to publish Maven artifacts from our
>> Gradle build, so users are still free to use either Maven or Gradle.
>>
>> That being said, if this is interesting to users@ we can notify them
>> as well.
>>
>> Reuven
>>
>> On Thu, Mar 29, 2018 at 1:45 PM Romain Manni-Bucau <
>> rmannibu...@gmail.com> wrote:
>>
>>>
>>>
>>> Le 29 mars 2018 21:20, "Reuven Lax"  a écrit :
>>>
>>>
>>>
>>> On Thu, Mar 29, 2018 at 12:17 PM Romain Manni-Bucau <
>>> rmannibu...@gmail.com> wrote:
>>>


 Le 29 mars 2018 20:35, "Reuven Lax"  a écrit :

 1. As Luke already mentioned, we should first have a subgoal of the
 Gradle jenkins jobs being equivalent to the Maven jobs. Hopefully 
 toward
 the end of the day, we'll make this change.

 2. Let's see how much progress we make on the third. There is a
 side problem we have here at Google - we have an internal product 
 called
 Dataflow built on Beam, and the Dataflow build still depends on those 
 pom
 files. I would request leaving the pom files around just a little bit
 longer even if Beam no longer needs them, just so that we don't breakd
 Dataflow (and I think we would do this for any community members with a
 similar issue). We will prioritize moving Dataflow ASAP, it's just 
 that the
 people who will do so will also be in the April 3 fixit so it can't 
 happen
 until after. I think the delay should only be one or two weeks to 
 delete
 the poms (assuming that Beam is ready at the end of the day).


 Can you try to put a date of that and we will communicate on it
 publicly if anyone else does (i dont think but i didnt expect 

Re: Gradle migration fixit: April 3

[Aljoscha Krettek]

Thanks Luke, that was helpful!

I've been playing around with gradle a bit and now have a question: By default 
the shadow plugin will include dependencies that are "compile" or "runtime" 
(compile dependencies are by default also runtime dependencies) into the shaded 
jar. Is that correct? The reason our build doesn't do that is that we have 
custom include/exclude rules that prevent that from happening. I just noticed 
that when I change the Flink dependencies from shadow to compile in the Flink 
Runner build file the shaded jar will not contain the Flink classes.

Btw, I'm not saying that I want to include the Flink classes, I was just 
messing around with things to get a feeling for how things work.

> On 31. Mar 2018, at 00:15, Lukasz Cwik  wrote:
> 
> I have started a doc[1] containing a Gradle primer to help people be more 
> productive during the fixit day. Feel free to add/update comments and content.
> 
> 1: 
> https://docs.google.com/document/d/1EiTwEMD8FNhU4Ok6jthASpmK3-1hiAYzVTrdl8qBLrs/edit?usp=sharing
>  
> 
> On Fri, Mar 30, 2018 at 8:33 AM Reuven Lax  > wrote:
> Here is the Kanban board tracking all the current tasks. It looks like 7 of 
> them have already been closed over the past two days, so we're down to 19!
> 
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=242 
> 
> 
> On Thu, Mar 29, 2018 at 11:21 PM Romain Manni-Bucau  > wrote:
> Yep - sorry if it was unclear. I know linux distro often do it (never 
> understood why though).
> 
> 
> Romain Manni-Bucau
> @rmannibucau  |  Blog 
>  | Old Blog 
>  | Github  
> | LinkedIn  | Book 
> 
> 2018-03-30 7:54 GMT+02:00 Reuven Lax  >:
> 
> 
> On Thu, Mar 29, 2018 at 10:28 PM Romain Manni-Bucau  > wrote:
> It was more about the drop of poms (same case as you for dataflow).
> 
> Ah - you're worried that some external users are building directly from the 
> poms rather than using the published artifact.
> 
> I think this is a valid concern, and I agree we should announce on users@ 
> before (probably some time before) deleting the poms  .
> 
> 
> On that there is a missing but highly important task: gradle to mvn 
> descriptors. All the one I saw were corrupted poms so we must take care of 
> that as part of the release work (I can work on it on the 3rd if you want).
> 
> Le 29 mars 2018 23:36, "Reuven Lax"  > a écrit :
> I don't mind notifying users@, but this does seem more interesting for dev@. 
> We will continue to publish Maven artifacts from our Gradle build, so users 
> are still free to use either Maven or Gradle.
> 
> That being said, if this is interesting to users@ we can notify them as well.
> 
> Reuven
> 
> On Thu, Mar 29, 2018 at 1:45 PM Romain Manni-Bucau  > wrote:
> 
> 
> Le 29 mars 2018 21:20, "Reuven Lax"  > a écrit :
> 
> 
> On Thu, Mar 29, 2018 at 12:17 PM Romain Manni-Bucau  > wrote:
> 
> 
> Le 29 mars 2018 20:35, "Reuven Lax"  > a écrit :
> 1. As Luke already mentioned, we should first have a subgoal of the Gradle 
> jenkins jobs being equivalent to the Maven jobs. Hopefully toward the end of 
> the day, we'll make this change. 
> 
> 2. Let's see how much progress we make on the third. There is a side problem 
> we have here at Google - we have an internal product called Dataflow built on 
> Beam, and the Dataflow build still depends on those pom files. I would 
> request leaving the pom files around just a little bit longer even if Beam no 
> longer needs them, just so that we don't breakd Dataflow (and I think we 
> would do this for any community members with a similar issue). We will 
> prioritize moving Dataflow ASAP, it's just that the people who will do so 
> will also be in the April 3 fixit so it can't happen until after. I think the 
> delay should only be one or two weeks to delete the poms (assuming that Beam 
> is ready at the end of the day).
> 
> Can you try to put a date of that and we will communicate on it publicly if 
> anyone else does (i dont think but i didnt expect you to do it too ;)).
> 
> What do you mean by communicate publicly? The dev list is already cced here :)
> 
> Misses a (not too far) date and users@ pby to be safe ;)
> 
> 
> 
> 
> 
> On Thu, Mar 29, 2018 at 5:20 AM Romain Manni-Bucau  > wrote:
> Hi Reuven, a few questions:
> 
> 1. any inputs on how we can work on the jenkins part? Do we tes

Build failed in Jenkins: beam_Release_NightlySnapshot #730

[Apache Jenkins Server]

See 


Changes:

[wcn] Fix golint issues

[tgroh] Reintroduce MetricName#name[space]

[XuMingmin] [BEAM-591] Support custom timestamps & CreateTime support (#4935)

[lcwik] [BEAM-3249] Allow for re-use of dependencies within other projects by

[robertwb] Optimize reshuffle.

[robertwb] Add PipelineRunner.run_async for non-blocking execution.

[boyuanz] Add distribution_counter_microbenchmark to apache_beam.tools.utils

--
[...truncated 2.79 MB...]
2018-03-31T08:51:16.536 [INFO] Excluding 
org.glassfish.jersey.media:jersey-media-jaxb:jar:2.22.2 from the shaded jar.
2018-03-31T08:51:16.536 [INFO] Excluding 
org.glassfish.jersey.containers:jersey-container-servlet:jar:2.22.2 from the 
shaded jar.
2018-03-31T08:51:16.536 [INFO] Excluding 
org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.22.2 from 
the shaded jar.
2018-03-31T08:51:16.536 [INFO] Excluding io.netty:netty-all:jar:4.0.43.Final 
from the shaded jar.
2018-03-31T08:51:16.536 [INFO] Excluding io.netty:netty:jar:3.9.9.Final from 
the shaded jar.
2018-03-31T08:51:16.536 [INFO] Excluding 
io.dropwizard.metrics:metrics-jvm:jar:3.1.2 from the shaded jar.
2018-03-31T08:51:16.536 [INFO] Excluding 
io.dropwizard.metrics:metrics-json:jar:3.1.2 from the shaded jar.
2018-03-31T08:51:16.536 [INFO] Excluding 
io.dropwizard.metrics:metrics-graphite:jar:3.1.2 from the shaded jar.
2018-03-31T08:51:16.536 [INFO] Excluding org.apache.ivy:ivy:jar:2.4.0 from the 
shaded jar.
2018-03-31T08:51:16.536 [INFO] Excluding oro:oro:jar:2.0.8 from the shaded jar.
2018-03-31T08:51:16.536 [INFO] Excluding net.razorvine:pyrolite:jar:4.13 from 
the shaded jar.
2018-03-31T08:51:16.536 [INFO] Excluding net.sf.py4j:py4j:jar:0.10.4 from the 
shaded jar.
2018-03-31T08:51:16.536 [INFO] Excluding 
org.apache.spark:spark-tags_2.11:jar:2.2.1 from the shaded jar.
2018-03-31T08:51:16.536 [INFO] Excluding 
org.apache.commons:commons-crypto:jar:1.0.0 from the shaded jar.
2018-03-31T08:51:16.536 [INFO] Excluding 
org.spark-project.spark:unused:jar:1.0.0 from the shaded jar.
2018-03-31T08:51:16.536 [INFO] Excluding 
org.apache.spark:spark-streaming_2.11:jar:2.2.1 from the shaded jar.
2018-03-31T08:51:17.531 [INFO] Replacing original artifact with shaded artifact.
2018-03-31T08:51:17.643 [INFO] 
2018-03-31T08:51:17.643 [INFO] --- maven-assembly-plugin:3.1.0:single 
(source-release-assembly) @ beam-sdks-java-javadoc ---
2018-03-31T08:51:17.646 [INFO] Skipping the assembly in this project because 
it's not the Execution Root
2018-03-31T08:51:17.754 [INFO] 
2018-03-31T08:51:17.754 [INFO] --- maven-source-plugin:3.0.1:jar-no-fork 
(attach-sources) @ beam-sdks-java-javadoc ---
2018-03-31T08:51:17.864 [INFO] 
2018-03-31T08:51:17.864 [INFO] --- maven-source-plugin:3.0.1:test-jar-no-fork 
(attach-test-sources) @ beam-sdks-java-javadoc ---
2018-03-31T08:51:17.972 [INFO] 
2018-03-31T08:51:17.972 [INFO] --- maven-javadoc-plugin:3.0.0-M1:jar 
(attach-javadocs) @ beam-sdks-java-javadoc ---
2018-03-31T08:51:17.975 [INFO] Not executing Javadoc as the project is not a 
Java classpath-capable package
2018-03-31T08:51:18.084 [INFO] 
2018-03-31T08:51:18.084 [INFO] --- 
reproducible-build-maven-plugin:0.4:strip-jar (default) @ 
beam-sdks-java-javadoc ---
2018-03-31T08:51:18.085 [INFO] Stripping 

2018-03-31T08:51:18.256 [INFO] 
2018-03-31T08:51:18.256 [INFO] --- maven-dependency-plugin:3.0.2:analyze-only 
(default) @ beam-sdks-java-javadoc ---
2018-03-31T08:51:18.257 [INFO] Skipping plugin execution
2018-03-31T08:51:18.403 [INFO] 
2018-03-31T08:51:18.403 [INFO] --- maven-install-plugin:2.5.2:install 
(default-install) @ beam-sdks-java-javadoc ---
2018-03-31T08:51:18.404 [INFO] Installing 

 to 

2018-03-31T08:51:18.469 [INFO] Installing 

 to 

2018-03-31T08:51:18.595 [INFO] 
2018-03-31T08:51:18.595 [INFO] --- maven-deploy-plugin:2.8.2:deploy 
(default-deploy) @ beam-sdks-java-javadoc ---
2018-03-31T08:51:18.599 [INFO] Downloading from apache.snapshots.https: 
https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-sdks-java-javadoc/2.5.0-SNAPSHOT/maven-metadata.xml
2018-03-31T08:51:18.825 [INFO] Downloaded from apache.snapshots.https: 
https://repository.apache.org/content/repositories/snapshots/org/apache/beam/be