Re: Gradle Status: Migrated!

2018-05-01 Thread Henning Rohde
JB - for your comparison, please also omit cross-compiling all the Go examples because they are only built using Gradle. On Tue, May 1, 2018 at 8:59 AM Jean-Baptiste Onofré wrote: > Thanks for the update Kenn, that makes sense. > > I'm checking the artifacts generated by Gradle right now. > >

Re: Gradle Status: Migrated!

2018-05-01 Thread Jean-Baptiste Onofré
Thanks for the update Kenn, that makes sense. I'm checking the artifacts generated by Gradle right now. Regards JB On 01/05/2018 17:42, Kenneth Knowles wrote: Raw execution time for tasks from clean is not the only thing to test. I would say it is not even important. Try these from clean:  

Re: Gradle Status: Migrated!

2018-05-01 Thread Kenneth Knowles
Raw execution time for tasks from clean is not the only thing to test. I would say it is not even important. Try these from clean: - Gradle: ./gradlew :beam-sdks-java-io-mongodb:test && ./gradlew : beam-sdks-java-io-mongodb:test - Maven: mvn -pl sdks/java/io/mongodb test -am && mvn -pl sdks/java

Re: Gradle Status: Migrated!

2018-05-01 Thread Jean-Baptiste Onofré
Thanks, for me, Maven 3.5.2 takes quite the same time than Gradle (using the wrapper). It's maybe related to my environment. Anyway, I'm doing a complete build review both in term of building time, and equivalence (artifacts publishing, test, plugin execution). I will provide an update soon.

Re: Gradle Status: Migrated!

2018-05-01 Thread Reuven Lax
Luke did gather data which showed that on our Jenkins executors the Gradle build was much faster than the Maven build. Also right now we have incremental builds turned off, but once we're confident enough to enable them (at least for local development) that will often drop build times a lot. On Tu

Re: Gradle Status: Migrated!

2018-05-01 Thread Jean-Baptiste Onofré
By the way, I'm curious: did someone evaluate the build time gap between Maven and Gradle ? One of the main reason to migrate to Gradle was the inc build and build time. The builds I have launched are quite the same in duration. I will do deeper tests to evaluate the gap. Regards JB On 05/01/2018

Re: Gradle Status: Migrated!

2018-05-01 Thread Łukasz Gajowy
Hi Scott, thanks for the update! Just a clarification about IO performance tests: those were fully migrated in Beam and all task necessary for running them are there but Jenkins jobs still run mvn commands. This is due the fact that PerfkitBenchmarker code (which is invoked by Jenkins and construc

Re: Gradle Status: Migrated!

2018-05-01 Thread Romain Manni-Bucau
Hi Scott While https://issues.apache.org/jira/plugins/servlet/mobile#issue/BEAM-4057 is open, gradle is a concurrent of maven but maven must stay the default build tool cause gradle breaks users. Le 1 mai 2018 01:59, "Scott Wegner" a écrit : > Many many of you have been hacking diligently on t

Gradle Status: Migrated!

2018-04-30 Thread Scott Wegner
Many many of you have been hacking diligently on the Gradle build, and I'm happy to announce that we now have a fully-functioning Gradle build! There's been a ton of progress since our last update [1]: * Improved nightly snapshot release [2] * Improve runner quickstarts [5] [11] * Python post-comm

Re: Gradle Status [April 11]

2018-04-18 Thread Nathan Fisher
Great thanks for that! Makes perfect sense. On Wed, Apr 11, 2018 at 7:59 PM, Kenneth Knowles wrote: > There are plenty of technical hurdles since Bazel is best suited to an > isolated monorepo. We could probably have overcome those problems with > effort. I got through a few of them in the short

Gradle Status [April 17th]

2018-04-17 Thread Scott Wegner
Gradle migration continues. Our last update was on April 13th, and since then there has been significant progress: Release artifacts: * Upgrade to latest Gradle PR5104 * Remove evaluationDependsOn and use shaded test jars PR5117 * Nightly java snapshot release fixed by PR5136 and PR 5142 and now p

Re: Gradle Status [April 6]

2018-04-15 Thread Romain Manni-Bucau
Well it is yes and no (build -> fails, build -> succeeds is weird for solething harnessing your code). Anyway, idea forces a clean to ensure the test you select runs. Le 15 avr. 2018 19:58, "Kenneth Knowles" a écrit : > Caching success/failure of a test is legitimate useful behavior. That > sai

Re: Gradle Status [April 6]

2018-04-15 Thread Kenneth Knowles
Caching success/failure of a test is legitimate useful behavior. That said, I'm not sure how it interacts with Idea. Kenn On Sun, Apr 15, 2018 at 12:23 AM Romain Manni-Bucau wrote: > If you dont the diff is empty and gradle runs nothing, no? Saw it with > gradle 4.6 > > Le 15 avr. 2018 00:49, "

Re: Gradle Status [April 6]

2018-04-15 Thread Romain Manni-Bucau
If you dont the diff is empty and gradle runs nothing, no? Saw it with gradle 4.6 Le 15 avr. 2018 00:49, "Kenneth Knowles" a écrit : > You shouldn't do :module:cleanTest. If that is necessary that's a major > bug in the build. > > Kenn > > On Fri, Apr 13, 2018 at 11:46 PM Romain Manni-Bucau > w

Re: Gradle Status [April 6]

2018-04-14 Thread Kenneth Knowles
You shouldn't do :module:cleanTest. If that is necessary that's a major bug in the build. Kenn On Fri, Apr 13, 2018 at 11:46 PM Romain Manni-Bucau wrote: > There is a fake module xxx_test which should have the right classpath but > since idea compilation is messed up you will still have to run

Re: Gradle Status [April 6]

2018-04-13 Thread Romain Manni-Bucau
There is a fake module xxx_test which should have the right classpath but since idea compilation is messed up you will still have to run :module:cleanTest :module:test --tests org...MyTest.myMethod, even with idea which leads to the same latency as the command line :( Le 13 avr. 2018 22:23, "Eugen

Gradle Status [April 13]

2018-04-13 Thread Pablo Estrada
Hello everyone, here's a high-level summary of Gradle migration progress since the last update. The progress this Friday was a bit slower, though we expect to continue at a good pace next week. Thanks again to everyone that is putting time into the migration. Ongoing work / PRs: * Migration of Jav

Re: Gradle Status [April 6]

2018-04-13 Thread Eugene Kirpichov
While we're talking about running tests in IntelliJ with Gradle... Anybody got advice on how to run a single NeedsRunner test in sdks-java-core, say, ParDoTest? With Maven, I used to just run the test in IntelliJ and specify "runners-direct-java" as the classpath; with Gradle, the best I got is to

Re: Gradle Status [April 6]

2018-04-13 Thread Reuven Lax
Is there a Jira for this 3 second delay? Also you're initial complaint was not about the 3 second delay, so it wasn't clear that's what you were complaining about. Reuven On Fri, Apr 13, 2018 at 4:42 AM Romain Manni-Bucau wrote: > When you launch a test with gradle runner it launches gradle whi

Re: Gradle Status [April 6]

2018-04-13 Thread Daniel Oliveira
Ah, I see. I was attempting to replicate Alexey's issues with DirectRunner tests, so I wasn't checking for that 3s overhead. A quick test for me shows that I also get that several second overhead when running tests with Gradle, ranging from 3-6 sec. However, on my machine this delay is constant, re

Re: Gradle Status [April 6]

2018-04-12 Thread Romain Manni-Bucau
When you launch a test with gradle runner it launches gradle which makes loose 3s on a very fast computer and more on a slower (6 on my personal one which is already fast but not as much as my work one). We are 5 to see that regression at least. So there is a reason to not use the gradle runner if

Re: Gradle Status [April 6]

2018-04-12 Thread Reuven Lax
I also don't quite understand what your question is, and it appears like Dan spent considerable time trying to reproduce your issue. For the record, I have had no issues running tests via Gradle in IntelliJ for the past few weeks. Reuven On Thu, Apr 12, 2018 at 9:47 PM Daniel Oliveira wrote: >

Re: Gradle Status [April 6]

2018-04-12 Thread Daniel Oliveira
Sorry Romain, I'm not quite sure what you're asking. Can you clarify? On Thu, Apr 12, 2018 at 12:22 PM Romain Manni-Bucau wrote: > Well you are the only one to not have the drawbacks to use it so maybe > dont do it? I know Luke is in holidays but anyone else with the knowledge > of why we nees t

Re: Gradle Status [April 6]

2018-04-12 Thread Romain Manni-Bucau
Well you are the only one to not have the drawbacks to use it so maybe dont do it? I know Luke is in holidays but anyone else with the knowledge of why we nees that noise compared to idea native tooling/flow? Le 12 avr. 2018 20:16, "Daniel Oliveira" a écrit : > Ah, I did not. Thanks Romain. > >

Re: Gradle Status [April 6]

2018-04-12 Thread Daniel Oliveira
Ah, I did not. Thanks Romain. I tried it again, restarting in between, and still had no differences. Since it seems like there's no reason not to use "Gradle Test Runner", I'll mention it in the contributor's guide. On Thu, Apr 12, 2018 at 10:31 AM Romain Manni-Bucau wrote: > @Daniel: did you r

Re: Gradle Status [April 6]

2018-04-12 Thread Romain Manni-Bucau
@Daniel: did you restart in between? Otherwise it does nothing. One launches JunitCoreRunner from idea and the other a gradle command. Le 12 avr. 2018 19:24, "Daniel Oliveira" a écrit : > I think it depends on what exactly switching to "Gradle Test Runner" from > "Platform Test Runner" does. I t

Re: Gradle Status [April 6]

2018-04-12 Thread Daniel Oliveira
I think it depends on what exactly switching to "Gradle Test Runner" from "Platform Test Runner" does. I tried it out on my machine and they seem to act identically to each other. The IntelliJ documentation says it determines what API to use to run the tests

Re: Gradle Status [April 11]

2018-04-12 Thread Romain Manni-Bucau
just created one https://issues.apache.org/jira/browse/BEAM-4057 Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book 2018-04-12 18:15 GMT+02:00 Ahmet Altay : >> Found another blocker in current artifacts creations: there is not > pom.xml and pom.properties in META-INF.

Re: Gradle Status [April 11]

2018-04-12 Thread Ahmet Altay
> Found another blocker in current artifacts creations: there is not pom.xml and pom.properties in META-INF. This is used by tools + libraries + integrations so it is quite important to not break it Romain, is there a JIRA for this issues? If not could you create one please? On Thu, Apr 12, 2018

Re: Gradle Status [April 6]

2018-04-12 Thread Alexey Romanenko
Daniel, actually I did run it with default IDEA JUnit test runner. Then, in “Settings > Build, Execution, Deployment > Build Tools > Gradle > Runner" I selected “Gradle Test Runner” in “Run tests using” selectbox and it works ok when I run my tests with IDEA shortcuts. So, probably, we should ad

Re: Gradle Status [April 11]

2018-04-12 Thread Etienne Chauchot
Nice ! thanks Kenn Le mercredi 11 avril 2018 à 18:21 +, Kenneth Knowles a écrit : > Initial Nexmark+Gradle run is in, though a hiccup in the Spark runner + Netty > has been introduced since yesterday. > Etienne mentioned he has worked toward setting up periodic runs on all > runners, so this

Re: Gradle Status [April 6]

2018-04-12 Thread Etienne Chauchot
My test was also on version 2017, 2017.2.7 to be precise Etienne Le mercredi 11 avril 2018 à 17:01 +, Daniel Oliveira a écrit : > Hi everyone, I was the one who initially wrote the PR with Idea instructions. > I was using 2017.3 as well while writing > it so all the instructions were tested on

Re: Gradle Status [April 11]

2018-04-12 Thread Romain Manni-Bucau
Found another blocker in current artifacts creations: there is not pom.xml and pom.properties in META-INF. This is used by tools + libraries + integrations so it is quite important to not break it. Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book 2018-04-12 0:59 GMT+

Re: Gradle Status [April 11]

2018-04-11 Thread Kenneth Knowles
There are plenty of technical hurdles since Bazel is best suited to an isolated monorepo. We could probably have overcome those problems with effort. I got through a few of them in the short sprint I did. Ultimately it was a community-driven decision: more people (as in "more than just me" :-) jump

Re: Gradle Status [April 11]

2018-04-11 Thread Nathan Fisher
Is there a document or similar outlining the decision behind Gradle over Bazel? For purely intellectual curiosity I’m curious what tradeoffs/benefits were considered when evaluating the two or if it was more a matter of community/contributor familiarity. I found this thread which seems to imply i

Re: Gradle Status [April 6]

2018-04-11 Thread Lukasz Cwik
Running Dataflow validates runner tests requires you to have permission to launch jobs on the 'apache-beam-testing' project. You'll need to override dataflowProject and dataflowTempRoot with a GCP project and GCS bucket you have access to. On Wed, Apr 11, 2018 at 3:17 PM Daniel Oliveira wrote: >

Re: Gradle Status [April 6]

2018-04-11 Thread Daniel Oliveira
Alexey, are you referring to tests run with "./gradlew :beam-runners-direct-java:needsRunnerTests"? That command works fine for me in both versions of IDEA, but I believe the same tests fail if you run them directly through "./gradlew test". However, I am having issues with a bunch of validatesRun

Re: Gradle Status [April 11]

2018-04-11 Thread Kenneth Knowles
Initial Nexmark+Gradle run is in, though a hiccup in the Spark runner + Netty has been introduced since yesterday. Etienne mentioned he has worked toward setting up periodic runs on all runners, so this should help get us towards that. We'll probably prefer to build standalone fat jars for selected

Re: Gradle Status [April 6]

2018-04-11 Thread Romain Manni-Bucau
I got tests running rrconfiguring gradle (which was setup for another project but seems beam didnt like it) but latency is still "high" using gradle runner for tests (like Etienne said ~3s on an i7 with 16G vs a few ms with default idea test runner, would be great to solve that). I also find the i

Re: Gradle Status [April 6]

2018-04-11 Thread Alexey Romanenko
I’ve managed to import a project as it’s described in documentation (starting from empty project) using Idea 2018 and run unit tests successfully. For some reasons, tests, that use DirectRunner to run a pipeline, were failed. WBR, Alexey > On 11 Apr 2018, at 19:01, Daniel Oliveira wrote: > >

Gradle Status [April 11]

2018-04-11 Thread Scott Wegner
Thanks everyone for the continued effort towards the Gradle migration. As a high-level summary of our progress since Friday: we have a viable build, with a number of minor issues that we're still working out. Please take a look at the new documentation in our contribution guide and log any bugs tha

Re: Gradle Status [April 6]

2018-04-11 Thread Daniel Oliveira
Hi everyone, I was the one who initially wrote the PR with Idea instructions . I was using 2017.3 as well while writing it so all the instructions were tested on that version. I'll try testing the instructions on 2018 to see if I can reproduce the issue

Re: Gradle Status [April 6]

2018-04-11 Thread Lukasz Cwik
I use 2017.3 and it has been reliable for me. I haven't tried 2018 yet. On Wed, Apr 11, 2018 at 11:30 AM Romain Manni-Bucau wrote: > Any of you using the idea 2018? the import works for me but then it is > not as smooth as it seems for you. I'm just trying to see if it is a > procedure thing or

Re: Gradle Status [April 6]

2018-04-11 Thread Romain Manni-Bucau
Any of you using the idea 2018? the import works for me but then it is not as smooth as it seems for you. I'm just trying to see if it is a procedure thing or a version issue. Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book 2018-04-11 17:28 GMT+02:00 Kenneth Knowles

Re: Gradle Status [April 6]

2018-04-11 Thread Kenneth Knowles
The only reason I did "empty project then add a module" procedure was to get all the IntelliJ files outside the source tree. IIRC directly creating from existing sources didn't give the necessary configuration options. If you don't care about being able to `git clean` then you can do the shorter ve

Re: Gradle Status [April 6]

2018-04-11 Thread Etienne Chauchot
Hi all, I just tested gradle environment from a fresh source clone with this procedure with just a tiny change: I used "new project from existing sources" rather than create empty project and then add module.  It works fine and junit runs from intellij also work.  with gradle we pay a 2s delay (

Re: Gradle Status [April 6]

2018-04-11 Thread Romain Manni-Bucau
Not really Lukasz, having the test scoped dependencies in test scope is a minimum for any consumers (junit in compile scope is wrong for instance in several poms) having the plugin with the project constraint is important cause it is what surfaces on the net and what users are used to rely on in t

Re: Gradle Status [April 6]

2018-04-11 Thread Lukasz Cwik
Romain, it seems like you want us to generate an exact copy of the existing poms that we have. There is a lot of stuff in there related to building/testing the Apache Beam project which our users don't need (integration/test profiles, test scope dependencies, parents, ...). Users who want to build

Re: Gradle Status [April 6]

2018-04-11 Thread Romain Manni-Bucau
Hi guys, checked the snapshot pom today and there are issues: 1. no parent (so 2 is hard to handle, no properties etc) 2. too much noise informations (all the parent data should be in the parent) 3. wrong scopes (all is compile) - which is likely a leak of gradle scripts 4. missing important conf

Re: Gradle Status [April 6]

2018-04-10 Thread Romain Manni-Bucau
Le 11 avr. 2018 02:30, "Reuven Lax" a écrit : Actually I always found the right-click to run tests to only sometimes work in Maven, especially if there were changes to dependent AutoValue classes where code had to be generated. Too often it would fail, and I would then need to use Maven to rebuil

Re: Gradle Status [April 6]

2018-04-10 Thread Reuven Lax
Actually I always found the right-click to run tests to only sometimes work in Maven, especially if there were changes to dependent AutoValue classes where code had to be generated. Too often it would fail, and I would then need to use Maven to rebuild the whole project. It would be cool if Gradle

Re: Gradle Status [April 6]

2018-04-10 Thread Kenneth Knowles
Reuven's point is good. Once we hit the bare minimum of having things working, let's collect usability improvements and engineering improvements on a separate JIRA from the main migration. I filed https://issues.apache.org/jira/browse/BEAM-4045 for these less critical issues to separate them from

Re: Gradle Status [April 6]

2018-04-10 Thread Romain Manni-Bucau
@jb: what did you change? I re-imported the project like 3 times earlier today and never got it working acceptably :( Personally if importing the project and right click on a test+debug works as good as maven in idea id be happy. I can manage other stuff in a console even if gradle reporting is no

Re: Gradle Status [April 6]

2018-04-10 Thread Reuven Lax
There are a lot of ideas on how to increase usability, but I think they'll get lost in the thread. I suggest we try to capture them in Jiras. I suggest we also find out what common use patterns are (people on this thread are probably sufficient), as different people will have different workflows.

Re: Gradle Status [April 6]

2018-04-10 Thread Jean-Baptiste Onofré
FYI, I did a new attempt and it works fine (pretty long). Previous try failed. Regards JB On 10/04/2018 19:52, Kenneth Knowles wrote: I've been on Idea+Gradle for ~two months, around the time I added https://github.com/apache/beam/pull/4583 and https://github.com/apache/beam/pull/4626 to make

Re: Gradle Status [April 6]

2018-04-10 Thread Romain Manni-Bucau
Le 10 avr. 2018 19:53, "Kenneth Knowles" a écrit : I've been on Idea+Gradle for ~two months, around the time I added https://github.com/apache/beam/pull/4583 and https://github.com/apache/ beam/pull/4626 to make the import require zero user work. I have no fear of deleting my project any time and

Re: Gradle Status [April 6]

2018-04-10 Thread Kenneth Knowles
I've been on Idea+Gradle for ~two months, around the time I added https://github.com/apache/beam/pull/4583 and https://github.com/apache/beam/pull/4626 to make the import require zero user work. I have no fear of deleting my project any time and re-importing. I agree with not having auto-import on

Re: Gradle Status [April 6]

2018-04-10 Thread Romain Manni-Bucau
Runner a test doesnt have the right classpath (idea uses out/ instead of build/) then when you switch on gradle runner the launching uses gradle which is not able to use submodules directly but reconsider the whole project which is quite slow for normal dev iterations compare to just run the test w

Re: Gradle Status [April 6]

2018-04-10 Thread Reuven Lax
Romain, Can you detail what's not working. I switched my IntelliJ over to Gradle about two weeks ago, and haven't had any trouble. Reuven On Tue, Apr 10, 2018 at 4:20 PM Romain Manni-Bucau wrote: > Ok, didn't find a way to make it working properly (only workaround > with direct commands and no

Re: Gradle Status [April 6]

2018-04-10 Thread Romain Manni-Bucau
Ok, didn't find a way to make it working properly (only workaround with direct commands and no good idea integration for debugging). I'm back with maven, if anyone knows how to properly solve it let's do it. If not I think JB point is to consider more than any other criteria. Romain Manni-Bucau @r

Re: Gradle Status [April 6]

2018-04-10 Thread Romain Manni-Bucau
side note: do NOT use auto-import until you are sure you can, it locks regularly on beam (pby too big for idea?) and makes idea ready to be killed :( Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book 2018-04-10 16:40 GMT+02:00 Jean-Baptiste Onofré : > It's what I did,

Re: Gradle Status [April 6]

2018-04-10 Thread Jean-Baptiste Onofré
It's what I did, I'm trying a complete reload now (maybe this step failed). On 10/04/2018 16:38, Lukasz Cwik wrote: beam-site PR/414 updates the instructions for using Intellij and how to import a module: 1. Create an empty IntelliJ project outside of the Beam source tree. 2. Under Project Str

Re: Gradle Status [April 6]

2018-04-10 Thread Lukasz Cwik
beam-site PR/414 updates the instructions for using Intellij and how to import a module: 1. Create an empty IntelliJ project outside of the Beam source tree. 2. Under Project Structure > Project, select a Project SDK. 3. Under Project Structure > Modules, click the + sign to add a module and se

Re: Gradle Status [April 6]

2018-04-10 Thread Jean-Baptiste Onofré
That's a very important issue for contribution. Up to now, I used Maven for setup IntelliJ (and it works just fine). If we remove the pom.xml, we have to support Eclipse and IntelliJ "smoothly". Let me try in IntelliJ. Regards JB On 10/04/2018 15:21, Romain Manni-Bucau wrote: You dont have i

Re: Gradle Status [April 6]

2018-04-10 Thread Lukasz Cwik
Romain, I haven't seen that error. At the very top of your test execution log it gives you the tasks that it is running, for example: 6:41:33 AM: Executing tasks ':beam-sdks-java-core:cleanTest :beam-sdks-java-core:test --tests "org.apache.beam.sdk.coders.AvroCoderTest.testAvroCoderEncoding"'... W

Re: Gradle Status [April 6]

2018-04-10 Thread Romain Manni-Bucau
You dont have issue due to the build setup with that option. I get: avr. 10, 2018 3:20:10 PM org.apache.beam.runners.direct.DirectTransformExecutor run GRAVE: Error occurred within org.apache.beam.runners.direct.DirectTransformExecutor@66761b7a com.google.common.util.concurrent.ExecutionError: jav

Re: Gradle Status [April 6]

2018-04-10 Thread Lukasz Cwik
I have found that the simplest setup is to delegate the build/test actions to Gradle. This allows you to run unit tests very easily and since its in the same manner that Gradle would have, you know that if its passing it will pass on the command line and on Jenkins. Here is one site that discusses

Re: Gradle Status [April 6]

2018-04-10 Thread Romain Manni-Bucau
What's the plan to make idea supporting gradle on beam project? Do we import the workaround mentionned in https://youtrack.jetbrains.com/issue/IDEA-175172? For the ones who didn't see this issue in action: idea will compile in out/ instead of build/ and you will just miss all the resources you need

Re: Gradle Status [April 6]

2018-04-10 Thread Etienne Chauchot
As a gradle beginner, I could not agree more !  +1 Etienne Le lundi 09 avril 2018 à 18:47 +0200, Jean-Baptiste Onofré a écrit : > Hi all, > > I did multiple gradle build since last week and I would like to share  > one of my concern: it's about the communities. > > If I think our users won't see

Re: Gradle Status [April 6]

2018-04-09 Thread Lukasz Cwik
Simplifying examples/java precommit is encompassed in BEAM-4033. Both you and Kenn pointed to the same thing. To my knowledge it is the only place where we have such a loop that generates tasks. On Mon, Apr 9, 2018 at 3:37 PM Romain Manni-Bucau wrote: > > > Le 9 avr. 2018 21:04, "Lukasz Cwik" a

Re: Gradle Status [April 6]

2018-04-09 Thread Romain Manni-Bucau
Le 9 avr. 2018 21:04, "Lukasz Cwik" a écrit : Romain, can you clarify by the weird task naming? (Give some examples using our current project and Gradle and what you would have expected.) Sure https://github.com/apache/beam/blob/fb6ba3bfd43605a0f4944828b5f19e75840fa7aa/examples/java/build.grad

Re: Gradle Status [April 6]

2018-04-09 Thread Jean-Baptiste Onofré
Hi Luke, let me take a concrete example. I'm developing a Beam extension. Most of the time, I will ask myself: 1. How do I do the build.gradle and include my extension in Beam project 2. How do my extension is compile (the equivalent of maven-compiler-plugin) and how to deal with dependency an

Re: Gradle Status [April 6]

2018-04-09 Thread Lukasz Cwik
Romain, can you clarify by the weird task naming? (Give some examples using our current project and Gradle and what you would have expected.) On Mon, Apr 9, 2018 at 2:49 PM Romain Manni-Bucau wrote: > +1 gradle not being mainstream - and even more in beam scope than other > java scope - it must

Re: Gradle Status [April 6]

2018-04-09 Thread Romain Manni-Bucau
+1 gradle not being mainstream - and even more in beam scope than other java scope - it must stay simple. I know that personally it takes me x2 or 3 to say "ok ill fork" a project using gradle, in particular with custom scripts and not only parent/module scripts. If needed we can write custom plug

Re: Gradle Status [April 6]

2018-04-09 Thread Kenneth Knowles
My phrasing sounded a bit too much like adding a blocking condition. I absolutely don't want to do that. Our maven build was far from clear, and had lots of tech debt and spooky action at a distance, I just want to emphatically agree with JB's sentiment that we have an opportunity to improve it be

Re: Gradle Status [April 6]

2018-04-09 Thread Lukasz Cwik
Fixed up prior e-mail. On Mon, Apr 9, 2018 at 1:50 PM Lukasz Cwik wrote: > JB, learning the build system in a project is hopefully avoided by most > users if the contribution guide can clearly explain what users need to do. > But for everyone else who wants to change a dependency version or add

Re: Gradle Status [April 6]

2018-04-09 Thread Lukasz Cwik
JB, learning the build system in a project is hopefully avoided by most users if the contribution guide can clearly explain what users need to do. But for everyone else who wants to change a dependency version or add a dependency it should be as simple as copy/paste (which I believe it already is).

Re: Gradle Status [April 6]

2018-04-09 Thread Kenneth Knowles
Huge +1 to the idea of investing in simplification and polish of the Gradle files before considering the migration complete. 1. build.gradle files should be as close to straight-line configuration as possible: - You should be able to understand a module's build easily, locally, without knowing Gr

Re: Gradle Status [April 6]

2018-04-09 Thread Jean-Baptiste Onofré
Hi all, I did multiple gradle build since last week and I would like to share one of my concern: it's about the communities. If I think our users won't see any change for them due to Gradle build (I think that most of our users will still use Maven with artifacts provided by Gradle), I'm mor

Re: Gradle Status [April 6]

2018-04-09 Thread Dan Halperin
On Sat, Apr 7, 2018 at 12:43 Reuven Lax wrote: > So if I understand correctly, we've migrated all precommit, most > postcommits, and we have a working release process using Gradle. There are > a few bugs left, but at this pace it sounds like we're close to fully > migrated. > > I know that multip

Re: Gradle Status [April 6]

2018-04-07 Thread Reuven Lax
So if I understand correctly, we've migrated all precommit, most postcommits, and we have a working release process using Gradle. There are a few bugs left, but at this pace it sounds like we're close to fully migrated. I know that multiple people put it long hours last getting this done last week

Re: Gradle Status [April 6]

2018-04-06 Thread Scott Wegner
Here's an end-of-day update on migration work: * Snapshot unsigned dailies and signed release builds are working (!!). PR/5048 [1] merges changes from Luke's branch * python precommit failing... will investigate python precommit Monday * All Precommits are gradle only * All Postcommits except pe

Re: Gradle Status [April 6]

2018-04-06 Thread Romain Manni-Bucau
Why building a zip per runner which its stack and just pointing out on that zip and let beam lazy load the runner: --runner=LazyRunner --lazyRunnerDir=... --lazyRunnerOptions=... (or the fromSystemProperties() if it gets merged a day ;)) Le 6 avr. 2018 20:21, "Kenneth Knowles" a écrit : > I'm w

Re: Gradle Status [April 6]

2018-04-06 Thread Romain Manni-Bucau
Le 6 avr. 2018 20:09, "Lukasz Cwik" a écrit : Romain, are you talking about the profiles that exist as part of the archetype examples? Was more thinking to this kind of profiles https://github.com/apache/beam/blob/master/sdks/java/nexmark/pom.xml (which should hit all IO at some point to ensure

Re: Gradle Status [April 6]

2018-04-06 Thread Kenneth Knowles
I'm working on finding a solution for launching the Nexmark suite with each runner. This doesn't have to be done via Gradle, but we anyhow need built artifacts that don't require user classpath intervention. It looks to me like the examples are also missing this - they have separate configuration

Re: Gradle Status [April 6]

2018-04-06 Thread Lukasz Cwik
Romain, are you talking about the profiles that exist as part of the archetype examples? If so, then those still exist and haven't been changed. If not, can you provide a link to the profile in a pom file to be clearer? On Fri, Apr 6, 2018 at 12:40 PM Romain Manni-Bucau wrote: > Hi Scott, > > i

Re: Gradle Status [April 6]

2018-04-06 Thread Romain Manni-Bucau
Hi Scott, is it right that 2 doesn't handle the hierachy anymore and that it doesn't handle profiles for runners as it is currently with maven? Romain Manni-Bucau @rmannibucau | Blog | Old Blog

Gradle Status [April 6]

2018-04-06 Thread Scott Wegner
I wanted to start a thread to summarize the current state of Gradle migration. We've made lots of good progress so far this week. Here's the status from what I can tell-- please add or correct anything I missed: * Release artifacts can be built and published for Snapshot and officlal releases [1]

Re: Gradle status

2018-03-23 Thread Scott Wegner
Thanks for organizing, Reuven. I too would like to see us move back to a single build system to reduce complexity. Count me in for the fixit. On Thu, Mar 22, 2018 at 11:27 PM Reuven Lax wrote: > I'll send an email tomorrow with a few proposed dates and set up a > burndown list of tasks. > > Reuv

Re: Gradle status

2018-03-22 Thread Reuven Lax
I'll send an email tomorrow with a few proposed dates and set up a burndown list of tasks. Reuven On Thu, Mar 22, 2018 at 11:24 PM Romain Manni-Bucau wrote: > Can work Reuven, where is the "todo" list? Thought we were done (= the > replacement was not blocking the dev) multiple times due to oth

Re: Gradle status

2018-03-22 Thread Romain Manni-Bucau
Can work Reuven, where is the "todo" list? Thought we were done (= the replacement was not blocking the dev) multiple times due to other threads but reading this week mails it sounds like we are not yet here. Romain Manni-Bucau @rmannibucau | Blog

Re: Gradle status

2018-03-22 Thread Reuven Lax
Let's back up for a second. Earlier in the thread we agreed to organize a community "fixit" day to try and migrate remaining Maven items to Gradle. I had thought that Romain had volunteered to run this, but reading back in the thread it appears that I misunderstood this. I would suggest that we or

Re: Gradle status

2018-03-22 Thread Dan Halperin
On Thu, Mar 22, 2018 at 11:19 AM, Chamikara Jayalath wrote: > I don't think incremental progress is a bad thing as long as we are making > progress towards the goal. Do we need better metrics (a weekly email ?) > about the progress towards moving everything to Gradle ? I agree with > others who p

Re: Gradle status

2018-03-22 Thread Chamikara Jayalath
I don't think incremental progress is a bad thing as long as we are making progress towards the goal. Do we need better metrics (a weekly email ?) about the progress towards moving everything to Gradle ? I agree with others who pointed out that there are many unresolved JIRAs and simply deleting Ma

Re: Gradle status

2018-03-22 Thread Romain Manni-Bucau
Le 22 mars 2018 18:49, "Dan Halperin" a écrit : It seems that a few groups are talking past each other. * A sizable contingent is interested in a move to Gradle -- it shows promise, but the work is incomplete. * Another contingent noticing the large burden of maintaining multiple build systems.

Re: Gradle status

2018-03-22 Thread Dan Halperin
It seems that a few groups are talking past each other. * A sizable contingent is interested in a move to Gradle -- it shows promise, but the work is incomplete. * Another contingent noticing the large burden of maintaining multiple build systems. FWICT, both test suites have been broken quite a l

Re: Gradle status

2018-03-22 Thread Lukasz Cwik
Romain, that is incorrect. Contributors are to maintain both systems until Gradle replaces that functionality in Maven and then it only needs to be maintained in Gradle. On Thu, Mar 22, 2018 at 10:30 AM Romain Manni-Bucau wrote: > Ok so to be clear for any contributor (which is the goal of this

Re: Gradle status

2018-03-22 Thread Romain Manni-Bucau
Ok so to be clear for any contributor (which is the goal of this thread): maven is still the main build system and no need to maintain gradle in PR then until beam switches. Im more than fine with that. Le 22 mars 2018 18:22, "Alan Myrvold" a écrit : > I think the investment in gradle is worthw

Re: Gradle status

2018-03-22 Thread Alan Myrvold
I think the investment in gradle is worthwhile, and incrementally we will continue to make progress. From what I've seem, gradle is a good fit for this project and a path to a faster, more reliable build system. pull/4812 creates the release artifacts, al

Re: Gradle status

2018-03-22 Thread Romain Manni-Bucau
2018-03-22 17:45 GMT+01:00 Lukasz Cwik : > what do we do? "Gradle migration will happen incrementally." > > "last months prooved beam cant maintain 2 systems, easier with that state > is then to drop gradle since it is a 0 investment compared to the opposite" > Its unfortunate that you feel this w

  1   2   >