Hi Robert, about your point about we never fully build the project, even if I agree, it's what we "sold" with Gradle. Because, with Maven you can also build a single module without problem.
So, if I don't get the argument about build time for single module: in that case, build speed was not a valid argument for Maven to Gradle topic. Again, I would like to emphasize about the contribution adoption. If you really think that documenting & improving the build system using Gradle, that's fine. About the effort to go back on Maven, I really think that it's do-able. Probably this thread will end nowhere and we will stay with Gradle, that's fair. I just hope that we won't have any "brake" on contribution. Again, don't get my wrong, I just wanted to bring the discussion on the table, no pressure, no harsh, just a fairly discussion and concern that I wanted to share ;) Regards JB On 09/10/2018 12:22, Robert Bradshaw wrote: > On Tue, Oct 9, 2018 at 10:04 AM Jean-Baptiste Onofré <j...@nanthrax.net > <mailto:j...@nanthrax.net>> wrote: > > Hi guys, > > I know that's a hot topic, but I have to bring this discussion on > the table. > > > Thank you for bringing this up and revisiting it now that we have some > experience. > > > Some months ago, we discussed about migrating our build from Maven to > Gradle. One of the key expected improvement was the time to build. > We proposed to do a PoC to evaluate the impacts and improvements, but > this PoC was actually directly a migrate on master. > > Now, I would like to bring facts here: > > 1. Build time > On my machine, the build time is roughly 1h15. It's pretty long, and > regarding what the build is doing, I don't see huge improvement provided > by Gradle. > > > I rarely, if ever, build from scratch so perhaps I have not been > impacted by this nearly as much. (In particular, build and test times > seem to have gone way down for me, probably due to better incremental > support, but that's just anecdotal.) > > Is this worse than it was on maven, or just not as much better as was > hoped? > > > 2. Build reliability > Even worse, most of the time, we need to use --no-parallel and > --no-daemon to have a reliable build (it's basically recommended for > release). It has an impact on build time, and we loose part of Gradle > benefits. > > > I think this is a matter of incorrect dependency declarations (and is > not unique to gradle). I'd have loved to been able to go with a build > system that simply didn't let you have incorrect dependency > declarations, but that wasn't an option for other reasons. > > I wonder if there's some automatic tooling we could leverage to fix (and > keep fixed) this. Regardless, this is unfinished work that remains to be > done so we can realize the full benefits. > > > 3. Release and repositories > Even if couple of releases has been performed with Gradle, it's not > obvious to see improvements around artifacts handling. I got my > repository polluted twice (that's part of the trick Gradle is doing to > speed up the build dealing around the repository). > > > Could you clarify what improvements we were expecting here? I thought > the goal was that we could publish the same artifacts, with no regression. > > > 4. IDE integration > We already had some comments on the mailing lists about the IDE > integration. Clearly, the situation is not good on that front too. The > integration on IDE (especially IntelliJ) is not good enough right now. > > > This is important. To be honest, I had also issues back in the day > getting the maven setup working well out of the box in IntelliJ and > Eclipse (mostly with respect to things like shadowing and protobufs), so > we shouldn't fall prey to the golden age fallacy. > > It seems the recent move to vendoring has caused more issues here; I'm > not sure that would be fixed just moving back to maven (or how to > resolve it going forward). > > On the other hand, just last week I set up a new computer according > to https://cwiki.apache.org/confluence/display/BEAM/IntelliJ+Tips and > that seems to be working fine. > > > We are working hard to grow up the community, and from a contributor > perspective, our build system is not good today IMHO. > As a contributor, I resumed my work on some PRs, and I'm spending so > much time of the build, largely more than working on the PRs code > itself. > > So, obviously, the situation is not perfect, at least from a contributor > perspective. > > The purpose of this thread is not again to have a bunch of replied > ending nowhere. I would like to be more "pushy" and let's try to be > concrete. So basically, we only have two options: > > 1. Improve the build, working hard on Gradle front. Not sure if it makes > such sense from a contributor perspective, as Maven is really well known > from most of contributors (and easier to start with IMHO). > 2. Back on Maven. That's clearly my preferred approach. IDE integration > is better, Maven is well known from the contributors as already said. > The effort is not so huge. We tried to use Gradle, we don't have the > expected results now, that's not a problem, it's part of a project > lifetime. > > Thoughts ? > > > I'd like to add some perspective as a primarily non-Java contributor > (having contributed "only" a couple thousand lines to the java side over > the last couple of years) that maven feels much more java-centric (as is > our repo, but that's a separate issue). Also, as someone not coming from > with years of maven (or gradle for that matter) experience before > working on Beam, I have found gradle much more intuitive and easier to > learn. Especially when it comes to developing changes that span multiple > modules (which for some reason I guess I tend to do a lot of, but with > so many of our core sdk tests being validates runner that's likely to > hit people just starting out as well). Now of course I don't want to > discount the Java community, indeed it's arguably the largest and most > important one at this point, but I also think that Beam's ability to not > be limited to that one language (and ecosystem) is one of its huge > selling points and differentiation (see the whole portability effort, > which is for both SDKs and Runners). > > Even from a Java perspective, neither is so obscure as to be a > significant barrier, and both are customizable enough that the average > new developer is probably going to be looking at the documentation to > see what commands to run to build/test/validate this project. So this is > probably something documentation can address. > > But if we're going to reap the benefits and minimize the downsides of > this migration, we have to finish the job. > > - Robert > > -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com
