Fixed up prior e-mail. On Mon, Apr 9, 2018 at 1:50 PM Lukasz Cwik <lc...@google.com> wrote:
> JB, learning the build system in a project is hopefully avoided by most > users if the contribution guide can clearly explain what users need to do. > But for everyone else who wants to change a dependency version or add a > dependency it should be as simple as copy/paste (which I believe it already > is). Note that people who want to do anything more complicated like add new > jenkins job configurations for new integration test targets needs to learn > how the build system works, how to plumb the required set of attributes > from Jenkins into the test target via the build system. > > Kenn, I have to disagree with a lot in 1 and 2. For users who are > unfamiliar with Maven, we didn't setup the Maven build in such a way that > anyone unfamiliar with Maven knew what was going on or try to use concepts > unnatural to Maven in an attempt to make it seem easier. I believe we > should stick with Gradle/Groovy conventions. Users who are not familiar > with how Gradle/Groovy works will either ask the community or look at > StackOverflow and doing things like passing the project around into > functions is extremely uncommon compared to using the current context and > applying closures to it. For users who are familiar with Gradle, the builds > should be natural Gradle. > > *- If you are going to refer to an identifier, it should have an explicit > point of origin (not strings smashed together in a loop)* > I assume your referring to how the examples java precommit is done. It is > the only case of it to my knowledge and extra hands to migrate it to be > like how the validates runner tests run would be useful. Filed > https://issues.apache.org/jira/browse/BEAM-4033 > > > On Mon, Apr 9, 2018 at 1:24 PM Kenneth Knowles <k...@google.com> wrote: > >> Huge +1 to the idea of investing in simplification and polish of the >> Gradle files before considering the migration complete. >> >> 1. build.gradle files should be as close to straight-line configuration >> as possible: >> - You should be able to understand a module's build easily, locally, >> without knowing Groovy >> - As little Groovy programming as possible, focused on providing >> well-defined utilities >> >> 2. Explicit > implicit, especially for config files >> - Ditto about being able to understand a module's build locally >> - Minimize globally inherited config, in favor of explicitly calling >> utilities >> - Passing current project to a utility is better than trying to make >> something the "closure" >> - If you are going to refer to an identifier, it should have an explicit >> point of origin (not strings smashed together in a loop) >> >> When in doubt, a little redundancy to improve readability is worth it in >> this context. >> >> Kenn >> >> On Mon, Apr 9, 2018 at 9:47 AM Jean-Baptiste Onofré <j...@nanthrax.net> >> wrote: >> >>> Hi all, >>> >>> I did multiple gradle build since last week and I would like to share >>> one of my concern: it's about the communities. >>> >>> If I think our users won't see any change for them due to Gradle build >>> (I think that most of our users will still use Maven with artifacts >>> provided by Gradle), I'm more concerned by the dev community and the >>> contribution. >>> >>> Maven is well known and straight forward for a large part of potential >>> contributors. I think we have to keep in mind that we still have to grow >>> up our contributors community. >>> >>> Today, maybe I'm wrong, but I have the feeling that gradle build is not >>> straight forward (build.gradle includes build_rules.gradle, gathering >>> all taks all together). >>> >>> I would like to add a task in the gradle "migration" process: simplify >>> the gradle structure and files, and document this. >>> >>> I know we already have a Jira about the documentation part, but I would >>> like to "polish" and use a clean structure for the Gradle resources. As >>> already quickly discussed, I think that having one gradle file per tasks >>> in the .gradle directory would be helpful. >>> >>> The goal is really to simplify the contribution. >>> >>> Do you agree if I add a Jira about "Gradle polish" ? >>> Thoughts ? >>> >>> Regards >>> JB >>> >>> On 07/04/2018 04:52, Scott Wegner wrote: >>> > Here's an end-of-day update on migration work: >>> > >>> > * Snapshot unsigned dailies and signed release builds are working (!!). >>> > PR/5048 [1] merges changes from Luke's branch >>> > * python precommit failing... will investigate python precommit >>> Monday >>> > * All Precommits are gradle only >>> > * All Postcommits except performance tests and Java_JDK_Versions_Test >>> > use gradle (after PR/5047 [2] merged) >>> > * Nightly snapshot release using gradle is ready; needs PR/5048 to be >>> > merged before switching >>> > * ValidatesRunner_Spark failing consistently; investigating >>> > >>> > Thanks for another productive day of hacking. I'll pick up again on >>> Monday. >>> > >>> > [1] https://github.com/apache/beam/pull/5048 >>> > [2] https://github.com/apache/beam/pull/5047 >>> > >>> > >>> > On Fri, Apr 6, 2018 at 11:24 AM Romain Manni-Bucau >>> > <rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>> wrote: >>> > >>> > Why building a zip per runner which its stack and just pointing out >>> > on that zip and let beam lazy load the runner: >>> > >>> > --runner=LazyRunner --lazyRunnerDir=... --lazyRunnerOptions=... (or >>> > the fromSystemProperties() if it gets merged a day ;)) >>> > >>> > Le 6 avr. 2018 20:21, "Kenneth Knowles" <k...@google.com >>> > <mailto:k...@google.com>> a écrit : >>> > >>> > I'm working on finding a solution for launching the Nexmark >>> > suite with each runner. This doesn't have to be done via >>> Gradle, >>> > but we anyhow need built artifacts that don't require user >>> > classpath intervention. >>> > >>> > It looks to me like the examples are also missing this - they >>> > have separate configuration e.g. sparkRunnerPreCommit but that >>> > is overspecified compared to a free-form launching of a main() >>> > program with a runner profile. >>> > >>> > On Fri, Apr 6, 2018 at 11:09 AM Lukasz Cwik <lc...@google.com >>> > <mailto:lc...@google.com>> wrote: >>> > >>> > Romain, are you talking about the profiles that exist as >>> > part of the archetype examples? >>> > >>> > If so, then those still exist and haven't been changed. If >>> > not, can you provide a link to the profile in a pom file to >>> > be clearer? >>> > >>> > On Fri, Apr 6, 2018 at 12:40 PM Romain Manni-Bucau >>> > <rmannibu...@gmail.com <mailto:rmannibu...@gmail.com>> >>> wrote: >>> > >>> > Hi Scott, >>> > >>> > is it right that 2 doesn't handle the hierachy anymore >>> > and that it doesn't handle profiles for runners as it >>> is >>> > currently with maven? >>> > >>> > >>> > Romain Manni-Bucau >>> > @rmannibucau <https://twitter.com/rmannibucau> | Blog >>> > <https://rmannibucau.metawerx.net/> | Old Blog >>> > <http://rmannibucau.wordpress.com> | Github >>> > <https://github.com/rmannibucau> | LinkedIn >>> > <https://www.linkedin.com/in/rmannibucau> | Book >>> > < >>> https://www.packtpub.com/application-development/java-ee-8-high-performance >>> > >>> > >>> > 2018-04-06 18:32 GMT+02:00 Scott Wegner >>> > <sweg...@google.com <mailto:sweg...@google.com>>: >>> > >>> > I wanted to start a thread to summarize the current >>> > state of Gradle migration. We've made lots of good >>> > progress so far this week. Here's the status from >>> > what I can tell-- please add or correct anything I >>> > missed: >>> > >>> > * Release artifacts can be built and published for >>> > Snapshot and officlal releases [1] >>> > * Gradle-generated releases have been validated >>> with >>> > the the Apache Beam archetype generation >>> quickstart; >>> > still needs additional validation. >>> > * Generated release pom files have correct project >>> > metadata [2] >>> > * The python pre-commits are now working in Gradle >>> [3] >>> > * Ismaël has started a collaborative doc of Gradle >>> > tips [4] as we all learn the new system-- please >>> add >>> > your own. This will eventually feed into official >>> > documentation on the website. >>> > * Łukasz Gajowy is working on migrating performance >>> > testing framework [5] >>> > * Daniel is working on updating documentation to >>> > refer to Gradle instead of maven >>> > >>> > If I missed anything, please add it to this thread. >>> > >>> > The general roadmap we're working towards is: >>> > (a) Publish release artifacts with Gradle (SNAPSHOT >>> > and signed releases) >>> > (b) Postcommits migrated to Gradle >>> > (c) Migrate documentation from maven to Gradle >>> > (d) Migrate perfkit suites to use Gradle >>> > >>> > For those of you that are hacking: thanks for your >>> > help so far! Progress is being roughly tracked on >>> > the Kanban [6]; please make sure the issues >>> assigned >>> > to you are up-to-date. Many of the changes are >>> > staged on lukecwik's local branch [7]; we'll work >>> on >>> > merging them back soon. >>> > >>> > >>> > [1] >>> https://github.com/lukecwik/incubator-beam/pull/7 >>> > [2] >>> https://github.com/lukecwik/incubator-beam/pull/3 >>> > [3] https://github.com/apache/beam/pull/5032 >>> > [4] >>> > >>> https://docs.google.com/document/d/1wR56Jef3XIPwj4DFzQKznuGPM3JDfRDVkxzeDlbdVSQ/edit >>> > [5] https://github.com/apache/beam/pull/5003 >>> > [6] >>> > >>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=242 >>> > >>> > [7] >>> > >>> https://github.com/lukecwik/incubator-beam/tree/gradle >>> > -- >>> > >>> > >>> > Got feedback? http://go/swegner-feedback >>> > >>> > >>> > -- >>> > >>> > >>> > Got feedback? http://go/swegner-feedback >>> >>
