On Mon, Apr 1, 2019 at 2:20 PM Lukasz Cwik <[email protected]> wrote: > > > On Mon, Apr 1, 2019 at 2:00 PM Kenneth Knowles <[email protected]> wrote: > >> >> As to building an aggregated "Java" project, I think the blocker will be >> supporting conflicting deps. For IOs like ElasticSearch and runners like >> Flink the conflict is essential and deliberate, to support multiple >> versions of other services. And that is not even talking about transitive >> dep conflicts. I think Python and Go don't have this issue simply because >> they haven't tackled those problems. >> >> Are you talking about just a shortcut for building (super easy to just >> add since we are using Gradle) or a new artifact that you want to >> distribute? >> >> On Mon, Apr 1, 2019 at 10:01 AM Lukasz Cwik <[email protected]> wrote: >> >>> During the gradle migration, we used to have something like: >>> >>> include(":sdks:java:core") >>> include(":sdks:java:extensions:sql") >>> include(":sdks:python") >>> >>> Just to be super clear, this is Gradle default and is equivalent to just >> leaving it blank. >> >> >>> but we discovered the Maven module names that were used during >>> publishing were "core" / "sql" / ... (effectively the directory name) >>> instead of "beam-sdks-java-core". >>> >> >> Isn't this managed by the publication plugin? >> https://docs.gradle.org/current/userguide/publishing_maven.html#sec:identity_values_in_the_generated_pom >> "overriding >> the default identity values is easy: simply specify the groupId, artifactId >> or version attributes when configuring the MavenPublication." >> > > During the gradle migration this wasn't that easy. The new maven publish > plugin improved a lot since then. > > >> Using the default at the time also broke the artifact names for intra >>> project dependencies that we generate[1]. Finally, we also ran into an >>> issue because we had more then one Gradle project with the same directory >>> name even though they were under a different parent folder (I think it was >>> "core") and that was leading to some strange build time behavior. >>> >> >> Weird. But I think the Jira should still stand as a move towards >> simplifying our build and making it more discoverable for new contributors. >> > > Agree on the JIRA makes sense, just calling out that there were other > issues that this naming had caused in the past which should be checked > before we call this done. >
Totally agree. It will be quite a large task with a lot of boilerplate that might not be separable from technical blockers that come up as you go through the boilerplate. Kenn Kenn >> >> >>> We didn't migrate to a flat project structure where each project is a >>> folder underneath the root project because of the existing Maven build >>> rules that were being maintained in parallel and I'm not sure if people >>> would want to have a flat project structure either. >>> >>> 1: >>> https://github.com/apache/beam/blob/a85ea07b719385ec185e4fc5e4cdcc67b3598599/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L1055 >>> >>> On Mon, Apr 1, 2019 at 9:49 AM Michael Luckey <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> although I did not yet manage to get deeper involved into actual >>>> development, I think this ability would be a useful addition. >>>> >>>> But I would also like to point out, that this is kind of implicit, as >>>> soon we get https://issues.apache.org/jira/browse/BEAM-4046 included. >>>> >>>> For instance, we would change the current setup from >>>> >>>> include "beam-sdks-java-core" >>>> project(":beam-sdks-java-core").dir = file("sdks/java/core") >>>> >>>> to something like >>>> >>>> include(":sdks:java:core") >>>> include(":sdks:java:extensions:sql") >>>> include(":sdks:python") >>>> >>>> >>>> With this in place a plain >>>> >>>> $ ./gradlew -p sdks/java build >>>> >>>> would exactly do what you want. And, of course, this will also work for >>>> 'sdks/java/io', 'runners/' etc. Hope, you get the point. >>>> >>>> Currently, we deviate from gradle default convention and therefore have >>>> to implement some quirks to restore default behaviour. And I somehow >>>> dislike the structure introduced by parent/child folders, which will be >>>> destroyed by our current project definitions. >>>> >>>> But, to be honest, although I have some clear understanding on how to >>>> proceed here - especially regarding the requirement to keep the change >>>> backwards compatible - we might decide not to switch. Because deeper >>>> investigation might reveal issues, which I am currently not aware of. >>>> >>>> Best, >>>> >>>> michel >>>> >>>> On Mon, Apr 1, 2019 at 5:52 PM Jean-Baptiste Onofré <[email protected]> >>>> wrote: >>>> >>>>> Hi guys, >>>>> >>>>> I would like to introduce a Gradle "meta" project for the build: >>>>> beam-sdks-java. >>>>> >>>>> The idea is to simply build all Java SDK related resources (core, IO, >>>>> ...). >>>>> >>>>> The purpose is also to be aligned with the other SDKs which provide >>>>> beam-sdks-go and beam-sdks-python. >>>>> >>>>> Thoughts ? >>>>> >>>>> Regards >>>>> JB >>>>> -- >>>>> Jean-Baptiste Onofré >>>>> [email protected] >>>>> http://blog.nanthrax.net >>>>> Talend - http://www.talend.com >>>>> >>>>
