Opened https://github.com/apache/beam/pull/6864 with your change but would love to get you GitHub contributor credit in the metrics FWIW.
Kenn On Sun, Oct 28, 2018 at 12:52 PM Kenneth Knowles <[email protected]> wrote: > Nice! > > Do you have a PR open? Let's get this change in immediately. > > This is probably the root cause of the flood of failures in > https://issues.apache.org/jira/browse/BEAM-5035 / > https://issues.apache.org/jira/browse/BEAM-5207 / > https://issues.apache.org/jira/browse/BEAM-5116 that led to us disabling > parallel builds. > > I've been slowly working towards two other independently useful changes > that I hoped would help ((1) don't depend on test jars and (2) vendor > dependencies so we just don't shade at all). Your solution will get us a > fast build sooner. > > Kenn > > On Sun, Oct 28, 2018 at 9:57 AM Michael Luckey <[email protected]> > wrote: > >> Hi, >> >> currently we bundle our tests within packageTests task [1] and >> additionally create a shadowed version with shadowTestJar. >> >> Unfortunately both task build the jar with the same classifier, which >> results in each task overwriting the others output, preventing any >> up-to-date checks from working. >> >> Changing the build slightly to produce different artifacts >> >> --- >> a/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy >> >> +++ >> b/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy >> >> @@ -588,7 +588,7 @@ class BeamModulePlugin implements Plugin<Project> { >> >> >> >> // Ensure that tests are packaged and part of the artifact set. >> >> project.task('packageTests', type: Jar) { >> >> - classifier = 'tests' >> >> + classifier = 'tests-unshaded' >> >> from project.sourceSets.test.output >> >> } >> >> project.artifacts.archives project.packageTests >> >> and so enabling up-to-date checking cuts build time [3] to half (on a >> build ignoring python/go/website where up-to-date checks currently do not >> work anyway) [4]. >> >> As I currently do not fully understand all consequences, I am wondering, >> whether we could apply that simple fix or wether there is some hidden >> magic, which relies on both artifacts be named the same. >> >> Anyone able to provide further insights here? >> >> Thx, >> >> michel >> >> >> [1] >> https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L609-L614 >> [2] >> https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L766-L780 >> [3] >> Profiled build: -x :beam-sdks-python:build -x :beam-sdks-go:build -x >> :beam-website:build build >> DescriptionDuration >> Total Build Time 1m28.94s >> Startup 1.657s >> Settings and BuildSrc 1.870s >> Loading Projects 0.035s >> Configuring Projects 4.565s >> Task Execution 1m17.11s >> [4] >> Profiled build: -x :beam-sdks-python:build -x :beam-sdks-go:build -x >> :beam-website:build build >> >> DescriptionDuration >> Total Build Time 46.781s >> Startup 1.633s >> Settings and BuildSrc 2.004s >> Loading Projects 0.061s >> Configuring Projects 4.883s >> Task Execution 34.121s >> >> >> >>
