High-level: ensure you have gradle cache enabled so only the first build is slow. If you encounter nondeterministic or noncached targets upstream of the module you are editing, that's worth discussing and probably fixing.
That's going to be a separate issue from wanting to build a single part of the GCP IO package without building the rest of the package. Details and questions below. On Mon, Nov 30, 2020 at 4:36 PM Daniel Collins <[email protected]> wrote: > Hello all, > > Any time I have the misfortune of creating a new beam branch, building a > subtarget (sdks/io/google-cloud-platform/.../pubsublite in my case) takes > O(30 mins) on my laptop. > Can you share the exact gradle command? > A lot of the steps seem to block on each other and even the leaf rebuild > can take minutes since all the GCP I/O transforms are in one target. A > couple of questions for the (hopefully?) gradle experts here: > > 1) Do you think that sharding these targets would increase parallelism in > the underlying build? > I'd start with --scan so you can see some details and share it with others easily. I'm not sure if --profile gives even finer-grained telemetry. To demonstrate, here are two scans of `./gradlew :sdks:java:io:google-cloud-platform:compileTestJava`: - from clean (8m): https://scans.gradle.com/s/j5jtqywn3uw4o/timeline - after modifying a file in the module (1m): https://gradle.com/s/g74hsjddl6x5g/timeline These are certainly slow, and there are decidedly nonideal bits in the dep graph (most of the execution-oriented bits should not be needed to just *compile* the tests). But most things aren't rebuilt anyhow. 2) Do you think doing so would have any knock-on negative effects, either > for compilation time or development speed? > The answer is always "avoid rebuilding" so smaller seems better. I'm not totally clear how much is to be gained in this case. The other answer is -PskipCheckerFramework which will net you a 4x speedup in Java compile time, at the cost of you probably having to rewrite your code once you un-disable it and discover you've got a bunch of lurking NPEs. Kenn > 3) Do you think this would be an hours, days or weeks time investment to > do? > > > The above implicitly comes with "willing to help out O(hours/days), but no > gradle knowledge so I would need some guidance". > > -Dan >
