Our incremental build performance has degraded a lot recently. On
Windows it's especially bad (as usual). There are multiple issues. Some
cases of running way too many find/grep during rule setup. Some files
get rebuilt on every incremental build. With all these changes, the
incremental build with no changes of "make product-bundles test-bundles"
on a Windows machine I was testing went from 4m to 24s. Here is a list
of the fixes:
We have a rather inefficient pattern where a makefile is responsible for
two top level targets, the general build of something and then the
copying of that something into images. The drawback of this pattern is
that all the rules in the makefile get evaluated twice, and this is
potentially expensive. My workaround for this is to enclose the
expensive parts in a conditional that checks what target the makefile
was called with. This pattern is already in use in at least Bundles.gmk.
In CompileJavaModules.gmk, if IMPORT_MODULES_CLASSES isn't set, wildcard
will try to look for /$(MODULE), which can sometimes be very costly and
freeze the system for 10-20s (likely due to going to check on the network).
Images.gmk, on Windows, the CDS archive was always being rebuilt due to
the rule targeting the wrong file.
NativeCompilation.gmk has two fixes. Reduce the number of $(shell find
...) calls by doing just one and then ordering the files found. The
second fix is more intricate. The adding of the IMPORT_LIBRARY to the
list of TARGETS was missing. This was causing trouble in combination
with a test lib, libHasNoEntryPoint.c, which does not produce a link.lib
file. When building incrementally, the makefile cannot find the
HasNoEntryPoint.lib file and so deletes HasNoEntryPoint.dll to force a
rebuild. By adding the IMPORT_LIBRARY to the targets, the
HasNoEntryPoint.lib will at least be touched. I also had to remove the
safety check that a .lib file was really created since it apparently
isn't always created.
In ProcessMarkdown.gmk, the $(shell grep ) expression was called for
every markdown file, even if no rebuilding was needed.
TestFilesCompilation.gmk called SetupNativeCompilation for each test
.c/cpp file, which would in turn call find on the src dir where that
file was located. I changed the call to use the EXTRA_FILES parameter
instead of SRC to avoid the extra calls to find.
In CompileJvm.gmk I added a call to FillCacheFind so we can reuse the
set of hotspot sources since they are used more than once.
I have run a comparison build of our main platforms with the patch.
There are certainly more improvements possible, but I chose to limit
myself to the most obviously broken parts here.
Webrev: http://cr.openjdk.java.net/~erikj/8220383/webrev.01/index.html
Bug: https://bugs.openjdk.java.net/browse/JDK-8220383
/Erik
