The build times for incremental builds look awesome. Now I'm really
looking forward to the new build system! Thanks!
Regarding partial builds: as long as the other components don't add
dependencies that we didn't use to have before (when working on the jdk
repo only), I guess it's all right, too.
--
best regards,
Anthony
On 09/13/12 00:32, Kelly O'Hair wrote:
Windows X64 CYGWIN Openjdk builds run between 16 and 25mins, depending on the
machine used (hardware vs. VM),
how many CPUs etc. But it's a bit erratic, depends on lots of things. We are
still working around some CYGWIN issues.
The incremental builds should be similar. I don't have any numbers right now.
We hope we can get MinGW/MSYS working which might reduce the overall time by
20% or more (a big guess)
(Remember MKS will not work with the new build-infra Makefiles).
Windows XP 32bit builds take a bit longer maybe 35mins?, but the XP systems
have limited RAM and cpu capacity.
Solaris, Linux, and Mac builds are pretty consistent, but Windows is another
story. :^(
What we have noticed is that a beefy PC running Windows XP can easily do 20min
builds, but building in
an area that is immune from On-Access AntiVirus scanning or turning On-Access
scanning off.
-kto
On Sep 12, 2012, at 11:52 AM, Igor Nekrestyanov wrote:
What about windows?
-igor
On 9/12/12 11:49 AM, Kelly O'Hair wrote:
Some stats on incremental builds. Not partial builds..
This is an older Solaris machine svc6.us.oracle.com, building the complete
openjdk forest from scratch
for 64bit including images took less than 14 minutes (parallel build setting
was 8) and images took 2mins of this 14mins:
-- Build times ----------
Target
Start 2012-09-12 10:56:17
End 2012-09-12 11:09:55
00:00:30 corba
00:04:51 hotspot
00:00:23 jaxp
00:00:33 jaxws
00:04:43 jdk
00:01:59 jdk-images
00:00:39 langtools
00:13:38 TOTAL
-------------------------
A repeated 'gmake images" with no changes took 31 seconds:
-- Build times ----------
Target
Start 2012-09-12 11:13:30
End 2012-09-12 11:14:01
00:00:02 corba
00:00:07 hotspot
00:00:02 jaxp
00:00:04 jaxws
00:00:08 jdk
00:00:07 jdk-images
00:00:01 langtools
00:00:31 TOTAL
-------------------------
I touched a few awt java sources (touch jdk/src/share/classes/java/awt/*.java)
and did just a 'gmake all' (no images)
and it took 2 minutes (this will improve significantly with smartjavac):
-- Build times ----------
Target
Start 2012-09-12 11:17:34
End 2012-09-12 11:19:32
00:00:00 corba
00:00:00 hotspot
00:00:00 jaxp
00:00:01 jaxws
00:01:56 jdk
00:00:01 langtools
00:01:58 TOTAL
-------------------------
I touched a few awt native sources (touch jdk/src/share/native/sun/awt/*/*.c)
and did the 'gmake all' (no images)
and it took 37 seconds):
-- Build times ----------
Target
Start 2012-09-12 11:22:24
End 2012-09-12 11:23:01
00:00:00 corba
00:00:00 hotspot
00:00:00 jaxp
00:00:01 jaxws
00:00:35 jdk
00:00:01 langtools
00:00:37 TOTAL
-------------------------
So as far as incremental builds go, I think this is a vast improvement. And the
incremental build should be accurate.
I'm not taking a position on partial builds here, just providing some hard data
on full openjdk builds with the
new build-infra makefiles.
The above builds are complete openjdk builds, with no import jdk needed, and no
concerns about
sync issues between jdk components (hotspot<->jdk).
-kto
On Sep 11, 2012, at 6:37 AM, Anthony Petrov wrote:
Magnus,
You've only explained how incremental builds could work for Java classes in the
new build-infra. What about incremental builds of native code? E.g. in AWT we
often do the following:
$ cd make/sun/awt (or make/java/awt, or make/sun/lwawt)
$ make
And this re-builds both AWT classes and AWT native code (and some 2D stuff too,
btw). What has build-infra got to offer for incremental builds of native code?
Also, how fast are those incremental solutions (both the temporary, and the
expected final one) in build-infra? Performing the above commands with the
current build system takes literally just a few seconds on any platform (given
you've built the JDK repo once before, and it might took like 10-30 minutes for
the first build).
--
best regards,
Anthony
On 9/10/2012 6:00 PM, Magnus Ihse Bursie wrote:
On 2012-09-10 14:13, Alan Bateman wrote:
When you say "sub-directory builds" then I think you mean incremental builds, or "poor-man
increment builds" as I call it. I think the majority of people working in the jdk repository, at least
in Oracle, do this because they know the area and know which make files that re-build the files that they
have changed. Incremental builds are generally not very reliable but seem to be "good enough" for
most people needs, at least those that aren't doing significant refactoring. The main motive of course is to
be able to re-build in a few seconds and the #1 goal of any new build system has to be fast incremental
builds.
In the new build system, fast incremental builds of Java code is dependent on the new
"smart javac", which unfortunately has still not proven stable enough to be
enabled by default, even in the experimental build-infra forest. It is still our hope
that it will turn out to be good enough to be used as default, but I don't want to trust
hope alone.
As an interim solution, I have just created the following two shortcuts. Both of these
bypass the proper dependency detection. This means that make will cut to the chase and
recompile faster, but it also means that you need to take over the responsibility of
making sure dependencies are correct. This is very similar to the "*-only"
(e.g. hotspot-only) targets that are current available in the new build system.
First shortcut is to skip parts of the JDK makefile. By typing e.g. "make jdk JDK_TARGET=classes", the JDK
build will stop after building the "classes" target (which compiles the majority of the JDK java files). An
even faster, but less safe, version is "make jdk-only JDK_TARGET=classes-only", which will jump directly to
the JDK Makefile, and only execute the "classes" target. In this way, absolutely no time is spent before
starting to compile Java classes.
Second shortcut is to limit the classes that are recompiled by the JDK "classes" target.
Currently, *all* (more or less) Java classes in the JDK is recompiled at once. While this speeds up
things considerably at a first time compilation, it makes all subsequent builds take the same time.
By specifying e.g. JDK_FILTER=java/nio, only files in the java/nio package will get compiled. This
is, I think, similar (but not strictly not identical) to the effect of "sub-directory
builds". It is possible to use an even more fine-grained filter, separated by comma, e.g.
JDK_FILTER=java/nio/channels,java/nio/charset -- the smaller number of Java files your filter
includes, the faster the build will go (more or less).
These two can of course be combined.
I would like to point out that this is hopefully a temporary solution -- if
proper dependency checking and incremental builds get fast enough, this kind of
trickery should not be needed. If this happens, we will remove the support for
these shortcuts since they do carry a risk of improper builds.
/Magnus
