On Sep 9, 2009, at 1:25 AM, Adam Murdoch wrote:
Hans Dockter wrote:
On Sep 2, 2009, at 1:44 AM, Adam Murdoch wrote:
Hi,
I've just committed fixes for some performance regressions which
have snuck in since the 0.7 release:
- Ported AnnotationProcessingTaskFactory from Groovy to Java, and
added some caching.
- Use ASM instead of Groovy to generate task subclasses which mix
in convention mapping and dynamic properties.
- Cache all scripts. In particular, cache empty/missing init
scripts and the default buildSrc script. More on this below.
Using our performance test multi-project build (which has 25
projects), gradle -t executes in:
Gradle 0.8-20090829161512+1000: 8.41 seconds (this is the version
we're using for gradlew)
Gradle 0.7: 3.98 seconds
Gradle head: 3.58 seconds
So, head is a now a little faster than the 0.7 release.
Also, the developerBuild is down from ~30 min to ~20 min on my not-
particularly-good laptop.
One implication of caching every script is that we don't always
have a directory in which to put the .gradle cache. So, I've
changed the script compilation to cache all scripts under
~/.gradle/scriptCache. An advantage of this is we no longer end up
with .gradle directories scattered all through the source tree
(unless you end up using an internal repository). The downside is
we will need some way to garbage collect this cache. This isn't
actually a new problem, because we needed to solve this any way -
It's just more important now.
There are other areas where we also need a place to store metadata.
In those cases we don't have the situation of possibly not having a
directory.
- Changedetection (at the moment stored in <checkedDir>/.gradle)
Should we move this also to ~/.gradle ?
I think we should cache everything in the same place. But that's
really just my personal preference.
- task history (build/.gradle/task-execution-history)
I thought it makes sense to put the task history in the build dir.
If the output is deleted you might as well delete the history
(which purpose is to control which output needs to be reproduced).
We avoid any GC/stale problems that way.
Except it's not quite right.
I know. We talked about it a little in our last dev conference with
Steve and Tom.
It assumes all tasks which generate output do so into the build
directory. This means that after a clean, we'll incorrectly re-
execute tasks which generate their output somewhere else,
Yes
or which do not generate any output.
Task with no output don't have a history in the current implementation
as the history is part of the output handling. The history returns if
the output was created successfully and the last time the output was
modified. That makes no sense for let's say the clean task. This
behavior is also owned to the fact that we use the build directory for
storing the history (which collides with a clean history). But the
major reason is that right now our only history use case if for output
generation optimization.
The rule for skipping a task is: skip the task if 1) its input files
have changed since last time it successfully executed, and 2) its
output files exist.
That would be the most reliable approach and what we should do that in
the future. Stuff would work out of the box. But for 0.8 we will go
with custom onlyIf rules I think.
They would look like:
test.onlyIf { !test.output.wasSuccessfullyCreated ||
test.output.lastModified < testCompile.output.lastModified ... }
That means if someone manually deletes build output the build would
behave in an erroneous way.
It's predicate 2) which deals with rebuilding after a clean, not
predicate 1). This means there's no reason to store this information
in the build directory.
No question about that. But I guess this is 0.9 stuff, isn't it?
- Hans
--
Hans Dockter
Gradle Project Manager
http://www.gradle.org
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
