On 9 April 2013 05:25, Luke Daley <[email protected]> wrote:
>
> On 09/04/2013, at 11:55 AM, Luke Daley <[email protected]> wrote:
>
>>
>> On 27/03/2013, at 3:43 PM, Daz DeBoer <[email protected]> wrote:
>>
>>> G'day
>>>
>>> Now in master is a pretty cool new feature: you can now implement an
>>> 'incremental' task that is informed about exactly which input files
>>> have changed when the task is out of date.
>>> This is very useful for something like a C++ compile task, as it means
>>> that only the changed files need to be recompiled, rather than the
>>> entire set of inputs.
>>>
>>> I've got a 'draft' DSL functioning, and would appreciate any feedback
>>> you guys have. Here's a sample:
>>>
>>> class IncrementalSync extends DefaultTask {
>>> @InputFiles
>>> def FileCollection src
>>>
>>> @OutputDirectory
>>> def File destination
>>>
>>> @TaskAction
>>> void execute(TaskInputChanges inputs) {
>>> if (inputs.allOutOfDate) {
>>> FileUtils.forceDelete(destination)
>>> }
>>>
>>> inputs.outOfDate({
>>> FileUtils.copyFile(change.file, targetFile(change.file))
>>> } as Action)
>>> .removed({
>>> FileUtils.forceDelete(targetFile(change.file))
>>> } as Action)
>>> .process()
>>> }
>>>
>>> def targetFile(def inputFile) {
>>> new File(destination, change.file.name)
>>> }
>>> }
>>
>> Thinking more about this, I don't think what we have is right. I think we
>> can come up with a simpler API.
>>
>> interface TaskExecutionContext {
>>
>> boolean isHasPriorOutputs(); // this is an incremental execution
>>
>> void eachInputFileChange(Action<InputFileChange> action);
>>
>> interface InputFileChange {
>> boolean isAdded();
>> boolean isModified();
>> boolean isRemoved();
>> File getFile();
>> }
>>
>> }
>>
>> @TaskAction
>> void execute(TaskExecutionContext executionContext) {
>> if (executionContext.hasPriorOutputs) {
>> executionContext.eachInputFileChange {
>>
>> }
>> } else {
>> // full rebuild
>> }
>> }
>>
>> Some issues I have with the existing design:
>>
>> * I'm not sure "outOfDate" is the right term here. It doesn't quite capture
>> it and can imply more than what we actually mean. You could say that an
>> output is out of date, but it doesn't quite work for inputs.
>> * why do we need the outOfDate() and removed() methods? After reading the
>> Javadoc I'm not sure how I would use this API. What's the difference between
>> out-of-date and removed? I could guess, but it's not clear.
>> * I think we'll want to provide information that is not strictly an aspect
>> of the inputs or outputs, but of something emergent from the input/output
>> concept. That's why I think modelling an encompassing type is better.
>
> Sorry, accidentally hit send…
>
>> What I think we are trying to do is make it easy for the user to decide
>> whether they should do a fine grained incremental execution or a coarse
>> grained full.
>
> The existing isAllOutOfDate() method goes too far, based on its Javadoc:
>
> /**
> * Specifies if incremental build is not possible due to changed Input
> Properties, Output Files, etc.
> * In this case, every file will be considered to be 'out-of-date'.
> */
> boolean isAllOutOfDate();
>
> It is too presumptuous of us to say that all outputs are invalid just because
> an input property changed. The task author may know better and can only
> rebuild what is effected by the input property change.
>
> We can tell if a full build is needed with relative confidence based on the
> lack of task history, or by verifying that the declared outputs don't exist
> (if that's fast enough), hence the isHasPriorOutputs() method. It would be
> nice to be able to tell the task that all of the input files have changed in
> some way, but this seems like it would be very expensive to compute.
>
>> --
>> Luke Daley
>> Principal Engineer, Gradleware
>> http://gradleware.com
>>
>> Join me at the Gradle Summit 2013, June 13th and 14th in Santa Clara, CA:
>> http://www.gradlesummit.com
>>
For our usual "up-to-date" check, these are the things that can cause
a task to be out-of-date (in order of checking):
1. Task has no declared outputs (always out-of-date with no declared outputs)
2. Task.upToDateWhen() returns false
3. Task has no record of previous execution
4. Task class has changed since previous execution
5. Task input property has changed
6. Task output files have changed
7. Task input files have changed
So the interesting thing is how to deal with all of these cases with
the incremental API:
(We are currently not honouring 1. for incremental tasks: this rule is
mostly to make lifecycle tasks (and similar) fast, by not
persisting/checking state. By making your task 'incremental' you're
also telling Gradle not to make this check.)
The initial approach was that the InputFileChanges parameter on
TaskAction was an indication that the task could handle changes to
input files: anything other sort of state change could not be handled
by the task. The "out-of-date" files would be the set of files that
you needed to re-evaluate, based on _all_ changes, including those
that you cannot handle directly (like, say, input property changes).
We would later add other parameter types to declare that the task
could handle other incremental changes: properties, output files, etc.
Having spent more time on this, I'm not really liking this approach so
much: I think that a simple "tell me the changed inputs" method is
more useful and less confusing than a clever "tell me what I should
re-evaluate" method (this is what Luke is suggesting, too).
We then need a way to tell an incremental task that other
(non-input-file) changes have occurred since the prior execution,
leaving the task implementation to decide what to do in this case.
This API needs to be able to adapt to the future when a task can
handle input-file and input-property changes, but still needs to know
if task.upToDate() is false or outputs have changed since previous
execution. Right now, we need to be able to ask "has anything other
than 'input-files' changed since the last execution"?
Perhaps something like:
IncrementalExecutionContext {
boolean isIncremental(); // false for cases 2,3,4 above
// We can pretty easily implement these right now
boolean isHasInputPropertyChanges()
boolean isHasInputFileChanges()
boolean isHasOutputFileChanges()
void eachInputFileChange(Action<InputFileChange> action)
// And to add to the API in the future
void eachInputPropertyChange(Action<InputPropertyChange> action)
void eachOutputFileChange(Action<OutputFileChange> action)
}
When isIncremental() == false, then the other methods could throw
IllegalStateException, since we really don't know the answer.
--
Darrell (Daz) DeBoer
Principal Engineer, Gradleware
http://www.gradleware.com
Join us at the Gradle Summit 2013, June 13th and 14th in Santa Clara,
CA: http://www.gradlesummit.com(We are currently not honouring 1. for
incremental tasks: this rule is mostly to make lifecycle tasks (and
similar) fast, by not persisting/checking state.
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
