On 27 Feb 2014, at 12:17 am, Joachim Nilsson <jocce.nils...@gmail.com> wrote:
> Thanks Adam.
>
> A follow up question:
> CONTRIBUTION.md states 'Avoid using features introduced in Java 1.6 or later'.
> Is this a hard requirement or could I use java.nio.Path from java7?
> That would really simplify the use of relative paths, which I will need if I
> use cache dir as base.
>
> I have one concern, that is if output or input resides in directories outside
> of the root project.
> In that case, forcing the use of relative paths may destroy portability as
> well, if the destination directory is not on 'same path depth' as the origin.
> With my first simple approach to use rootDir as detector of when a path could
> be rewritten, paths and files outside the project would remain absolute when
> serialised.
You can (and should) still use the same approach. You can always change
TaskArtifactStateCacheAccess to provide the root directory instead of the cache
directory.
> This may be a minor concern and can probably be handled by documentation.
> Just something to think about.
The plan is at some point to switch from using paths, and instead use input and
output content as the key incremental state. Then, when a task executes, we can
check if any task of the same type has run with the same input content anywhere
on the machine, and what output content it produced. We can then compare the
content of the actual outputs to see if the task is out of date.
Then, the incremental build will be able to deal with many more kinds of
changes that it currently does not - things like moving the build directory,
moving the source directories, renaming tasks, and so on.
Of course, it’s not that simple, but this should give you some idea of the plan.
>
> Regards,
> Joachim
>
>
>
> On Wed, Feb 26, 2014 at 4:46 AM, Adam Murdoch <adam.murd...@gradleware.com>
> wrote:
>
> On 21 Feb 2014, at 6:02 pm, Joachim Nilsson <jocce.nils...@gmail.com> wrote:
>
>> Hi, I ran into a tricky part of the code that I would like some hints with.
>> Background: I am investigating how to make it possible to make incremental
>> builds after a rename or move of the rootDir (normal for incremental builds
>> in CI servers running many executors on different slaves).
>>
>> The first part was fairly easy to solve, in a not so pretty way, but with
>> minimal changes.
>> Now I can read the task history even after a move of directories.
>>
>> My solution for the first part was to use the incoming task in
>> CacheBackedTaskHistoryRepository to get the project root dir and then strip
>> that part from the path in the binary file just before serializing. Changing
>> the internal representation to non absolute will have way too big impact for
>> me to grasp so I figured small change (a new boolan before the path) in the
>> binary format was better.
>>
>> public void write(Encoder encoder, LazyTaskExecution execution)
>> throws Exception {
>> ...
>> encoder.writeInt(execution.getOutputFiles().size());
>> for (String outputFile : execution.getOutputFiles()) {
>> if (canBeConvertedFromAbsolutePath(outputFile)){
>> encoder.writeBoolean(true);
>>
>> encoder.writeString(convertFromAbsolutePath(outputFile));
>> }else{
>> encoder.writeBoolean(false);
>> encoder.writeString(outputFile);
>> }
>> }
>> ...
>> }
>>
>> However, it was not done yet. Now I face that the
>> CacheBackedFileSnapshotRepository is also involved in setting the UP-TO-DATE
>> flag to false since it compares output files directly between runs. And this
>> time the whole FileCollectionSnapshot is stored so my minimal change attempt
>> will fail.
>>
>> Any suggestions for solving the CacheBackedFileSnapshotRepository?
>> Here, the whole object is serialized and in the serializer, there is no
>> access to Task, Project or Gradle. Replacing the object with relative file
>> references would probably fail the serialization since it uses File.exists
>> for each file.
>
> I would use the cache base directory as the base directory, in both
> CacheBackedFileSnapshotRepository and CacheBackedFileSnapshotRepository. It
> can be queried from TaskArtifactStateCacheAccess, but you’ll have to add a
> getter on it to expose the cache base dir.
>
>
> --
> Adam Murdoch
> Gradle Co-founder
> http://www.gradle.org
> VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
> http://www.gradleware.com
>
>
>
>
--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com
