On 15/07/2013, at 3:26 AM, Luke Daley <luke.da...@gradleware.com> wrote:
> Hi, > > I started looking into this area this week out of interest. I was surprised > about how naive our family of copy tasks are about incremental build, and I'm > surprised we aren't being pushed harder to fix this stuff. My guess is that > most people get unexpected UP-TO-DATES, curse, run `clean`, then move one. > > I have some flights coming up and want to work on a spec for this stuff. That'd be great. > Upon thinking about it, I've realised it's not a trivial problem to solve > effectively and efficiently. I'm wondering if there are any existing plans or > directions for this. > > Right now, we only consider the source files for up-to-date determination > (taking include/exclude into account). This means we effectively evaluate the > input source files at least twice: once for up-to-date calculations, and then > again for the actual operation. To consider other settings (e.g. > includeEmptyDirs, expand(), duplicatesStrategy, rename), a naive > implementation would introduce a third evaluation. This is because in terms > of comparison, we need associate the configuration options with the files > that they act on. If we could persist and compare such an association then we > can detect changes. > > If we had a point in time where we knew that the copy task configuration was > no longer going to change, we could do the source file calculation once and > cache it. We could add this kind of thing reasonably easily. The task could > be informed that it's about to executed and have its inputs queried. At that > point we could build up the comparable data structure. > > At this point I'm just wondering if there were some existing ideas before I > put any time into spec'ing that out. You can think of the copy tasks as being made up of two pieces: 1. A bunch of settings that we evaluate to produce a stream of FileCopyDetail objects. 2. An action that takes the stream of FileCopyDetails and does the work of producing the outputs. We could do the change detection at either point, by either building a description of the settings and comparing against last time, or by building the stream of FileCopyDetails and comparing against last time. The second option is going to be more accurate and would allow the copy action to do work on just those inputs that have changed, but it's going to need some infrastructure for this. It's probably the same infrastructure that we'll need for incremental C++ compilation, where we need to track the set of headers for a given source file. -- Adam Murdoch Gradle Co-founder http://www.gradle.org VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting http://www.gradleware.com
