On 31 March 2012 02:15, Luke Daley <[email protected]> wrote: > On 30/03/2012, at 9:24 PM, Adam Murdoch <[email protected]> > wrote: > > We want to keep the transports and the caching as separate as possible, so > we can reuse the caching across transports. This may not necessarily mean > that every caching strategy will work with every transport, but it would be > nice to have at least one strategy that can work across any transport. And > it looks like the 'best' option we've come up with for http also happens to > be a generic option, too (except for the etag check, but I'm sure we can > deal with that), so perhaps we only need one strategy. At some point we > might end up with some other transport-specific strategies, but ideally we > can base these on optional abstract capabilities of the transport (e.g. > 'can you provide content-length+last-modified-time efficiently?', 'can you > do a get-content-if-sha1-does-not-match?' and so on) rather than on > concrete transports. > > > This is more or less what we have now. > > > https://github.com/gradle/gradle/blob/master/subprojects/core-impl/src/main/groovy/org/gradle/api/internal/externalresource/transfer/DefaultCacheAwareExternalResourceAccessor.java > > The ExternalResourceAccessor contract is kinda flexible so I think we > could make it work for most transports and still use this general caching > algorithm. > > At the moment this is built in ExternslResourceRepository, but we could > allow injection of a custom one easily enough. > > There are two things I still four things I want to do before wrapping this > up: > > * treat 403 and 405 HEAD requests as "metadata unknown" > * if the server is googlecode, treat 404 HEAD requests as "metadata > unknown" > * when reusing a locally found resource, store the real metadata in the > index. > * where it's safe to, extract the sha1 from the etag (e.g. Artifactory). > > All of this things are small. >
Cool. One more thing we should do soon (but post 1.0) is allow caching of 'file' repositories. The recent change to FileSystemResolver means that we've removed the previous workaround for people with slow filesystem access (eg network shares). I think the solution we discussed was to always cache local repository artifacts (more consistent), but to always treat them as 'changing'. That is, on every resolve we would check if the cached artifact was up-to-date, by comparing modification-date+size, sha1, etc. There are probably some cases where we want to allow the users to only check every X unit of time, when the network is very slow or flaky. The beauty of Luke's solution is that this looks like it will be pretty straightforward: - Implement the meta-data methods on the File implementation of ExternalResourceAccessor. - Add something to the default CachePolicy to set the cache expiry for 'local' repositories to 0 seconds. cheers Daz
