Artifactory produces "X-checksum-SHA1" and "X-checksum-MD5" headers: I wasn't sure if they were supported on a wider basis, but it appears not. I like your idea of being able to declare the ETag to be a universal checksum - is that something easy to switch on in Apache for example? Daz
On 12 December 2011 05:02, Luke Daley <[email protected]> wrote: > > On 11/12/2011, at 9:37 PM, Daz DeBoer wrote: > > G'day > > We've been working on enhancing our dependency resolution so that we don't > download artifact files unnecessarily. This is done by: > 1) Look for any candidate artifacts that we suspect may be the one we want > but we're not certain > 2) Calculate the SHA1 hashes of these candidate artifacts > 3) Request a "*.sha1" file from the server, and comparing the declared > SHA1 value against each of the candidate artifacts > 4) If one of the candidate matches the SHA1 value on the server, we use > the candidate. If not, we download. > > Searching for candidates involves looking for previously downloaded > artifacts that have the same ArtifactId as the one we're looking for: this > is effectively [org]:[module]:[revision]:[type]:[artifact-name]:[extension]. > We currently search in: > - The artifact cache of the current Gradle version, looking for artifacts > that have been downloaded from a different repository > - The artifact cache of the previous Gradle version, allowing us to > upgrade the cache format without requiring re-download > In the future, the plan is to look in ~/.m2 as well. This will make the > out-of-the-box experience better for people switching from maven. > > So in all, this should reduce the number of downloads required when: > - You upgrade Gradle versions > - You use virtual repositories in Artifactory, with different URLs mapping > to the same artifacts > - You switch to use a repository manager (like Artifactory) from accessing > repositories directly > > The nice thing about this model is that we only request the sha1 file from > the server if we have a likely candidate to match against. > This currently only works when the server has a .sha1 file for the > requested artifact. The plan is to extend this to include md5 hash files, > as well as custom HTTP headers like those provided by Artifactory. > > > Awesome stuff Daz. > > What are the special Artifactory headers? As we discussed before, I think > some kind of switch to indicate that the repository uses sha1 for ETags > would be better. This would work in more situations. For example, if I'm > using a repository to share non maven artifacts (e.g. arbitrary zip files). > It's going to be easier to configure the HTTP server to use sha1 ETags than > to put .sha1 files alongside the artifact. It may also enable conditional > GETs, but I don't know enough about the algorithm to say so. > > I see this as being a property of the resolver: > > repositories { > http { > url = «» > checksums.sha1Etags() > } > } > > As an example. > > -- > Luke Daley > Principal Engineer, Gradleware > http://gradleware.com > > -- Darrell (Daz) DeBoer Principal Engineer, Gradleware http://www.gradleware.com
