On 11/12/2011, at 9:37 PM, Daz DeBoer wrote:
> G'day
>
> We've been working on enhancing our dependency resolution so that we don't
> download artifact files unnecessarily. This is done by:
> 1) Look for any candidate artifacts that we suspect may be the one we want
> but we're not certain
> 2) Calculate the SHA1 hashes of these candidate artifacts
> 3) Request a "*.sha1" file from the server, and comparing the declared SHA1
> value against each of the candidate artifacts
> 4) If one of the candidate matches the SHA1 value on the server, we use the
> candidate. If not, we download.
>
> Searching for candidates involves looking for previously downloaded artifacts
> that have the same ArtifactId as the one we're looking for: this is
> effectively [org]:[module]:[revision]:[type]:[artifact-name]:[extension].
> We currently search in:
> - The artifact cache of the current Gradle version, looking for artifacts
> that have been downloaded from a different repository
> - The artifact cache of the previous Gradle version, allowing us to upgrade
> the cache format without requiring re-download
> In the future, the plan is to look in ~/.m2 as well. This will make the
> out-of-the-box experience better for people switching from maven.
>
> So in all, this should reduce the number of downloads required when:
> - You upgrade Gradle versions
> - You use virtual repositories in Artifactory, with different URLs mapping to
> the same artifacts
> - You switch to use a repository manager (like Artifactory) from accessing
> repositories directly
>
> The nice thing about this model is that we only request the sha1 file from
> the server if we have a likely candidate to match against.
> This currently only works when the server has a .sha1 file for the requested
> artifact. The plan is to extend this to include md5 hash files, as well as
> custom HTTP headers like those provided by Artifactory.
Awesome stuff Daz.
What are the special Artifactory headers? As we discussed before, I think some
kind of switch to indicate that the repository uses sha1 for ETags would be
better. This would work in more situations. For example, if I'm using a
repository to share non maven artifacts (e.g. arbitrary zip files). It's going
to be easier to configure the HTTP server to use sha1 ETags than to put .sha1
files alongside the artifact. It may also enable conditional GETs, but I don't
know enough about the algorithm to say so.
I see this as being a property of the resolver:
repositories {
http {
url = «»
checksums.sha1Etags()
}
}
As an example.
--
Luke Daley
Principal Engineer, Gradleware
http://gradleware.com
