I was asked by the client to perform a quick spike on conflict resolution
rules. I decided spike it first before kicking off a discussion. Here're my
findings.
Intro.
Currently, a version conflict means that there are modules with the same
group+name but different version. However, there is also a different class
of conflicts - different modules (different group or name) are conflicting
and only one of them should be chosen. In the spike, the ResolutionStrategy
a) allows declaring that certain modules are conflicting and b) allows to
choose preferred module version in case of such conflict.
Some specific use cases I found so far:
1. 'com.google.collections:google-collections' and
'com.google.guava:guava' are conflicting, prefer guava.
2. 'org.jboss.netty:netty' and 'io.netty:netty*' are conflicting, prefer
io.netty.
3. 'org.springframework:spring' and 'org.springframework:spring-*' are
conflicting, prefer 'org.springframework:spring-*' (for example,
spring-core).
4. 'kafka:kafka_2.8.2' VS 'kafka:kafka_2.10' VS 'kafka:kafka' are
conflicting, prefer _2.10 (say)
5. A team in an organization decided to extract a standalone project out
of a bigger project. The group id of the modules moved to a new project
needed to change. Now there's a risk that consumers will have problems with
conflict resolution.
In a detached project, solving above conflicts if fairly easy: figure out
where those dependencies are coming from, update/upgrade the dependencies;
alternatively, add some module exclusions or version forcing rules to the
build script.
In a corporate environment I'm looking for a solution that can be rolled
out to all projects, instead of fixing the problem again and again when it
pops up in various projects across the organization. With the current state
of Gradle, the approach I use currently is a combination of validation
(reading the graph via the ResolutionResult API) + module forcing
(dependency resolve rules, forced versions):
1. Corporate plugin suite contains dependency violation rule that fail the
build when google-collections and guava appear both in the same classpath.
Typically when this failure happens, google-collections is pulled in
transitively. The build fails and the user is asked to 'manually' fix the
conflict. Typically this is done by doing something like allprojects {
configurations.all { exclude ... It could be fixed in a smarter way, with
selective exclusion and with some smart reuse of dependency declarations to
avoid duplication. In a real world however, most often I see global
excludes. This works but it's not very clean. Also, the solution contains a
break-fix cycle that could be avoided if corporate plugin ships with
specific conflict resolution rules.
2. There's a conflict between kafka:kafka and kafka:kafka_2.10 (say). The
conflict is detected and the build fails asking the user to 'manually' fix
the conflict. Sometimes the conflict can be fixed by updating/upgrading the
dependencies so that the conflict goes away. Sometimes it's not
possible/practical and the user configures a dependency resolution rule to
force specific kafka version/variant. This works, but it's not especially
clean and it's another break-fix cycle.
In a quick spike, I came up below dsl. I'm not really fishing for dsl
feedback because it's too early and the model is not yet discussed enough.
I'm showing it for better understanding:
resolutionStrategy.conflict {
modules 'com.google.guava:guava',
'com.google.collections:google-collections'
resolution { it.name == 'guava' } // it - ModuleVersionIdentifier
}
resolutionStrategy.conflict {
modules 'org.springframework:spring'
modules { it.group == 'org.springframework' &&
it.name.startsWith('spring-') } // it - ModuleIdentifier
resolution { it.name.startsWith 'spring-' }
}
Nitty gritty: The heart of the dependency resolution is our dear
DependencyGraphBuilder. It gets extra information from the
ResolutionStrategy:
- for given module, what are the conflicting modules (e.g.
Spec<ModuleIdentifier>)
- for a set of conflicting module versions, which one to choose (a
Spec<ModuleVersionIdentifier>)
The spike lives in 'sf-conflict' branch, there's one commit there:
https://github.com/gradle/gradle/commit/6dcfd9760017999d4659e6d2fd0e9b7ac5b1e461
It does seem to work in the projects where I was trying out the spike
(there's also a rudimentary integ test committed). It does not merge with
Daz' changes because I did the spike early last week.
Thoughts? Feedback?
--
Szczepan Faber
Principal engineer@gradle; Founder@mockito
Join us for Gradle Summit 2014, June 12th and 13th in Santa Clara, CA:
http://www.gradlesummit.com
