On 25 Sep 2014, at 6:12 am, Daz DeBoer <darrell.deb...@gradleware.com> wrote:
> So I'm battling a bit as to whether we should really be modelling these
> replacements at the component level (using rules as suggested) or should be
> designing these at a higher level, on some sort of new model concept.
>
> Reviewing some of the old emails on this, I found this from Adam:
>
> I think it’s still an open question as to whether we use the component
> (instance) meta-data rules or introduce a new type of meta data for streams
> of work and how they map to modules over time.
> If we decide to add this to component meta-data then we would use whatever
> DSL happens to be available for defining the component meta-data, which
> happens to be `dependencies.components.eachComponent`. We’d later add the
> modules(_) { … } convenience DSL.
>
> So the alternative to component metadata rules will be modelling some new
> data structure, which we've done in the current implementation with
> "component module" (ComponentModuleMetadata). This doesn't quite represent a
> 'stream of work', but perhaps it could evolve in that way.
>
> So the big question in my mind is do we:
> a) Implement 'replaces/replacedBy' at the component instance level, and
> configure it via component metadata rules
> OR
> b) Do a bit more work to model component replacements in the context of a
> 'stream of work' and clearly separate the DSL from the metadata rules.
It feels like ultimately this is something we need to know at the component
instance level. Given an instance x, we need to know:
1. For each instance y that also appears in the graph, do x and y conflict?
2. When x and y conflict, which one should we select? Do we replace x with y or
y with x?
3. When x is replaced by y, which other instances must also be included in the
result?
4. When x is replaced by y, which other instances must also be replaced by y?
#3 is for the case where a module is busted up into several others. #4 is for
the case where several modules are combined into a single module. This can
happen over time (eg groovy 1.x was busted up into pieces from groovy 2.0 and
later) and can happen as an aggregation (eg if I choose groovy-all I need to
pull all of the groovy modules out of the result).
So this is the kind of model we want DependencyGraphBuilder to work with. For
now, it doesn’t need to be anywhere near this general, as there are performance
implications to trying to make this work generally with any arbitrary set of
instances.
From the build author’s point of view, we want a different model. We should be
able to declare concrete and specific facts in the DSL and use those facts to
derive answers to the above questions. We don’t want to expose the above
questions in the DSL, at least not yet.
The model we want to aim for is one where we describe the history of a
component instance: where did this instance come from?
So, we have a few options for the user model:
1. This is just more stuff you add to the component instance meta-data. Each
component instance would describe its full history - all the other component
instances that it is newer than (ie replaces). Not literally - you’d use some
kind of declarative convenience. You’d use the existing component meta-data
rules to attach this history.
2. This is stuff you add to the meta-data about a module/component. We’d use a
new module meta-data rules DSL for this.
3. Some combination of the two, where there isn’t any concept of a
module/component, but there is a separate convenience DSL to describe the
history of all components instances with a particular module id, without
requiring the meta-data of each instance.
The big downside of #1 is when there are transitive changes. For example: a was
replaced by b and then split into (c,d). Given this, if I end up with a and d
in the graph, I need to figure out that a and d conflict and that I need to
replace a with c and d.
With #1, we’d need to find some meta-data for an instance of b in order to
apply the rule, even though b is never referenced in the graph. So I reckon #1
is out.
From a user’s point of view, there probably isn’t much difference between #2
and #3.
>
>
> On Wed, Sep 24, 2014 at 10:41 AM, Szczepan Faber <szcze...@gmail.com> wrote:
> Awesome feedback! Let's skype on it today.
>
> On Wed, Sep 24, 2014 at 5:50 PM, Daz DeBoer
> <darrell.deb...@gradleware.com> wrote:
> > Thanks Szczepan!
> >
> > I've finally found the time to get fully across this feature. I haven't yet
> > looked into the implementation.
> >
> > Here's some initial feedback on the model.
> >
> > 1. We've named this feature 'component replacements', but the modelling is
> > done at the "module" level (ComponentModuleMetadata). I don't think this is
> > ideal: I'd like to think about a 'replacement' being modelled at the
> > component level ("org:name:version" instead of "org:name"). In most cases
> > we'll consider all components with the same 'org:name as having the same
> > replacements, and the DSL would make it convenient to define this. But the
> > underlying model should associate 'replacements' with a component, I think.
> > 2. Eventually, we'd like a published component to contain replacement
> > information. For this to be convenient, we'd normally publish a component
> > and say it 'replaces' or 'supercedes' a set of other components. I think it
> > might be helpful to model the relationship in this way, using 'replaces'
> > rather than 'replaced by'.
> >
> > And on the DSL:
> > The DSLs for all of our 'rules' that apply to dependency resolution are
> > converging on a pattern that will hopefully allow them to be migrated into
> > the new configuration rule infrastructure at some stage. The pattern is:
> >
> > model-rule-type {
> > eachXXX { SomeMutableType mutable[, OtherType immutableInput]* ->
> > }
> > module("group:name") { SomeMutableType mutable[, OtherType
> > immutableInput]* ->
> > }
> > }
> >
> > Exceptions:
> > - For component selection rules, 'all' is used instead of
> > 'eachComponentSelection'.
> > - I'm thinking this works better as a partner method to 'module()'
> > - For component metadata rules, the module() method doesn't exist yet.
> > - For dependency resolve rules, the 'model-rule-type' containing element
> > doesn't exist.
> >
> > If we stick with the same pattern for component replacements, we'd have:
> >
> > dependencies {
> > components {
> > module("com.google.guava:guava") { ComponentSubstitutionDetails
> > component ->
> >
> > component.replacesModule("com.google.collections:google-collections")
> > }
> > }
> > }
> >
> > That's it for now. Thanks for restarting the discussion!
> > Daz
> >
> > On Wed, Sep 24, 2014 at 3:31 AM, Szczepan Faber <szcze...@gmail.com> wrote:
> >>
> >> Upcoming Gradle 2.2 contains new incubating component replacement rules:
> >>
> >> dependencies {
> >> components {
> >>
> >> module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
> >> }
> >> }
> >>
> >> See more in http://www.gradle.org/docs/nightly/release-notes. This
> >> declaration is used during conflict resolution, Gradle will prevent
> >> both collections and guava appear in the same dependency tree,
> >> preferring any version of guava over every version of collections.
> >> Most importantly, if the dependency tree _only_ contains collections
> >> it will _not_ be replaced (because there is no conflict). Multiple
> >> modules can have the same target replacement. However, we don't
> >> support (yet) having single module replaced by multiple modules.
> >>
> >> The full DSL in the current form, based on our previous discussion on
> >> the mailing list. I'm starting new email thread on purpose (for good
> >> or bad, let's see whether it helps).
> >>
> >> dependencies {
> >> components {
> >> //declaring replacement:
> >>
> >> module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
> >> module(someModuleIdentifier).replacedBy("com.google.guava:guava")
> >>
> >> //querying for the replacement target:
> >> ModuleIdentifier id =
> >> module("com.google.collections:google-collections").getReplacedBy()
> >>
> >> //querying for the replacement source:
> >> ModuleIdentifier id =
> >> module("com.google.collections:google-collections").getId()
> >> }
> >> }
> >>
> >> Javadoc (for interface names, etc.):
> >>
> >> http://www.gradle.org/docs/nightly/javadoc/org/gradle/api/artifacts/dsl/ComponentMetadataHandler.html#module(java.lang.Object)
> >>
> >> This DSL can grow to accommodate features like:
> >> a) replacing single module with a set of modules. I'd love to have
> >> this. I also want to make incremental progress.
> >> b) other component module metadata (e.g. releasable units, impl module
> >> consistent with api module). I'd love to have this, too.
> >>
> >> Let's confirm this API and/or make changes to it before 2.2 release.
> >> Other related APIs that we should have in mind (for consistency):
> >>
> >> 1. component selection rules:
> >>
> >> configurations.conf.resolutionStrategy {
> >> componentSelection {
> >> all { ComponentSelection selection ->
> >> if (selection.candidate.group == 'org.sample') {
> >> selection.reject("rejecting experimental")
> >> }
> >> }
> >> module("org.sample:api") { ComponentSelection selection ->
> >> if (selection.candidate.version == "1.1") {
> >> selection.reject("known bad version")
> >> }
> >> }
> >> }
> >> }
> >>
> >> 2. component metadata rules:
> >>
> >> dependencies {
> >> components {
> >> eachComponent { ComponentMetadataDetails details,
> >> IvyModuleDescriptor ivyModule ->
> >> if (details.id.group == 'my.org' && ivyModule.branch ==
> >> 'testing') {
> >> details.changing = true
> >> }
> >> }
> >> }
> >> }
> >>
> >> Cheers!
> >> --
> >> Szczepan Faber
> >> Core dev@gradle; Founder@mockito
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe from this list, please visit:
> >>
> >> http://xircles.codehaus.org/manage_email
> >>
> >>
> >
> >
> >
> > --
> > Darrell (Daz) DeBoer
> > http://www.gradleware.com
>
>
>
> --
> Szczepan Faber
> Core dev@gradle; Founder@mockito
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
> http://xircles.codehaus.org/manage_email
>
>
>
>
>
> --
> Darrell (Daz) DeBoer
> http://www.gradleware.com
--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com
