On 26 Sep 2014, at 4:44 am, Daz DeBoer <darrell.deb...@gradleware.com> wrote:
> On Wed, Sep 24, 2014 at 4:19 PM, Adam Murdoch <adam.murd...@gradleware.com>
> wrote:
>
> On 25 Sep 2014, at 6:12 am, Daz DeBoer <darrell.deb...@gradleware.com> wrote:
>
>> So I'm battling a bit as to whether we should really be modelling these
>> replacements at the component level (using rules as suggested) or should be
>> designing these at a higher level, on some sort of new model concept.
>>
>> Reviewing some of the old emails on this, I found this from Adam:
>>
>> I think it’s still an open question as to whether we use the component
>> (instance) meta-data rules or introduce a new type of meta data for streams
>> of work and how they map to modules over time.
>> If we decide to add this to component meta-data then we would use whatever
>> DSL happens to be available for defining the component meta-data, which
>> happens to be `dependencies.components.eachComponent`. We’d later add the
>> modules(_) { … } convenience DSL.
>>
>> So the alternative to component metadata rules will be modelling some new
>> data structure, which we've done in the current implementation with
>> "component module" (ComponentModuleMetadata). This doesn't quite represent
>> a 'stream of work', but perhaps it could evolve in that way.
>>
>> So the big question in my mind is do we:
>> a) Implement 'replaces/replacedBy' at the component instance level, and
>> configure it via component metadata rules
>> OR
>> b) Do a bit more work to model component replacements in the context of a
>> 'stream of work' and clearly separate the DSL from the metadata rules.
>
> It feels like ultimately this is something we need to know at the component
> instance level. Given an instance x, we need to know:
>
> 1. For each instance y that also appears in the graph, do x and y conflict?
> 2. When x and y conflict, which one should we select? Do we replace x with y
> or y with x?
> 3. When x is replaced by y, which other instances must also be included in
> the result?
> 4. When x is replaced by y, which other instances must also be replaced by y?
>
> #3 is for the case where a module is busted up into several others. #4 is for
> the case where several modules are combined into a single module. This can
> happen over time (eg groovy 1.x was busted up into pieces from groovy 2.0 and
> later) and can happen as an aggregation (eg if I choose groovy-all I need to
> pull all of the groovy modules out of the result).
>
> So this is the kind of model we want DependencyGraphBuilder to work with. For
> now, it doesn’t need to be anywhere near this general, as there are
> performance implications to trying to make this work generally with any
> arbitrary set of instances.
>
> From the build author’s point of view, we want a different model. We should
> be able to declare concrete and specific facts in the DSL and use those facts
> to derive answers to the above questions. We don’t want to expose the above
> questions in the DSL, at least not yet.
>
> The model we want to aim for is one where we describe the history of a
> component instance: where did this instance come from?
>
> So, we have a few options for the user model:
>
> 1. This is just more stuff you add to the component instance meta-data. Each
> component instance would describe its full history - all the other component
> instances that it is newer than (ie replaces). Not literally - you’d use some
> kind of declarative convenience. You’d use the existing component meta-data
> rules to attach this history.
>
> 2. This is stuff you add to the meta-data about a module/component. We’d use
> a new module meta-data rules DSL for this.
>
> 3. Some combination of the two, where there isn’t any concept of a
> module/component, but there is a separate convenience DSL to describe the
> history of all components instances with a particular module id, without
> requiring the meta-data of each instance.
>
> The big downside of #1 is when there are transitive changes. For example: a
> was replaced by b and then split into (c,d). Given this, if I end up with a
> and d in the graph, I need to figure out that a and d conflict and that I
> need to replace a with c and d.
>
> With #1, we’d need to find some meta-data for an instance of b in order to
> apply the rule, even though b is never referenced in the graph. So I reckon
> #1 is out.
>
> That's a fair argument. So in effect it's not enough to know the metadata of
> the 2 nodes involved in the potential conflict, we'd also need metadata for
> intervening nodes in a replacement chain. Given this, maybe we should go with
> something very similar to Szczepan's original work, but using a different
> enclosing block (not 'components'). This block would represent logic where we
> deal with groups of components, be that modules, releasable units, etc.
>
> Something like:
>
> dependencies {
> units { // Need a better name for this
I’d just call them ‘modules’ for now. The logic applies to components that
happen to be published as modules in an ivy/maven repo.
>
> module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
> releasableUnit("spring2") {
> module("org.springframework:spring-core")
> module("org.springframework:spring-beans")
> // Would need to list the entire set here, for the
> 'replacedBy' to work.
> }
> module("org.springframework:spring").replacedBy("spring2”)
I would make these blocks:
module(“…”) { ModuleMetaDataDetails module ->
… some stuff about the module ...
}
eachModule { ModuleMetaDataDetails module -> … }
> }
> }
>
> Alternatively, we could go with using 'components' block, but ensure that all
> component-instance-rule methods are clearly named:
>
> dependencies {
> components {
> // component-instance-rules
> eachComponent { .. rule .. }
> eachComponent.forModule("org.google.collections:google-collections")
> { ... }
>
> // model configuration as above
>
> module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
> releasableUnit("spring2") {
> ....
> }
> }
>
> Feedback? We need to come up with a reasonable pattern for this DSL for 2.2
> Daz
>
>
> From a user’s point of view, there probably isn’t much difference between #2
> and #3.
>
>
>>
>>
>> On Wed, Sep 24, 2014 at 10:41 AM, Szczepan Faber <szcze...@gmail.com> wrote:
>> Awesome feedback! Let's skype on it today.
>>
>> On Wed, Sep 24, 2014 at 5:50 PM, Daz DeBoer
>> <darrell.deb...@gradleware.com> wrote:
>> > Thanks Szczepan!
>> >
>> > I've finally found the time to get fully across this feature. I haven't yet
>> > looked into the implementation.
>> >
>> > Here's some initial feedback on the model.
>> >
>> > 1. We've named this feature 'component replacements', but the modelling is
>> > done at the "module" level (ComponentModuleMetadata). I don't think this
>> > is
>> > ideal: I'd like to think about a 'replacement' being modelled at the
>> > component level ("org:name:version" instead of "org:name"). In most cases
>> > we'll consider all components with the same 'org:name as having the same
>> > replacements, and the DSL would make it convenient to define this. But the
>> > underlying model should associate 'replacements' with a component, I think.
>> > 2. Eventually, we'd like a published component to contain replacement
>> > information. For this to be convenient, we'd normally publish a component
>> > and say it 'replaces' or 'supercedes' a set of other components. I think it
>> > might be helpful to model the relationship in this way, using 'replaces'
>> > rather than 'replaced by'.
>> >
>> > And on the DSL:
>> > The DSLs for all of our 'rules' that apply to dependency resolution are
>> > converging on a pattern that will hopefully allow them to be migrated into
>> > the new configuration rule infrastructure at some stage. The pattern is:
>> >
>> > model-rule-type {
>> > eachXXX { SomeMutableType mutable[, OtherType immutableInput]* ->
>> > }
>> > module("group:name") { SomeMutableType mutable[, OtherType
>> > immutableInput]* ->
>> > }
>> > }
>> >
>> > Exceptions:
>> > - For component selection rules, 'all' is used instead of
>> > 'eachComponentSelection'.
>> > - I'm thinking this works better as a partner method to 'module()'
>> > - For component metadata rules, the module() method doesn't exist yet.
>> > - For dependency resolve rules, the 'model-rule-type' containing element
>> > doesn't exist.
>> >
>> > If we stick with the same pattern for component replacements, we'd have:
>> >
>> > dependencies {
>> > components {
>> > module("com.google.guava:guava") { ComponentSubstitutionDetails
>> > component ->
>> >
>> > component.replacesModule("com.google.collections:google-collections")
>> > }
>> > }
>> > }
>> >
>> > That's it for now. Thanks for restarting the discussion!
>> > Daz
>> >
>> > On Wed, Sep 24, 2014 at 3:31 AM, Szczepan Faber <szcze...@gmail.com> wrote:
>> >>
>> >> Upcoming Gradle 2.2 contains new incubating component replacement rules:
>> >>
>> >> dependencies {
>> >> components {
>> >>
>> >> module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
>> >> }
>> >> }
>> >>
>> >> See more in http://www.gradle.org/docs/nightly/release-notes. This
>> >> declaration is used during conflict resolution, Gradle will prevent
>> >> both collections and guava appear in the same dependency tree,
>> >> preferring any version of guava over every version of collections.
>> >> Most importantly, if the dependency tree _only_ contains collections
>> >> it will _not_ be replaced (because there is no conflict). Multiple
>> >> modules can have the same target replacement. However, we don't
>> >> support (yet) having single module replaced by multiple modules.
>> >>
>> >> The full DSL in the current form, based on our previous discussion on
>> >> the mailing list. I'm starting new email thread on purpose (for good
>> >> or bad, let's see whether it helps).
>> >>
>> >> dependencies {
>> >> components {
>> >> //declaring replacement:
>> >>
>> >> module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
>> >> module(someModuleIdentifier).replacedBy("com.google.guava:guava")
>> >>
>> >> //querying for the replacement target:
>> >> ModuleIdentifier id =
>> >> module("com.google.collections:google-collections").getReplacedBy()
>> >>
>> >> //querying for the replacement source:
>> >> ModuleIdentifier id =
>> >> module("com.google.collections:google-collections").getId()
>> >> }
>> >> }
>> >>
>> >> Javadoc (for interface names, etc.):
>> >>
>> >>http://www.gradle.org/docs/nightly/javadoc/org/gradle/api/artifacts/dsl/ComponentMetadataHandler.html#module(java.lang.Object)
>> >>
>> >> This DSL can grow to accommodate features like:
>> >> a) replacing single module with a set of modules. I'd love to have
>> >> this. I also want to make incremental progress.
>> >> b) other component module metadata (e.g. releasable units, impl module
>> >> consistent with api module). I'd love to have this, too.
>> >>
>> >> Let's confirm this API and/or make changes to it before 2.2 release.
>> >> Other related APIs that we should have in mind (for consistency):
>> >>
>> >> 1. component selection rules:
>> >>
>> >> configurations.conf.resolutionStrategy {
>> >> componentSelection {
>> >> all { ComponentSelection selection ->
>> >> if (selection.candidate.group == 'org.sample') {
>> >> selection.reject("rejecting experimental")
>> >> }
>> >> }
>> >> module("org.sample:api") { ComponentSelection selection ->
>> >> if (selection.candidate.version == "1.1") {
>> >> selection.reject("known bad version")
>> >> }
>> >> }
>> >> }
>> >> }
>> >>
>> >> 2. component metadata rules:
>> >>
>> >> dependencies {
>> >> components {
>> >> eachComponent { ComponentMetadataDetails details,
>> >> IvyModuleDescriptor ivyModule ->
>> >> if (details.id.group == 'my.org' && ivyModule.branch ==
>> >> 'testing') {
>> >> details.changing = true
>> >> }
>> >> }
>> >> }
>> >> }
>> >>
>> >> Cheers!
>> >> --
>> >> Szczepan Faber
>> >> Core dev@gradle; Founder@mockito
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe from this list, please visit:
>> >>
>> >> http://xircles.codehaus.org/manage_email
>> >>
>> >>
>> >
>> >
>> >
>> > --
>> > Darrell (Daz) DeBoer
>> > http://www.gradleware.com
>>
>>
>>
>> --
>> Szczepan Faber
>> Core dev@gradle; Founder@mockito
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>> http://xircles.codehaus.org/manage_email
>>
>>
>>
>>
>>
>> --
>> Darrell (Daz) DeBoer
>> http://www.gradleware.com
>
>
> --
> Adam Murdoch
> Gradle Co-founder
> http://www.gradle.org
> CTO Gradleware Inc. - Gradle Training, Support, Consulting
> http://www.gradleware.com
>
>
>
>
>
>
> --
> Darrell (Daz) DeBoer
> http://www.gradleware.com
--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com
