On 05/09/2012, at 1:06 PM, Adam Murdoch wrote:
>
> On 05/09/2012, at 10:18 AM, Adam Murdoch wrote:
>
>>
>> On 05/09/2012, at 6:55 AM, Hans Dockter wrote:
>>
>>>
>>>
>>> On Tuesday, September 4, 2012, Adam Murdoch wrote:
>>> Hi,
>>>
>>> One question we need to answer soon is how variants should be mapped to Ivy
>>> and Maven repositories. Have a look at the dependency model spec for a
>>> definition of variant:
>>> https://github.com/gradle/gradle/blob/master/design-docs/dependency-model.md#variant
>>>
>>> We want to define some conventions (or a standard in the cases where
>>> there's no obvious convention), and build some plugins that can do useful
>>> stuff based on these. As always, the infrastructure needs to be flexible to
>>> some degree, so you can choose to use some other convention (but the
>>> further you go from our convention, the more work you have to do).
>>>
>>> Some concrete examples we're interested in:
>>> * I want to publish Groovy 1.8 and Groovy 2.0 variants of my Groovy library.
>>> * I want to publish x86 and amd64 variants of my native library.
>>> * I want to publish minified and non-minified variants of my Javascript
>>> library.
>>>
>>> There's another dimension that we're not interested in here (it's a
>>> separate discussion), and that is packaging:
>>> * I want to publish my Groovy library as both a jar and as a distribution.
>>> * I want to publish both a jar and a war from my Java project.
>>>
>>> There are a few basic options for mapping component variants to a
>>> repository:
>>>
>>> 1. Publish every variant as a single module. So, for my native library,
>>> Gradle would publish module 'my-org:my-lib' that contains both the x86 and
>>> amd64 binaries, the header archive, maybe source and API documentation
>>> archives, plus meta-data for both variants.
>>>
>>> For a maven repository, this might look like:
>>>
>>> my-org/my-lib/1.2
>>> pom.xml
>>> my-lib-1.2-x86.so
>>> my-lib-1.2-amd64.so
>>> my-lib-1.2-cpp-headers.zip
>>> my-lib-1.2-source.zip
>>> my-lib-1.2-doxygen.zip
>>>
>>> For an ivy repository, it would look more or less the same. The artefacts
>>> might have different names - e.g. my-lib-x86-1.2.so instead of
>>> my-lib-1.2-x86.so.
>>>
>>> 2. Publish every variant as a separate module. So, for my native library,
>>> Gradle would publish module 'my-org:my-lib-x86' that contains the x86
>>> binaries, and 'my-org:my-lib-amd64' for the amd64 binaries. Variant
>>> independent archives would go in both places.
>>>
>>> For a maven repository, this might look like:
>>>
>>> my-org/my-lib-x86/1.2
>>> pom.xml
>>> my-lib-1.2.so
>>> my-lib-1.2-cpp-headers.zip
>>> my-lib-1.2-source.zip
>>> my-lib-1.2-doxygen.zip
>>>
>>> 3. Publish each variant as a separate module, plus publish a
>>> variant-independent module. For my native library, Gradle would publish
>>> 'my-org:my-lib-x86' for the x86 binaries and meta-data,
>>> 'my-org:my-lib-amd64' for the amd64 binaries and meta-data, and
>>> 'my-org:my-lib' for the headers, source, documentation and meta-data about
>>> the available variants.
>>>
>>> For a maven repository, this might look like:
>>>
>>> my-org/my-lib-x86/1.2
>>> pom.xml
>>> my-lib-1.2.so
>>>
>>> my-org/my-lib/1.2
>>> pom.xml
>>> my-lib-1.2-cpp-headers.zip
>>> my-lib-1.2-source.zip
>>> my-lib-1.2-doxygen.zip
>>>
>>> Option 1. has some downsides
>>> * Often, the meta-data for each variant is different. For example, my
>>> Groovy 1.8 variant depends on groovy:1.8, and my Groovy 2.0 variant depends
>>> on groovy:2.0. Or, in native space, my windows variants need library a, and
>>> my linux variants need completely different library b. For this option,
>>> there's a single descriptor that we have to jam everything into.
>>>
>>> Which is impossible with Maven. Though I think the non Java world is not
>>> strongly bound to Maven. So why not have it in one descriptor in the case
>>> of Ivy. If we would model our own descriptor, how would we do it? Having
>>> one descriptor provides one place to get all the Metadata you need.
>>
>> I'm tending towards separate meta-data files for each variant.
>
> Which, of course, doesn't necessarily mean separate modules. So, let's add a
> few more options:
>
> Option 4. Publish all variants as a single module and include Gradle specific
> meta-data:
>
> my-org/my-lib/1.2
> pom.xml
> my-lib-1.2-component.xml (for example. This is the component meta-data)
> my-lib-1.2-windows-x86.xml (for example. This is the variant meta-data)
> my-lib-1.2-windows-x86.dll
> my-lib-1.2-windows-x86.lib
> my-lib-1.2-linux-amd64.xml (for example, This is the variant meta-data)
> my-lib-1.2-linux-amd64.so
> my-lib-1.2-cpp-headers.zip
> my-lib-1.2-sources.zip
>
> The pom.xml would be mostly empty. The module would be unusable from anything
> other than Gradle.
>
> Option 5. Publish all variants as a single module and choose one as the
> 'default' to be used by other build tools:
>
> my-org/my-lib/1.2
> pom.xml
> my-lib-1.2-component.xml
> my-lib-1.2-groovy18.xml
> my-lib-1.2.jar (the Groovy 1.8 variant)
> my-lib-1.2-groovy20.xml
> my-lib-1.2-groovy20.jar
> my-lib-1.2-sources.zip
>
> The pom.xml would contain meta-data for the Groovy 1.8 variant. You can't
> (easily) use the Groovy 2.0 variant from anything other than Gradle.
>
> Option 6. Publish all variants as a single module. When publishing jvm-based
> libraries to a Maven repository, also publish each variant in a separate
> Maven-usable module:
>
> my-org/my-lib/1.2
> pom.xml (mostly empty)
> my-lib-1.2-component.xml
> my-lib-1.2-groovy18.xml
> my-lib-1.2-groovy18.jar
> my-lib-1.2-groovy20.xml
> my-lib-1.2-groovy20.jar
> my-lib-1.2-sources.zip
>
> my-org/my-lib-groovy18/1.2
> pom.xml (Groovy 1.8 variant meta-data)
> my-lib-1.2.jar
> my-lib-1.2-sources.zip
>
> my-org/my-lib-groovy20/1.2
> pom.xml (Groovy 2.0 variant meta-data)
> my-lib-1.2.jar
> my-lib-1.2-sources.zip
>
> This is a combo of option 3 and option 4, and it feels a bit better than
> either. Some extra duplication, though, but this would be relatively easy to
> make tweakable. This could be the default wiring, and you can use the DSL to
> add or remove artefacts from each of the modules as you choose.
>
> Or, option 4 could be the default wiring, and you can use the DSL to ask for
> a maven-usable wiring to be added over the top.
Another thing to consider is how we resolve a dependency against these mappings.
The idea is that you should keep dependency declarations independent of both
variant and packaging. So:
dependencies {
compile 'my-org:my-lib:1.2'
}
This should resolve to my-lib-1.2-groovy18.jar when I'm compiling for Groovy
1.8 and my-lib-1.2-groovy20.jar when I'm compiling for Groovy 2.0.
Let's assume option 6. So, when resolving compile-time dependencies for Groovy
1.8 and Java runtime 1.6, given a dependency declaration 'my-org:my-lib:1.2',
then we might do something like this:
1. Attempt to download my-org/my-lib/1.2/my-lib-1.2-component.xml
1.1. If present, use this to determine which variants are compatible with
Groovy 1.8 and Java runtime 1.6. If none, fail.
1.2. Let's say it maps to variant 'groovy18'. Download
my-org/my-lib/1.2/my-lib-1.2-groovy18.xml
1.3. Use this to determine which artefacts + dependencies are required for
compilation.
2. Attempt to download my-org/my-lib/1.2/my-lib-1.2-variant.xml (for the module
per variant cases)
2.1 If present, use this to validate that this variant is compatible with
Groovy 1.8 and Java runtime 1.6. If not, fail.
2.2 Use this to determine which artefacts + dependencies are required for
compilation.
3. Attempt to download my-org/my-lib/1.2/pom.xml
3.1 If present, use this to determine which dependencies are required for
compilation. Assume my-lib-1.2.jar to be used for compilation.
3.2 Validate that this module depends on a compatible version of Groovy, or
no Groovy at all. If not, fail.
4. Check if my-org/my-lib/1.2/my-lib-1.2.jar exists
4.1 If so, assume my-lib-1.2.jar and no dependencies are to be used for
compilation. Assume the jar is compatible with Groovy 1.8 and Java runtime 1.6.
5. Module not found.
Logically, this is what's going on. The implementation may choose to use a
directory listing of my-org/my-lib/1.2 to skip to a certain step.
For Ivy, substitute in a slightly modified step 3, where we check for Ivy.xml
and use it to determine dependencies and artefacts used for compilation.
Step 2 is only required if we publish module per variant (either option 3 or
option 6).
Step 4 will be removed at some point, or at least moved out of the default
search. Too many assumptions.
Dynamic version resolution (let's lump Maven snapshots in there) would not need
to change.
--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com
