On 17/11/2012, at 3:27 AM, Luke Daley wrote:
>
> On 16/11/2012, at 5:57 AM, Adam Murdoch wrote:
>
>> Hi,
>>
>> One issue we need to sort out with the new publication DSL is what should be
>> published by default, and how the user can override this.
>>
>> So, for example, if I apply both the 'java' and 'war' plugins, what should
>> be published by default? The war and its meta-data? The war + jar + their
>> meta-data? Nothing? How do I change this? What happens when I then apply the
>> 'ear' plugin?
>>
>> Another question is whether we make a distinction between those publications
>> that should end up in a repository and those publications that should be
>> used locally within the same build.
>>
>> So, for example, if I apply both the 'java' and 'war' plugin and I specify
>> (however it is I do this) that only the war should be published (to a
>> repository), should the jar + meta-data still be available via a project
>> dependency? How do I change this?
>>
>> Right now, we use a series of hacks that don't really work. I think we
>> should take this opportunity to move away from this and fix some issues
>> while we're there.
>
> It would be a shame to drag this mess with us into the new world.
>
>> Given that we want to move to a high-level model that describes components
>> and their relationships, I think the solution should start us down this
>> path. Components feel like a good solution here.
>>
>> So let's say that when you apply the 'java' plugin, you're declaring that
>> the project produces a Java-based component of some kind. Some kind of
>> JavaBasedComponent instance is created and shoved in a container somewhere.
>> And when you apply the 'war' plugin, you're declaring that the project
>> produces an J2EE based web application. Some kind of J2EEWebApplication
>> instance is created and shoved in a container somewhere.
>>
>> We have a few options as to what to publish:
>>
>> 1. Infer the 'main' component and publish this, as we do now. To publish an
>> additional component, you'd add it to the appropriate publication:
>>
>> This would publish the web application as a war + meta-data, as we do now
>> (sort of):
>>
>> apply plugin: 'java'
>> apply plugin: 'war'
>> apply plugin: 'ivy-publish'
>>
>> This would publish the web application as a war + meta-data and the java
>> component as a jar + meta-data:
>>
>> apply plugin: 'java'
>> apply plugin: 'war'
>> apply plugin: 'ivy-publish'
>>
>> publishing.publications.ivy {
>> publish components.java // a few too many 'publish'es in there, but you
>> get the idea
>> }
>>
>> 2. Don't publish anything. You need to declare it, perhaps something like:
>>
>> apply plugin: 'java'
>> apply plugin: 'war'
>>
>> publishing.publications.ivy {
>> publish components.java
>> publish components.webApplication
>> }
>>
>> 3. Split the 'java' plugin into a Java language plugin that knows how to
>> compile and test Java based stuff, and a Java library plugin that knows how
>> to publish a Java library for use outside the project.
>>
>> This does not publish anything:
>>
>> apply plugin: 'java'
>>
>> This would publish the Java library as a jar + meta-data:
>>
>> apply plugin: 'java'
>> apply plugin: 'java-library'
>>
>> This would publish the web application as a war + meta-data:
>>
>> apply plugin: 'java'
>> apply plugin: 'war'
>>
>> This would publish both the war and the jar:
>>
>> apply plugin: 'java'
>> apply plugin: 'java-library'
>> apply plugin: 'war'
>>
>> This would publish an application distribution and the web application as a
>> war:
>>
>> apply plugin: 'java'
>> apply plugin: 'application'
>> apply plugin: 'war'
>>
>> 4. Don't publish anything and add an opinionated 'publish-main-component'
>> plugin that works like #1 above.
>>
>> I don't really like any of these options, except maybe the 'publish nothing
>> by default' option. Any other suggestions?
>
> I think we can simplify things a lot if we assume that nothing is published
> by default, at the cost of convenience for the simple case of course.
>
> Huge +1 for a 'java-library' plugin and not having the 'java' plugin assume
> that there is a jar to be created (tricky backwards compatibility issue of
> course).
>
> One other option I can think of:
>
> 1. Components have 'publishedByDefault' flag, and there are 'java-library'
> and 'published-java-library' plugins. Or more generally, 'published-«X»'
> plugins that pre wire a component into the project. Plugins like
> 'ivy-publish' automatically try and publish anything that's marked as publish
> by default.
>
>> As far as what should be available via a project dependency, it feels like
>> we should make every component available, assuming components have some kind
>> of identity so that they can be referenced from another project. So, perhaps
>> we would add an implicit local publication for each component.
>
> I don't think this is right. I'm convinced, at this stage, that using a
> fundamentally different mechanism for project dependencies vs. external
> dependencies is flawed. Project dependencies are just inferred dependencies
> on publications of a different project.
My plan is that dependencies (project or otherwise) are on components and
between components. Publications are just a way of serialising a component
across a project boundary. So components are the unifying concept here, not the
publications. We know that external publications and local publications are
going to work differently (or should be able to work differently).
>
> 2 main reasons:
> 1. Makes dynamic composites simpler on the consumer side
I'm not sure what that means. I'm guessing from the stuff below you mean
composites of projects?
> 2. Unifies the behaviour when the dependency is sometimes source and
> sometimes binary
>
> For #1, what I have in my head going forward is this:
>
> 1. Each project forward declares its publications, including their identities
> (think org/module/group)
> 2. Projects forward declare their dependencies in terms of the identity of
> dependencies
> 3. Gradle can detect if a dependency can be satisfied locally, if so it wires
> in a project dependency. If it can't it looks externally for it.
>
> This means that projects are islands that are oblivious to their context WRT
> the multi project environment they are in. You don't need to reconfigure
> projects based on their context. At the moment, to achieve this we have to
> introduce an indirection layer at the client which I think is wrong.
Right. This is all good stuff, but it works regardless of whether we make you
declare things that are locally available or whether we infer it. Components
will have an identity and you'll declare dependencies on components. We'll be
able to substitute a local component for an external one and vice versa.
The problem is slightly different. Some components are never intended to be
published outside the build. For example, when I have a project that produces
both a war and a java library, the war is to be published to the Ivy repository
and the jar is to be used bundled into a war produced by another project. I
don't want the jar to end up in the repository. So the question is how we model
this?
>
> For #2, I want _exactly_ the same semantics when I'm consuming something via
> “project dependency” vs. external dependency… most of the time. There are
> issues with this though…
>
> 1. It appears to restrict our ability to make performance optimisations, such
> as using class files instead of jars for project dependencies.
> 2. More expensive because we have to convert to the publishable model, and
> consume that.
>
> I don't think doing this prevents #1. I think it is still possible.
> I'm also not concerned about #2 as we have to be fast at consuming the
> descriptors/models of dependencies and incremental build compensates on the
> producer side I think.
I don't think you do always want the same semantics. But I think we can make
this the consumer's choice. Each component generally has multiple ways it can
be serialised. A java library can be represented in a bunch of ways: a jar +
meta-data, or a classes dir + meta-data, or a fat jar + meta-data, or an api
jar + impl jar + meta-data, or a native shared library + meta-data, or an
Android dex file + meta-data, or a source jar + meta-data (and probably a bunch
of others). This we've been calling the 'packaging'. So, generally, a producer
will make a component available as one ore more packagings and the client will
have some way to request (or hint) which packaging they'd like. Maybe each
packaging is a separate publications, or maybe they are all bundled into the
same publication. Not sure yet. Probably separate publications.
Our opinionated plugins might take a stance on which packaging to use, so the
'java' plugin might prefer to use classes dir + meta-data over any of the other
available packagings for unit testing, but to use jar + meta-data for
integration testing.
--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com
