On 10/12/2012, at 9:10 PM, Hans Dockter wrote:
>
> On Nov 23, 2012, at 9:50 PM, Adam Murdoch <[email protected]> wrote:
>
>> Hi,
>>
>> One question that we need to answer to make progress on the publication DSL
>> is how should change our dependency notation, so that we can:
>>
>> 1. Substitute a reference to a locally built thing with something downloaded
>> from a repository [1]. For example, when I partially check out a source
>> tree, I want to build the things for which there is source, and download the
>> things for which there is no source, without changing the build scripts.
>
> To add what we have been talking about for a long time: I want to use a
> global cache to make use of all the artifacts that have been already build
> (by other dev machines or by CI). If an artifacts has been already build for
> the commit id of a particular project I have checked out, I just get the
> artifact from the global cache.
>
>> 2. Substitute a reference to a thing in a repository with something built
>> locally. For example, when I'm working on multiple source trees, I want to
>> build a local version of some things that would otherwise be downloaded,
>> without changing the build scripts.
>>
>> 3. Skip the configuration of projects which are not required to build the
>> local things we need. For example, when I'm working on source tree with many
>> projects, I want Gradle to configure only those projects that are relevant
>> to whatever I'm doing.
>>
>> 4. Distinguish between the various things built by a project. For example, I
>> have a project that builds a web app and a java library, and in some places
>> I want to refer to the java library and in other places I want to refer to
>> the web application.
>>
>> [1] "Downloaded from a repository" really means "fetched from somewhere
>> else". It may or may not be downloaded and it may or may not come from a
>> repository.
>>
>> Just as some background, the basic model we want to aim for is this:
>>
>> 1. Projects produce components.
>> 2. A component is some logical thing that I can use, such as a java library
>> or web application.
>> 3. Dependencies are criteria that select one or more components.
>>
>> There's more detail here:
>> https://github.com/gradle/gradle/blob/master/design-docs/dependency-model.md
>>
>> A few options (not all good options - just brainstorming here):
>>
>> 1. References to locally built things and things in a repository share the
>> existing namespace we use for things in a repository. That is, every
>> dependency declaration would use (group, component, version), including
>> project dependencies:
>>
>> dependencies {
>> compile 'my.org:some-repo-thing:1.2'
>> compile 'my.org:some-other-project:1.4' // A project dependency
>> }
>>
>> The obvious downside here is that version is required for a project
>> dependency, but version is almost always a calculated value. Another
>> downside is that projects already have a convenient identifier in their
>> path, whereas this approach would require a project to define an additional
>> identifier.
>
> This is also about expressiveness. Despite the fact that the difference
> between external libraries and project dependencies is fuzzy from an
> organizational point of view there is often a big difference. So it is nice
> to easily see the from an organization point of view different types of
> dependencies.
Not sure what you mean here.
>
>>
>> 2. References to locally built things and things in a repository share a new
>> namespace. Every dependency declaration would use a single identifier, and
>> there would be a separate mapping, shared by all the projects, from
>> identifier to (group, module, version) or project.
>>
>> The idea here is that we also deal with the 'dependencyManagement' use case
>> as well, so that you define (group, module, version) in one place and use a
>> simple name everywhere that you need to refer to the dependency. This is, in
>> practise what larger builds do already, so it's not necessarily more
>> 'complicated'.
>>
>> allprojects {
>> modules {
>> someRepoThing 'my.org:some-repo-thing:1.2'
>> }
>> }
>>
>> dependencies {
>> compile someRepoThing
>> runtime someLocalThing // Probably some default rule to look for a
>> locally built component called 'someLocalThing'
>> }
>>
>> The obvious downside here is that you need to extra work if the dependency
>> is not reused. Plus it means every dependency declaration in every Gradle
>> build script in existence would need to change.
>>
>> 3. A combination of the above, so that all dependency declarations refer to
>> things using (group, component, version), but group and version are
>> optional. If not specified, we look for a mapping from component name to
>> (group, component, version) or (project, component). We'd have some place
>> where you can register these mapping to things in repositories, and probably
>> also have a default mapping where we look for a locally built component with
>> the given name and same group as the referring project.
>>
>> allprojects {
>> dependencies.modules {
>> junit "junit:junit:4.11"
>> }
>> }
>>
>> dependencies {
>> compile "com.google.guava:guava:13"
>> testCompile "junit" // or perhaps ":junit:" - maps to "junit:junit:4.11"
>> testCompile "junit:4.8.2" // maps to "junit:junit:4.8.2"
>> runtime "some-local-thing" // default rule maps to component
>> 'some-local-thing' built by another project
>> }
>>
>> For me, this one is almost all upsides. It gives me a unified way to refer
>> to things, using (if I like) a simple name, and a place to define meta-data
>> about external things in a single place (such as, every dependency to
>> 'junit' should use 'junit:junit:4.11' unless otherwise specified, or
>> "my-lib" uses semantic versioning, or "groovy-all" conflicts with "groovy",
>> etc). It also gives me a point where I can put logic in to influence whether
>> a given dependency declaration is mapped to a locally built thing or a thing
>> in a repository.
>
> I like this very, very much.
>
>>
>> One major downside is that every project dependency in the world would need
>> to be changed, because project dependencies would go away (via a long period
>> of deprecation).
>
> They wouldn't necessarily need to go away. We should easily be able to map
> them as long as we want.
If they don't go away, then you're really talking about option 4. below. The
point of option 3. is that there is exactly one way to express a dependency and
it's a separate mapping step as to whether that ends up pointing to something
external or something local.
>
>>
>> 4. Keep things as they are, but remove (via deprecation) the ability to
>> navigate to the target project via a ProjectDependency. We would offer a way
>> to ask Gradle to substitute a project dependency with an external dependency
>> and vice versa.
>>
>> Major downside for me is that it doesn't push the move to components far
>> enough.
>>
>> One not-so-obvious advantage of this approach is that it is very easy to
>> implement configure-projects-on-demand, whereas the other approaches would
>> require some kind of caching.
>
> But would't this caching be anyhow part of the incremental evaluation logic,
> e.g. don't even evaluate a project dependency if the output is up to date?
> Which is even better that configure-projects-on-demand.
It depends. They are complementary and we'd eventually want to do both. If you
just do incremental-configuration without configure-on-demand, then you'd still
need to scan all the source files and build scripts and other inputs for the
entire build, which can be expensive. Better to short-circuit this by
eliminating whole projects with configure-on-demand, before making individual
projects faster with incremental-configuration.
The analogy is with tasks: we started by first skipping tasks that were not
required for whatever we're building, and then later by skipping tasks that
were required but up-to-date (and even later, by skipping individual output
files that were required but up-to-date). We'd do the same with project
configuration: first skip projects that are not required, and then later skip
projects that are required but up-to-date (and even later, by skipping
individual domain objects that are required but up-to-date).
--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com
