On 10/12/2012, at 9:17 PM, Hans Dockter wrote: > > On Dec 7, 2012, at 2:10 AM, Adam Murdoch <[email protected]> wrote: > >> >> Here are a couple of interesting cases we'll need to consider in whatever >> plan we come up with for dependency notations. >> >> 1. BuildSrc as a regular project. >> >> The plan to get rid of BuildSrc was to allow project dependencies in the >> build script classpath, and Gradle take care of building those projects >> before running the build scripts that need them. > > Just to be clear. We don't want to get rid of buildSrc but no longer make it > a special case and make the concept much more flexible.
Absolutely. The basic problem is this: you can use the output of a buildSrc project at build time, but cannot use it outside the build, and you can use the output of a regular project outside the build, but cannot use it at build time. We'd change things so that the output from any project can be used at build time or outside the build or both (or neither, I guess). > > Hans > >> >> If we decide to use the same namespace for project and external >> dependencies, then we'll have to have some way to know at script compilation >> time which build script dependencies are local and which are not. Which >> means running the scripts to find out what each project produces. >> >> I think the caching approach still works here. We'd need to defer the >> configuration of a given project until we know which of the script >> dependencies are local and which are not, or the project has no build script >> dependencies. Things might get tricky when the build time project build >> scripts have external dependencies (not the build time projects, just their >> build scripts). Maybe we'd need some guidance from the settings.gradle in >> this instance or perhaps just not support this case. >> >> Or maybe we change the plan. >> >> 2. Substituting a partial dependency graph >> >> Say we have the following dependencies: >> >> lib-A -> lib-B -> lib-C >> >> Let's say I want to build lib-A and lib-C locally, but download lib-B (for >> example, let's say 'lib-B' is actually 150 separate things and I don't >> really want to build them, because I'm not changing the API of lib-C). >> >> If we decide to use the same namespace for project and external >> dependencies, then in order to decide which projects to configure, and which >> things to build, we effectively have to do a full resolve of the >> dependencies of lib-A. Which is fine - We need to do this at some point in >> the build anyway. The issue here is more around when and how this happens. >> >> One option is to do this on demand, so that as we execute tasks and resolve >> their inputs, we discover the projects that need to be configured and built. >> The downside here is that we can no longer assemble the full task graph >> before we start executing tasks. In fact, we can't tell you what the full >> task graph is until the end of the build. >> >> Another option is to do the resolve when assembling the task graph, so that >> we resolve the task inputs enough so that we can figure out which projects >> and tasks are required to build the inputs of a given task. The downside >> here is that you won't be able to change these dependencies after the task >> graph has been built. But this is the case now, so I think this is fine. >> >> Either way, we're going to have to be able to run some tasks as we're >> building the task graph, for the 'BuildSrc as regular projects' case. >> >> >> On 24/11/2012, at 7:50 AM, Adam Murdoch wrote: >> >>> Hi, >>> >>> One question that we need to answer to make progress on the publication DSL >>> is how should change our dependency notation, so that we can: >>> >>> 1. Substitute a reference to a locally built thing with something >>> downloaded from a repository [1]. For example, when I partially check out a >>> source tree, I want to build the things for which there is source, and >>> download the things for which there is no source, without changing the >>> build scripts. >>> >>> 2. Substitute a reference to a thing in a repository with something built >>> locally. For example, when I'm working on multiple source trees, I want to >>> build a local version of some things that would otherwise be downloaded, >>> without changing the build scripts. >>> >>> 3. Skip the configuration of projects which are not required to build the >>> local things we need. For example, when I'm working on source tree with >>> many projects, I want Gradle to configure only those projects that are >>> relevant to whatever I'm doing. >>> >>> 4. Distinguish between the various things built by a project. For example, >>> I have a project that builds a web app and a java library, and in some >>> places I want to refer to the java library and in other places I want to >>> refer to the web application. >>> >>> [1] "Downloaded from a repository" really means "fetched from somewhere >>> else". It may or may not be downloaded and it may or may not come from a >>> repository. >>> >>> Just as some background, the basic model we want to aim for is this: >>> >>> 1. Projects produce components. >>> 2. A component is some logical thing that I can use, such as a java library >>> or web application. >>> 3. Dependencies are criteria that select one or more components. >>> >>> There's more detail here: >>> https://github.com/gradle/gradle/blob/master/design-docs/dependency-model.md >>> >>> A few options (not all good options - just brainstorming here): >>> >>> 1. References to locally built things and things in a repository share the >>> existing namespace we use for things in a repository. That is, every >>> dependency declaration would use (group, component, version), including >>> project dependencies: >>> >>> dependencies { >>> compile 'my.org:some-repo-thing:1.2' >>> compile 'my.org:some-other-project:1.4' // A project dependency >>> } >>> >>> The obvious downside here is that version is required for a project >>> dependency, but version is almost always a calculated value. Another >>> downside is that projects already have a convenient identifier in their >>> path, whereas this approach would require a project to define an additional >>> identifier. >>> >>> 2. References to locally built things and things in a repository share a >>> new namespace. Every dependency declaration would use a single identifier, >>> and there would be a separate mapping, shared by all the projects, from >>> identifier to (group, module, version) or project. >>> >>> The idea here is that we also deal with the 'dependencyManagement' use case >>> as well, so that you define (group, module, version) in one place and use a >>> simple name everywhere that you need to refer to the dependency. This is, >>> in practise what larger builds do already, so it's not necessarily more >>> 'complicated'. >>> >>> allprojects { >>> modules { >>> someRepoThing 'my.org:some-repo-thing:1.2' >>> } >>> } >>> >>> dependencies { >>> compile someRepoThing >>> runtime someLocalThing // Probably some default rule to look for a >>> locally built component called 'someLocalThing' >>> } >>> >>> The obvious downside here is that you need to extra work if the dependency >>> is not reused. Plus it means every dependency declaration in every Gradle >>> build script in existence would need to change. >>> >>> 3. A combination of the above, so that all dependency declarations refer to >>> things using (group, component, version), but group and version are >>> optional. If not specified, we look for a mapping from component name to >>> (group, component, version) or (project, component). We'd have some place >>> where you can register these mapping to things in repositories, and >>> probably also have a default mapping where we look for a locally built >>> component with the given name and same group as the referring project. >>> >>> allprojects { >>> dependencies.modules { >>> junit "junit:junit:4.11" >>> } >>> } >>> >>> dependencies { >>> compile "com.google.guava:guava:13" >>> testCompile "junit" // or perhaps ":junit:" - maps to "junit:junit:4.11" >>> testCompile "junit:4.8.2" // maps to "junit:junit:4.8.2" >>> runtime "some-local-thing" // default rule maps to component >>> 'some-local-thing' built by another project >>> } >>> >>> For me, this one is almost all upsides. It gives me a unified way to refer >>> to things, using (if I like) a simple name, and a place to define meta-data >>> about external things in a single place (such as, every dependency to >>> 'junit' should use 'junit:junit:4.11' unless otherwise specified, or >>> "my-lib" uses semantic versioning, or "groovy-all" conflicts with "groovy", >>> etc). It also gives me a point where I can put logic in to influence >>> whether a given dependency declaration is mapped to a locally built thing >>> or a thing in a repository. >>> >>> One major downside is that every project dependency in the world would need >>> to be changed, because project dependencies would go away (via a long >>> period of deprecation). >>> >>> 4. Keep things as they are, but remove (via deprecation) the ability to >>> navigate to the target project via a ProjectDependency. We would offer a >>> way to ask Gradle to substitute a project dependency with an external >>> dependency and vice versa. >>> >>> Major downside for me is that it doesn't push the move to components far >>> enough. >>> >>> One not-so-obvious advantage of this approach is that it is very easy to >>> implement configure-projects-on-demand, whereas the other approaches would >>> require some kind of caching. >>> >>> >>> Thoughts? Other options? >>> >>> >>> -- >>> Adam Murdoch >>> Gradle Co-founder >>> http://www.gradle.org >>> VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting >>> http://www.gradleware.com >>> >> >> >> -- >> Adam Murdoch >> Gradle Co-founder >> http://www.gradle.org >> VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting >> http://www.gradleware.com >> > -- Adam Murdoch Gradle Co-founder http://www.gradle.org VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting http://www.gradleware.com
