Here are a couple of interesting cases we'll need to consider in whatever plan we come up with for dependency notations.
1. BuildSrc as a regular project. The plan to get rid of BuildSrc was to allow project dependencies in the build script classpath, and Gradle take care of building those projects before running the build scripts that need them. If we decide to use the same namespace for project and external dependencies, then we'll have to have some way to know at script compilation time which build script dependencies are local and which are not. Which means running the scripts to find out what each project produces. I think the caching approach still works here. We'd need to defer the configuration of a given project until we know which of the script dependencies are local and which are not, or the project has no build script dependencies. Things might get tricky when the build time project build scripts have external dependencies (not the build time projects, just their build scripts). Maybe we'd need some guidance from the settings.gradle in this instance or perhaps just not support this case. Or maybe we change the plan. 2. Substituting a partial dependency graph Say we have the following dependencies: lib-A -> lib-B -> lib-C Let's say I want to build lib-A and lib-C locally, but download lib-B (for example, let's say 'lib-B' is actually 150 separate things and I don't really want to build them, because I'm not changing the API of lib-C). If we decide to use the same namespace for project and external dependencies, then in order to decide which projects to configure, and which things to build, we effectively have to do a full resolve of the dependencies of lib-A. Which is fine - We need to do this at some point in the build anyway. The issue here is more around when and how this happens. One option is to do this on demand, so that as we execute tasks and resolve their inputs, we discover the projects that need to be configured and built. The downside here is that we can no longer assemble the full task graph before we start executing tasks. In fact, we can't tell you what the full task graph is until the end of the build. Another option is to do the resolve when assembling the task graph, so that we resolve the task inputs enough so that we can figure out which projects and tasks are required to build the inputs of a given task. The downside here is that you won't be able to change these dependencies after the task graph has been built. But this is the case now, so I think this is fine. Either way, we're going to have to be able to run some tasks as we're building the task graph, for the 'BuildSrc as regular projects' case. On 24/11/2012, at 7:50 AM, Adam Murdoch wrote: > Hi, > > One question that we need to answer to make progress on the publication DSL > is how should change our dependency notation, so that we can: > > 1. Substitute a reference to a locally built thing with something downloaded > from a repository [1]. For example, when I partially check out a source tree, > I want to build the things for which there is source, and download the things > for which there is no source, without changing the build scripts. > > 2. Substitute a reference to a thing in a repository with something built > locally. For example, when I'm working on multiple source trees, I want to > build a local version of some things that would otherwise be downloaded, > without changing the build scripts. > > 3. Skip the configuration of projects which are not required to build the > local things we need. For example, when I'm working on source tree with many > projects, I want Gradle to configure only those projects that are relevant to > whatever I'm doing. > > 4. Distinguish between the various things built by a project. For example, I > have a project that builds a web app and a java library, and in some places I > want to refer to the java library and in other places I want to refer to the > web application. > > [1] "Downloaded from a repository" really means "fetched from somewhere > else". It may or may not be downloaded and it may or may not come from a > repository. > > Just as some background, the basic model we want to aim for is this: > > 1. Projects produce components. > 2. A component is some logical thing that I can use, such as a java library > or web application. > 3. Dependencies are criteria that select one or more components. > > There's more detail here: > https://github.com/gradle/gradle/blob/master/design-docs/dependency-model.md > > A few options (not all good options - just brainstorming here): > > 1. References to locally built things and things in a repository share the > existing namespace we use for things in a repository. That is, every > dependency declaration would use (group, component, version), including > project dependencies: > > dependencies { > compile 'my.org:some-repo-thing:1.2' > compile 'my.org:some-other-project:1.4' // A project dependency > } > > The obvious downside here is that version is required for a project > dependency, but version is almost always a calculated value. Another downside > is that projects already have a convenient identifier in their path, whereas > this approach would require a project to define an additional identifier. > > 2. References to locally built things and things in a repository share a new > namespace. Every dependency declaration would use a single identifier, and > there would be a separate mapping, shared by all the projects, from > identifier to (group, module, version) or project. > > The idea here is that we also deal with the 'dependencyManagement' use case > as well, so that you define (group, module, version) in one place and use a > simple name everywhere that you need to refer to the dependency. This is, in > practise what larger builds do already, so it's not necessarily more > 'complicated'. > > allprojects { > modules { > someRepoThing 'my.org:some-repo-thing:1.2' > } > } > > dependencies { > compile someRepoThing > runtime someLocalThing // Probably some default rule to look for a > locally built component called 'someLocalThing' > } > > The obvious downside here is that you need to extra work if the dependency is > not reused. Plus it means every dependency declaration in every Gradle build > script in existence would need to change. > > 3. A combination of the above, so that all dependency declarations refer to > things using (group, component, version), but group and version are optional. > If not specified, we look for a mapping from component name to (group, > component, version) or (project, component). We'd have some place where you > can register these mapping to things in repositories, and probably also have > a default mapping where we look for a locally built component with the given > name and same group as the referring project. > > allprojects { > dependencies.modules { > junit "junit:junit:4.11" > } > } > > dependencies { > compile "com.google.guava:guava:13" > testCompile "junit" // or perhaps ":junit:" - maps to "junit:junit:4.11" > testCompile "junit:4.8.2" // maps to "junit:junit:4.8.2" > runtime "some-local-thing" // default rule maps to component > 'some-local-thing' built by another project > } > > For me, this one is almost all upsides. It gives me a unified way to refer to > things, using (if I like) a simple name, and a place to define meta-data > about external things in a single place (such as, every dependency to 'junit' > should use 'junit:junit:4.11' unless otherwise specified, or "my-lib" uses > semantic versioning, or "groovy-all" conflicts with "groovy", etc). It also > gives me a point where I can put logic in to influence whether a given > dependency declaration is mapped to a locally built thing or a thing in a > repository. > > One major downside is that every project dependency in the world would need > to be changed, because project dependencies would go away (via a long period > of deprecation). > > 4. Keep things as they are, but remove (via deprecation) the ability to > navigate to the target project via a ProjectDependency. We would offer a way > to ask Gradle to substitute a project dependency with an external dependency > and vice versa. > > Major downside for me is that it doesn't push the move to components far > enough. > > One not-so-obvious advantage of this approach is that it is very easy to > implement configure-projects-on-demand, whereas the other approaches would > require some kind of caching. > > > Thoughts? Other options? > > > -- > Adam Murdoch > Gradle Co-founder > http://www.gradle.org > VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting > http://www.gradleware.com > -- Adam Murdoch Gradle Co-founder http://www.gradle.org VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting http://www.gradleware.com
