On 01/05/2012, at 10:30 PM, Steve Ebersole wrote:
> I would just caution about the statement "I think eventually we want to get
> rid of buildSrc altogether" by emphatically seconding your other statement "I
> like the idea behind the buildSrc project.... It would be a shame to lose
> this."
>
> buildSrc is how every one of my plugins grew. In fact, I find this concept
> one of the really flexible things about Gradle plugin development.
You'll still be able to do this. It'll work a little differently:
1. You add a project that will own the build logic.
2. You tell Gradle how to build the project.
3. You tell Gradle that the project is a 'build logic' project, and needs to be
built before the other build scripts are executed.
1) and 2) are no different to any other kind of project in Gradle.
For 2), there will be a couple of conveniences:
* A 'plugin dev' plugin, which sets up the appropriate dependencies and other
things you need to implement reusable build logic.
* A way to inject configuration into all projects, including the build logic
project. This would work in a similar way as allprojects { } does.
For 3), there will be some kind of convenience, but I'm not sure exactly how
that will look yet. It might be some kind of declaration in settings.gradle,
when the project is defined. Let's say for now, there's a global script
classpath that you can mess with in settings.gradle:
gradle {
buildscripts {
dependencies { classpath project(':my-build-logic-project') }
}
}
I'm sure we can make this a little more concise.
Sitting on top of this might be a convention-based plugin, to take care of 1),
2) and 3) for you. You'd do something like this in your settings.gradle:
apply plugin: 'build-logic-project'
This would, say, add a project called 'build' whose project dir is
$rootDir/gradle, and add that project as a dependency for all script class
paths.
Otherwise, things will work the same way as buildSrc: you start off with the
build logic in your build, flesh out the plugin, then later move it to a
separate build so that you can reuse it in other builds.
One thing that using a regular project offers over buildSrc is an intermediate
step before you move the build logic to a separate build: You'll be able to
start publishing the plugin from the build logic project, exactly as if it were
in a separate build, but without having to actually move it. Depending on what
the build logic represents, it may be quite fine that it never moves out.
>
> On Mon 30 Apr 2012 08:06:22 PM CDT, Adam Murdoch wrote:
>> Hi,
>>
>> Something we want to do soon is to replace the buildSrc project with a
>> regular project. There are a few motivations for this:
>>
>> * To improve the user experience for those builds that need dedicated
>> build logic. For example, currently the buildSrc project's 'build'
>> target is used. But this runs all the tests and checks, whereas for
>> 95% of the time, the user is only interested in compiling the classes.
>> Or, currently we need to clean the buildSrc project when the Gradle
>> version changes, whereas for regular projects we don't need to do
>> this. Or, currently the buildSrc project does not end up in the IDE
>> model, but would be included if it were a regular project.
>>
>> * To allow build logic to both be published and used in the same build
>> (but not in the same project, for now). This will mean that you can
>> use your enterprise plugins in the same build that produces them. For
>> example, you can use your custom release plugin to release your custom
>> release plugin. We may use this in Gradle, too, when we add a plugin
>> dev plugin.
>>
>> * To detangle project configuration from the project hierarchy. In
>> particular, this required for parallel execution, so that projects can
>> be configured in an arbitrary order, and across multiple JVMs and/or
>> threads.
>>
>> DSL-wise, there are 3 main use cases:
>> 1. Declare that a given script depends on the build logic from some a
>> project.
>> 2. Declare that every script depends on the build logic from some
>> project. Or there might be a convention for this, so that you give a
>> project a particular name or put it in a particular directory, and it
>> is automatically picked up as a build logic project.
>> 3. Inject configuration to all projects, including those projects that
>> are built during configuration time.
>>
>> Use case 1
>> -------------
>>
>> I think this is as simple as being able to add project dependencies to
>> the build script's classpath configuration:
>>
>> buildscript {
>> dependencies { classpath project(':buildLogic') }
>> }
>>
>> When we simplify the DSL for applying plugins, this might become
>> something like:
>>
>> apply project: ':buildLogic', plugin: 'my-custom-plugin'
>>
>> Implementation-wise, the configuration phase would look something like
>> this:
>>
>> 1. Queue up the configuration of each project, in parent-first order
>> (like we do now).
>> 2. For each project, if not already configured, then execute the
>> project's build script.
>> 3. For each script that is executed:
>> * Execute the buildscript { } section of the build script.
>> * For each project dependency in the build script classpath,
>> recursively configure and build the target project. Fail if the target
>> project is currently being configured.
>> * Resolve the build script classpath and execute the script.
>> * For each call to evaluationDependsOn(), recursively configure the
>> target project. Fail if the target project is currently being configured.
>> 4. For each project that is built during configuration:
>> * Configure the project as above
>> * For each project dependency required to build the project,
>> recursively configure the target project. Fail if the target project
>> is currently being configured.
>> * Add the tasks that build the runtime class path for the project to
>> the DAG.
>> * Execute the tasks.
>>
>> I think this boils down to some changes to dependency resolution:
>>
>> During the configuration of a project:
>> 1. When a Configuration is resolved, for each project dependency we
>> trigger configuration of the target project and building of its artefacts.
>> 2. When a Configuration's buildDependencies are queried, for each
>> project dependency we trigger configuration of the target project.
>>
>> At other times (e.g. task execution):
>> 1. When a Configuration is resolved, for each project dependency
>> assert that the target project has been configured and the artefacts
>> built. It's an error if not.
>> 2. When a Configuration's buildDependencies are queried, for each
>> project dependency assert that the target project has been configured.
>> It's an error if not.
>>
>> And the same kind of thing for task dependencies:
>>
>> * When a task's dependencies are resolved during configuration,
>> trigger the configuration of the target project.
>> * When a task's dependencies are resolved at other times, assert that
>> the target project has been configured.
>>
>>
>> Some open issues:
>>
>> * Currently, the buildSrc classes are available in the settings
>> script. This would not be the case if a regular project is used. Some
>> possible solutions:
>> - Use an external script for any shared logic.
>> - Allow the settings script to add projects in it's settingsscript { }
>> section, and resolve configurations as above.
>> - Move the logic to an external project, and allow plugins to be
>> applied to the Settings object.
>> - Allow build scripts to add projects.
>> - Chop your settings script into 2: one which defines the build logic
>> projects, and a second one that declares a dependency on that project
>> and uses it to define the remaining projects.
>>
>> * Tasks can be executed before the DAG is fully populated, and before
>> the 'DAG ready' event has been fired. This means that some conditional
>> configuration may not have been executed when these tasks are
>> executed. Introducing build types might be an option here, so that the
>> conditional stuff is applied much earlier in the configuration phase.
>>
>> * Projects can be configured and tasks executed before the parent
>> project has had a chance to do configuration injection. More on this
>> below.
>>
>> Use case 2
>> ------------
>>
>> I like the idea behind the buildSrc project: you just put your build
>> logic in a certain place, and it is just made available. It would be a
>> shame to lose this. I wonder, however, if we really need this,
>> assuming we can reduce the boilerplate for adding a project dependency
>> to a build script classpath down to a single statement. We might also
>> tackle this by making script 'plugins' work more like plugins, so that
>> something like:
>>
>> apply plugin: 'my-plugin'
>>
>> might come from a compiled class from another project, or might apply
>> $rootDir/gradle/my-plugin.gradle (or whatever).
>>
>> This way, plugins are provided by the environment and the consuming
>> script doesn't care where they come from. What is currently in
>> buildSrc would turn into one of the following:
>> * A regular project in some external build, with plugins published to
>> a repository.
>> * A regular project in the same build, with plugins built locally.
>> * A script in some conventional (or declared) location.
>>
>>
>> Use case 3
>> ------------
>>
>> The current approach of using allprojects {} and friends for
>> configuration injection isn't going to work, as the build logic
>> project will potentially have been configured and built before the
>> injecting script has a chance to execute.
>>
>> There are a couple of existing approaches that would work (but are a
>> bit awkward):
>> * Move the shared logic to a script, and apply it from various locations
>> * Move the shared logic to a plugin in a second build logic project,
>> and depend on it from various locations.
>>
>> The existing configuration injection methods have some other problems.
>> First, these methods guarantee that the code is called for every
>> project, and that every project is configured. However, this stops us
>> doing some things:
>> * Skip the configuration of projects that aren't relevant to the
>> current build. Eg in the Gradle build, don't configure all the plugin
>> projects if I'm running the unit tests for core.
>> * Short-circuit the configuration of projects whose outputs are up to
>> date. Eg in the Gradle build, when I'm working on the c++ plugin,
>> don't configure all the core projects when none of their source or
>> configuration has changed.
>> * Use compatible pre-built artefacts from a binary repository, rather
>> than configuring the projects and building their artefacts. Eg in the
>> Gradle build, when I'm working on the c++ plugin, just get the rest of
>> the binaries from the CI server (not a great example, but you get the
>> idea).
>>
>> Second, these methods guarantee that the code is always called in the
>> same context. This stops us doing some of these things:
>> * Building separate chunks of the model concurrently.
>> * Building the model across multiple JVMs or machines.
>>
>> So, I think we need a new DSL here. Some options:
>>
>> 1. Just change the injection methods, so that they drop these guarantees.
>> 2. Change the injection methods so that they have 2 modes. Allow a
>> build script to declare which mode it needs.
>> 3. Add new injection methods, with different names to the existing
>> methods.
>> 4. Use scripts in conventional locations. So, perhaps
>> $rootDir/gradle/allprojects.gradle is applied to each project before
>> it is configured.
>> 5. Allow configuration to be injected from the settings script (with
>> the new semantics).
>> 6. Add a new type of build script, with injection methods that have
>> the same names as the existing ones, but with the new semantics.
>>
>> Option 1) is not really an option. Options 2), 3) and 6) don't solve
>> the build logic project problem. Personally, I like 5), because it
>> detangles the build configuration from the root project. What is
>> interesting about this option is that it allows you to have a single
>> .gradle file for an entire multi-project build, that both defines the
>> projects and injects configuration into them.
>>
>> An open issue is exactly what the semantics of the injection methods
>> would be. They're going to have to deal with the fact that the
>> configuration code may end up running in various different JVMs. This
>> has some implications as to how values are shared across projects,
>> e.g. a calculated version.
>>
>>
>> Migration
>> ----------
>>
>> I think eventually we want to get rid of buildSrc altogether. The plan
>> would be to implement the above use cases as experimental features,
>> leaving buildSrc alone. Then, we should shake out the new
>> configuration mechanism further with some of the parallel execution
>> and partial configuration features. Once we're fairly happy with how
>> this looks, we would deprecate the buildSrc project, and later remove it.
>>
>>
>> --
>> Adam Murdoch
>> Gradle Co-founder
>> http://www.gradle.org
>> VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
>> http://www.gradleware.com
>>
>
> --
> [email protected]
> http://hibernate.org
--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com
