On 26/12/2012, at 8:57 PM, Jay Berkenbilt wrote: > > On 12/19/2012 04:24 PM, Adam Murdoch wrote: >> >> On 18/12/2012, at 9:47 PM, Jay Berkenbilt wrote: >> >>> >>> The idea of code generation should be pretty familiar to people who are >>> used to gradle or maven since it is a common thing to have to do in Java >>> builds. Abuild supports code generation for C++ builds with support for >>> two important cases: >>> >>> * The code generator is built by an earlier part of the build >>> >>> * The code generator itself may not be available on all platforms or >>> environment, but the generated code is portable >>> >>> The first case of the code generator being built by an earlier part of >>> the build is important because it means that you can't have a naive >>> approach to the build like saying that all sources are compiled to >>> object code before any executables are built. Hopefully this is pretty >>> obvious anyway. In abuild, the build is divided into "build items" >>> which are roughly equivalent to projects in gradle if my understanding >>> is correct. At the build item level, dependencies are explicitly >>> specified to other build items. Within the build item, dependencies can >>> be specified at a more granular level. (File-level dependencies like >>> the dependency of an object file on a header file are automatic work >>> fine across build item boundaries, but that's another topic.) So with >>> abuild, you can have build item A depend on build item B and have one of >>> B's artifacts be a code generator. B's build creates the code generator >>> (if needed -- maybe it's a script) and adds rules to the build >>> environment for using that generator. When A is built, by virtue of the >>> fact that A depends on B, those rules are available. Let's say A builds >>> some "y" files out of "x" files and B provides rules to generate "y" >>> files from "x" files. All A has to do is depend on B and list the y >>> files in its list of targets. Abuild would fully build B before >>> starting to build A (because of the A -> B dependency), so that when A >>> is ready to be built, the code generator is in place and abuild will >>> know automatically how to build y files from x files. >> >> Generally, Gradle deals with this in a similar way - you declare a >> dependency on the code generator and by the time the task that needs >> to use the generator is executed, the generator has been built or >> downloaded. In practice, however, it's not quite as simple as that: >> >> The approach doesn't work well if the code generator is packaged as a >> Gradle plugin or Gradle task, because the projects are configured and >> the task graph is assembled before any tasks are executed, and this >> means that all the plugin or task implementations must be available >> before any tasks are executed. Which means they cannot be built as >> part of the current build. You can use the buildSrc project, but then >> you can't publish the code generator for use outside the build, plus >> buildSrc has some other issues (IDE integration, etc). > > Yeah, I ran into problems like this too. Abuild has support for > plugins, which are basically nothing more than make rules or the > equivalent with abuild's groovy-based Java support, but plugins get > resolved too early in the build process for them to have their own > dependencies, which makes it hard (not impossible, but you have to jump > through hoops) to have plugins use anything that's automatically > generated. My feeling is that strictly defined build phases will > probably always lead to this problem, which is basically what you are > saying below. > >> >> We do plan to change the configuration phase of Gradle so that we can >> build the things that are required at configuration time on demand. A >> project would declare a configuration time dependency on a code >> generator produced in another project, and Gradle would take care of >> configuring the code generator project and building the code generator >> before configuring the consuming project. Right now, we're working on >> some experimental support for configuring projects on demand, so that >> projects are configured only as we discover that its outputs will be >> needed by the build, rather than configuring every project in every >> build. This should be available in Gradle 1.4. We can then extend this >> to also build the things that project configuration need on demand. > > As I was suggesting, I suspect that ultimately this needs to be > "recursive" for lack of a better term...in other words, the part of the > build that is restrictive because it's bootstrapping the build has to be > kept to a minimum. With abuild, dividing the build into discrete build > items such that each item gets completely built before anything that > depends on it even starts getting built (which seems similar to what > you're saying about on-demand configuration)
Pretty much, but it's a little different (or at least could be). There's 3 things we need to do to build stuff: 1. Figure out what to build. In Gradle, this means configuring the model. 2. Figure out how to build it. In Gradle, this means creating and configuring tasks based on this model. 3. Actually build the things. In Gradle, this means executing the tasks. At the moment in Gradle, steps 1 and 2 are bound together, and are run for all projects before step 3 is started for any project, and does not consider dependencies. Step 3 does consider dependencies, and only does the work that needs to be done to build the requested things. Each of these steps can have various inputs, such as the plugins and tasks that implement the step, or the dependencies of the thing being built. We want to detangle the steps so that we do only the minimum amount of work, and so that the steps for things can be reordered and interleaved based on their dependencies. This means being able to: * Skip configuring the model or creating tasks for things that won't be included in the build. * Skip creating tasks for things that are included in the build but that we don't actually need to build - for example when we're importing the model into an IDE, or if we're going to download a thing instead of building it. * When step 1 or 2 for thing A requires thing B as input, run steps 1, 2 and 3 for thing B first. * Allow the steps for thing A to run in parallel to the steps for thing B, synchronising only when there are dependencies. We could implement this by simply executing steps 1 - 3 as soon as a thing is referenced, but I think we want a bit more flexibility than that. In Gradle, we're looking at using project as the 'thing' for the above stuff. This feels like a reasonable granularity and works pretty well from a backwards-compatibility point of view. -- Adam Murdoch Gradle Co-founder http://www.gradle.org VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting http://www.gradleware.com
