On Wed, Feb 21, 2018 at 11:03:13AM +0000, Russel Winder wrote: [...] > I have been (still am?) a SCons fan but it has some serious problems > in some workflows. > > I think all of the things you mention are solved in a system that has > a general purpose programming language to describe the project and the > build. There is clearly a tension between specifying a project in a > purely declarative way (cf. Cargo, Dub, Maven) where all build and > deploy activities are effectively hardwired (though Maven has plugins > to amend things) versus systems that use a programming language. [...]
I think the ideal situation straddles the divide between declarative build specs and a full-fledged general programming language. You don't want it to get too general, lest you end up with the build equivalent of spaghetti code where the build script becomes unreadable and unmaintainable. OTOH a purely declarative approach is limited by how well the DSL is designed. An insufficiently-expressive declarative language leads to much frustration when you find yourself unable to express something that you need to do with your build. Ultimately, at the bottom level, we're just dealing with DAGs, so at some level the build spec should simply be a bunch of node specs, where each node consists of some set of inputs, some set of outputs, and a computational black box (the build action) to get from the former to the latter. This black box can be anything at all, and should not be unnecessarily constrained. On a higher level, though, specifying individual nodes may not always be desirable (e.g., you want to be able to express things like "all .d files in this directory", or "all recursive import dependencies of source file F"). Typically you'd want a slightly higher-level language for generating DAG node specs. Here is where some serious thought needs to be put into designing a language that's both straightforward for common tasks, yet expressive enough to handle unusual tasks. To abuse a metaphor, DAG node specs are the assembly language of build systems, and generally you want to "program" your builds in a higher-level language, but you could either end up with the build equivalent of C with the build analogues of unchecked array bounds, uncontrolled pointer arithmetic, and hundreds of gotchas completely non-obvious to the uninitiated, resulting in ugly, unmaintainable build scripts, or the build equivalent of D with its expressive power yet packaged in a clean syntax, resulting in clean build scripts that are comfortable to write and maintain. So far I've not yet decided which approach is better: using a full-fledged programming language as the high-level API, like SCons does, or a more constrained but more easily controlled (and implemented!) declarative DSL that is not necessarily Turing-complete. Either way, I prefer baking in as little special behaviour as possible, in the sense that any special capabilities like build-in source code dependency scanning or linker flag control ought to be built on top of primitives that are exposed to user build scripts, so that at least in theory, user build scripts could also attain to equivalent capability without needing to hack the build tool. Generally, I dislike any disparity between what built-in constructs can do "magically" vs. what user code (build scripts) can achieve, because that usually means that things will work well as long as you remain within the parameters the author has conceived, but should you ever encounter a situation the author didn't anticipate, you're stuck up the creek without a paddle. T -- Philosophy: how to make a career out of daydreaming.
