On Monday, 12 February 2018 at 10:35:06 UTC, Russel Winder wrote:
In all the discussion of Dub to date, it hasn't been pointed
out that JVM building merged dependency management and build a
long time ago. Historically:
Make → Ant → Maven → Gradle
and Gradle can handle C++ as well as JVM language builds.
So the integration of package management and build as seen in
Go, Cargo, and Dub is not a group of outliers. Could it be then
that it is the right thing to do. After all package management
is a dependency management activity and build is a dependency
management activity, so why separate them, just have a single
ADG to describe the whole thing.
SCons, CMake, and Meson (also Reggae?) are traditional build
tools, but they assume all package dependency management is
handled elsewhere, i.e. they state what is required for a build
but assume some other tool provides those packages, usually OS
package management, but note with C++ Conan is rapidly becoming
a big player. JFrog via Bintray and Artifactory do appear to be
the leaders in repository management and Gradle works bery well
with it.
Rust, Ceylon, and D have to date chosen to eschew systems like
Bintray to create their own language specific versions for
whatever reason. This leads to language specific dependency
management and build. Go is also in this category really except
that Git, Mercurial, and Breezy (née Bazaar) repositories are
the only package storage used.
Then, is a DevOps world, there is deployment, which is usually
a dependency management task. Is a totally new tool doing ADG
manipulation really needed for this?
The lessons from all the tools from SCons to Gradle is that it
is all about ADG manipulation and constraint satisfaction.
SCons is really quite restricted even though it does it very
well (*). Gradle really tries hard not just to solve the
problems of Maven (**), but to do end- to-end project
management well. In a sense it is the antithesis of each tool
does one thing and one thing only model, it is the "there is
one and only one ADG to describe the life of the project".
Maven and Gradle, and to a lesser extent Cargo and Go,
emphasise project management as a wholistic thing, rather than
making people deside on each item of the tool chain. Gradle
proves a good plugin system to allow changes to the default
standard project lifecycle.
Gradle uses Groovy scripts or Kotlin scripts for project
specifications. Most projects are easily described by a purely
declarative specification using internal DSLs. However, for
those awkwards bits for some projects a bit of programming in
the specification solves the problem. So this goes against the
"a project specification must be purely declarative so use
TOML/JSON/SDL" but is easy DevOps worth it.
Atila's Reggae has already shown how easy it is to use D (or
Python, Ruby, JavaScript, Lua) to define a build in an internal
DSL.
Merging ideas from Dub, Gradle, and Reggae, into a project
management tool for D (with C) projects is relatively
straightforward of plan albeit really quite a complicated
project. Creating the core ADG processing is the first
requirement. It has to deal with external dependencies, project
build dependencies, and deployment dependencies. The initial
problem is creating an ADG from what is potentially a CDG and
then doing constraint satisfaction to generate actions. SCons
and Gradle have a lot to offer on this.
Having been obsessed by build and project management since
about 1976, I'd be interested in doing some work on this.
(*) The O(N) vs. O(n), SCons vs. Tup thing that T raised in
another
thread is important, but actually it is an implementation thing
of how
do you detect change, it isn't an algorithmic issue at a system
design
level. But it is important.
(**) Which many people ignore because Maven remains the major
project management tool in the JVM-verse.
Lot's of stuff here. I would love for a build/package management
tool to emerge and take over like git did with source control.
However, my guess is that this problem is very hard and that's
why there's so many tools that have their own pros and cons. But
it sounds like you're trying to sift through them all and find
the gems in each tool.
I myself have made attempts to tackle the problem. I created a
tool called "dbuild" that allowed you to use D code to describe
your project and then build it for you (kinda like Raggae I
suppose). And my latest project I call "bidmake" which is more
general. It implements the model of what I call "build
contracts". A project describes a "contract" and various
"contractors" can be used to fulfill the contract. You could
have a "build contractor" that builds a contract, or a
"documentation contractor" that generates documentation, etc.
Anyway, I currently use the tool successfully on a few projects
but I'm still not sure how much potential it has. Maybe with
your experience you can take a look at it and see if there's any
substance to the idea. (https://github.com/marler8997/bidmake)
