On 1/13/15 3:55 PM, H. S. Teoh via Digitalmars-d wrote:
On Tue, Jan 13, 2015 at 03:20:12PM -0800, Andrei Alexandrescu via Digitalmars-d
wrote:
Hey folks,
Over the time a number of stuff has become quite duplicated across our
makefiles for dmd, druntime, and phobos.
These include fetching OS and model but (newer) general-purpose macros for
e.g. rebasing repos, see
https://github.com/D-Programming-Language/dlang.org/blob/master/posix.mak#L37.
I think it's time to reduce clutter and duplication by migrating such
common stuff into one common.mak file. Question is, where should that
file sit? One candidate is tools/ but it's not really a tool. Another
candidate is dmd/src/ because that would be the first thing anyone
depends on.
Ideas?
[...]
Actually, my first reaction is to go the opposite direction. Currently,
we have way too many complicated interdependencies between different
repos. For example, the 'html' target of phobos/posix.mak depends on
files in the dlang.org repos, and the website build scripts of dlang.org
in turn depends on files generated by the Phobos ddoc targets.
Yah but when you look closer it's not that bad; just an attribution
tradeoff. Building the phobos docs belongs with the phobos makefile
because that's where the module informations resides; however, part of
the build requires macro definitions that are at best consolidated in
the website directory.
Building phobos's docs without dlang.org around is not sensible, just
the same as building dlang.org without phobos. So I'm not too worried
about that.
If someone builds phobos without caring about the docs, that's fine as
long as they don't try to build html.
One simplification would be to not set STDDOC in
https://github.com/D-Programming-Language/phobos/blob/master/posix.mak#L95,
and refuse to build html if STDDOC is not set from the outside. But
again I see this as just minor progress.
This may
be fine for the core devs and hard-core bleeding-edge people like myself
who don't mind running off git HEAD and dealing with breakages, but it's
bad for anybody else:
1) These interdependencies are currently expressed by hard-coded
filesystem paths, which presumes a specific directory layout for
checking out dmd, dlang.org, phobos, druntime. This means the build will
break if somebody doesn't know the correct directory structure to use
when checking out these repos.
Yah, that should be in a wiki. I keep on thinking about producing a
10-minutes video "Your First Contribution to D" in which I explain step
by step how to get a pull request in starting from a scratch.
We also have tools/update.sh which should probably be more prominently
advertised.
But in the grand scheme of things I don't see a foolproof solution to
that - you need dmd for building druntime, druntime for building phobos,
etc. If you got dmd, druntime, and phobos in the wrong places then
You're Gonna Have A Bad Time.
2) This makes the build fragile, because any changes to makefiles that
might affect these dependencies have to be merged in sync between
multiple repos, e.g., a PR that changes the layout of dlang.org needs to
have a corresponding PR that updates the Phobos makefiles as well. This
is kinda what we already have to do with compiler-dependent changes to
druntime/phobos anyway, but in the case of documentation, any oversight
is not caught by the autotester and may wind up on the public-facing
dlang.org website before the problem is noticed.
I don't understand how this is a problem, how my suggestion makes it any
worse, what a solution to it all would be.
3) Due to these interdependencies, you always have to checkout all 4
repos (in a specific directory layout), otherwise the build will fail
one way or another.
This is expected, no? What is an alternative? How would one e.g. build
druntime without dmd?
Again, not a problem for core devs, but a gratuitous
obstacle for the casual user who wants to try things out. (Actually, to
this day I'm almost certain I *still* don't have the exact layout
assumed by the build scripts, because dlang.org appears to deposit
generated html files in dlang.org/web, whereas phobos/posix.mak's 'html'
target generates them in ../web/, which in my setup is one directory
level off. I hacked around the problem with a symlink from web to
dlang.org/web, but I'm pretty sure many users who wanted to experiment
with Phobos docs gave up because they didn't have the patience to figure
out how to get the thing to work properly.)
That's an inconsistency we should fix. Right now phobos is best build
starting from the dlang.org build. We should probably disable phobos'
own docs build.
Because of this, my inclination right now is to *reduce*
interdependencies between these repos, rather than introduce yet more.
Either that, or we should just forget about trying to manually keep 3-4
disparate repos in sync, and just use a single repo for dmd, druntime,
phobos, dlang.org, etc.. Except that some people will then complain that
they just want the bare compiler and nothing else, so why should they
have to clone excess baggage like druntime/phobos. *shrug* Some days you
just can't win.
Yah, that's why I think a set of simple tactical tools a la
tools/update.sh are the proportional response here.
(Of course, the *ideal* solution is that the makefiles will tell you if
something is missing and where to get it and how to set it up correctly.
But I'm skeptical whether this will ever materialize. I'm not holding
my breath.)
dlang.org does something of that kind already - "make rebase".
Andrei
