On Sun, 02 Jun 2013 22:27:12 -0700, Dylan Knutson <tcdknut...@gmail.com>
wrote:
Hello,
I'm a bit confused as to how the DMD buildbot is supposed to work: it
seems like 50% of the time (ballparked from the first 15 or so pull
requests), the buildbot just doesn't report failures or successes. This
bugs (heh) me a little bit because there are tons of months old pull
requests just waiting in the pipeline, stuck at "Determining merge
status". I don't know if this is part of the review process or caused by
the yellow status, but for instance I've opened up a bug report a week
or so prior:
http://d.puremagic.com/issues/show_bug.cgi?id=10113
and a few days afterwards, a pull request was submitted:
https://github.com/D-Programming-Language/dmd/pull/2080
which, turns out, was more or less a dup of another pull request,
submitted 6 months ago:
https://github.com/D-Programming-Language/dmd/pull/1358
I'll see if I can't give you a stab at an explanation. Note that I did not
write the Auto-Tester, that is the work of Brad Anderson. Here are some
facts about the AT as I understand them:
- The AT works on a most current pull first basis, so a pull that is older
than 6 months will naturally be scheduled much later.
- The AT schedules work based on how many testing machines are available,
so each pull is tested only once per listed platform, and when multiple
build servers for a given platform are available, different pulls can be
tested simultaneously on that platform.
- Pull requests are tested by cloning [DMD/DRuntime/Phobos] master and
merging the pull into it. This ensures a clean pull against master.
- The AT must restart pull request testing every time code is merged into
DMD/DRuntime/Phobos because of the above testing strategy.
- If many pulls are merged into [DMD/DRuntime/Phobos] master in a given
period of, for example a day, the AT will restart prior to completing a
full test pass.
- With the current number of open pulls the AT can complete a full testing
pass in roughly 8 hours.
- Each pull takes a rough average of 15 minutes to test.
The lack of success/failure reports on older pulls is directly due to the
above testing strategy, essentially the AT never made it that far before
someone merged a pull into [DMD/DRuntime/Phobos] master and forced a
restart.
However, neither of these pull requests have been merged. And neither of
the bugs have been fixed (my report was a dup of
http://d.puremagic.com/issues/show_bug.cgi?id=2950, submitted 4 years
ago!).
So we're stuck with a buildbot system that leaves valuable pull requests
from ever getting merged, and duplicate fixes being written because
previous fixes were submitted and never accepted so long ago.
The AT system is intentionally incapable of merging to master, so it
itself is not leaving pulls behind. However if a pull is out-of-date it is
likely that it will fail the AT anyways due the codebase delta in the
target project. This is why pulls are tested in a newest first order.
On a somewhat related note: Why is the Puremagic bugtracker still used,
when Github has bug tracking with (IMO) better repo integration,
discussion, and searching (which I think we can all agree on), on a
repository *hosted at Github*? I can understand the reluctance to switch
over, but what's keeping us from accepting new issues on Github, closing
Puremagic down from accepting new requests, and working through the open
requests on Puremagic until all can be taken care of have been?
According to the Bug Stats page (http://dlang.org/bugstats.php) there are
nearly 3000 open issues in the BugZilla and over 7000 closed issues! Now I
tend to agree that better systems than BugZilla exist. But consider the
effort required to move all of those bugs to a new system, automated
tooling or no, it wouldn't be easy or quick. And consider that GitHub
issues lacks much of meta information capability that BugZilla has, we
would loose a fantastic amount of information. Yes of course GitHub has
the best integration with it's own issue tracker, but we've achieved a
fairly high level of integration between GitHub and BugZilla, so that has
become almost a non-issue these days.
Thanks for your time,
Dylan Knutson
--
Adam Wilson
IRC: LightBender
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/
