On 9/28/20 8:12 PM, Augie Fackler wrote:
On Sep 28, 2020, at 10:43, Raphaël Gomès <raphael.go...@octobus.net
<mailto:raphael.go...@octobus.net>> wrote:
Hi all,
As you may know, we (Octobus) and other Mercurial contributors have
been using
the Heptapod [1] CI to vet our patches pre-submission against 7
variants of
Mercurial:
- Python 2 pure
- Python 3 pure
- Python 2 python+c
- Python 3 python+c
- Python 2 rust+c
- Python 3 rust+c
- Python 2 chg
- (also code checks and Rust unit tests, of course)
All variants above are run on x86_64 Debian machines. Our CI does not
(yet) have
the MacOS, Windows, or FreeBSDoptionsthat the Mercurial Buildbot has, but
does test more variants on Linux.
Nevertheless, it catches a lot of the mistakes that we all make
sometimes,
before we even send the patches upstream: regressions in Python 3,
API breakage
with Rust, bad formatting, and other bugs of various nature. It does
so way
before it touches the `committed` repo, which means that if I screw up my
patch, it only has to bother me.
The value of such a system is highly dependent on it being
trustworthy: flaky
tests do exist, but they should be addressed and fixed in a timely
manner and
my CI being red should mean that *I* missed something. Yet, I have
heard the
phrase "I don't even look at the pipelines anymore, they're always
broken for
something I haven't done".
Most of the time it's something minor: I've even been guilty of
breaking the CI
myself in my last patch series (oops) due to bad formatting! However,
these
issues should be few and far between: the developer experience
suffers greatly
and we grow to accept that the tests are always broken, don't bother
to run
them at all, etc.
# Proposal
I thought a start to fixing this situation would be for me to setup
an automation to send a small email to this mailing-list every time
either the
`stable` or `default` branches (that track `hg-committed`) are
broken, with a link
to the pipeline results.
This will not catch everything (only a subset of the possible targets
is tested), but
I think this would be an improvement for all contributors, even those
that don't
use Heptapod.
What do you all think?
I'm open to it. Do you think the CI that you're running on heptapod is
faster than the buildbot? Should we keep the buildbot or shut it down?
Faster in terms of latency, almost certainly, at least for now because
of the number of workers, and probably in terms of the actual speed of
the runners, though I'm just making a guess.
However, the Buildbot still has value since it covers cases that our CI
still does not cover and not everyone is using Heptapod, so let's keep
it around for a while. To further drive the point home, I'm not
comfortable recommending Heptapod for the project or anything of that
sort yet, even ruling out personal preferences for workflow. Maybe we
can have that discussion in a few months, who knows.
The fact that the Buildbot runs *after* the patches have landed makes me
sad, but that's an issue for another day probably!
Raphaël
[1] https://foss.heptapod.net/octobus/mercurial-devel/-/pipelines
_______________________________________________
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
<mailto:Mercurial-devel@mercurial-scm.org>
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
_______________________________________________
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
_______________________________________________
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
