Hi all,
As you may know, we (Octobus) and other Mercurial contributors have been
using
the Heptapod [1] CI to vet our patches pre-submission against 7 variants of
Mercurial:
- Python 2 pure
- Python 3 pure
- Python 2 python+c
- Python 3 python+c
- Python 2 rust+c
- Python 3 rust+c
- Python 2 chg
- (also code checks and Rust unit tests, of course)
All variants above are run on x86_64 Debian machines. Our CI does not
(yet) have
the MacOS, Windows, or FreeBSDoptionsthat the Mercurial Buildbot has, but
does test more variants on Linux.
Nevertheless, it catches a lot of the mistakes that we all make sometimes,
before we even send the patches upstream: regressions in Python 3, API
breakage
with Rust, bad formatting, and other bugs of various nature. It does so way
before it touches the `committed` repo, which means that if I screw up my
patch, it only has to bother me.
The value of such a system is highly dependent on it being trustworthy:
flaky
tests do exist, but they should be addressed and fixed in a timely
manner and
my CI being red should mean that *I* missed something. Yet, I have heard the
phrase "I don't even look at the pipelines anymore, they're always
broken for
something I haven't done".
Most of the time it's something minor: I've even been guilty of breaking
the CI
myself in my last patch series (oops) due to bad formatting! However, these
issues should be few and far between: the developer experience suffers
greatly
and we grow to accept that the tests are always broken, don't bother to run
them at all, etc.
# Proposal
I thought a start to fixing this situation would be for me to setup
an automation to send a small email to this mailing-list every time
either the
`stable` or `default` branches (that track `hg-committed`) are broken,
with a link
to the pipeline results.
This will not catch everything (only a subset of the possible targets is
tested), but
I think this would be an improvement for all contributors, even those
that don't
use Heptapod.
What do you all think?
Raphaël
[1] https://foss.heptapod.net/octobus/mercurial-devel/-/pipelines
_______________________________________________
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
