On Wed, Feb 15, 2017 at 9:11 AM, Tristan Seligmann <mithra...@mithrandi.net> wrote:
> On Wed, 15 Feb 2017 at 15:23 Jean-Paul Calderone < > exar...@twistedmatrix.com> wrote: > >> >> I wonder about the details of how treq's failures went unnoticed. Is >> development sufficiently inactive that no one looked at CI between the >> breakage and the release? Wasn't that a period of several months? Or does >> it have a Twisted trunk@HEAD build that none of the contributors really >> pay attention to? >> > > The latter is the case; there are Twisted trunk builds, but they're > flagged as "failures allowed" to avoid disrupting development with > transient breakage. Unfortunately this means that basically nobody ever > notices when they fail. For example, here's a failing build: > > https://travis-ci.org/twisted/treq/builds/194691849 > > I haven't found a good solution to this problem yet in my own projects; > I'm hoping someone else has some ideas? > Given an arbitrarily powerful CI system, you could do something. Assume two projects, A and B, with a dependency between them (doesn't matter which direction, I think; doesn't even have to be one-way though obviously it should be). When A changes, do a build of B. Use the new A and the newest version of B which had a successful build. If this build succeeds, A and B are probably still working well together. If this build fails, there's a good chance A just introduced a regression. Apply the same logic with A and B reversed. I expect you could implement this on BuildBot. It might be more realistic to do something slightly simpler - like pinning the version of B in the A-changed builds (and vice versa). This would miss regressions relating to changes in B introduced after the pinned version - but maybe that's okay (you could update pinned versions with each release of B or once every few months or something). I think you could implement this on something like travis-ci without *much* difficulty. Just duplicate (some of) the treq (for example) travis-ci configuration in Twisted's travis-ci configuration. It doesn't sound crazy to say "treq 16.12.0 test suite should be required to pass for Twisted trunk@HEAD (at least until around Twisted 17.12, probably?). And of course there's bound to be noise out of this system, too. I'd guess it would be little enough that the result is still valuable but hard to say for sure without trying... Jean-Paul
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python