Great stuff, thanks Michael. Is there a way to indicate, for each of the jobs, how many times the job failed vs the number of times it was run during the time period? I often see that when one test fails it will cause others to fail...
Patrick On Tue, Apr 11, 2017 at 8:07 PM, Edward Ribeiro <[email protected]> wrote: > Very cool feature! Congratulations for initiative! :D > > Em 11 de abr de 2017 11:48 PM, "Jordan Zimmerman" < > [email protected]> escreveu: > > > Nice work - we could really use this on Curator. I'll be stealing this > > soon ;D > > > > > On Apr 11, 2017, at 9:33 PM, Michael Han <[email protected]> wrote: > > > > > > tl;dr > > > https://builds.apache.org/job/ZooKeeper-Find-Flaky-Tests/ > > lastSuccessfulBuild/artifact/report.html > > > > > > > > > We all know quality is important to this project. One of major > advantage > > of > > > ZooKeeper comparing to other similar solutions is its quality: solid, > > > stable, and well tested. Unit tests are playing an important role for > > ZK's > > > quality so we need take these tests seriously. In particular for failed > > > tests, it's easy to blame they are failed because of flaky but usually > > > there are bugs (sometimes very subtle ones) behind the failures. > > > > > > Inspired by some good work done on HBase project, I've borrowed and > > tweaked > > > their script so it applies to ZooKeeper project. The dashboard is now > up > > > and a Jenkins job is running on a daily base to monitor our builds and > > pick > > > up flaky tests. I hope this dashboard can help increase the visibility > > and > > > raise awareness of the quality of unit tests, and also help developers > > > track and diagnose failures. > > > > > > Any feedback will be appreciated, and of course, contributions are very > > > welcome. > > > > > > -- > > > Cheers > > > Michael. > > > > >
