don't we have already a flakey detector? HBASE-8018 https://github.com/apache/hbase/commit/4dc52261a19ed1055b837b136c97ea36e362ba6e
On Tue, Mar 29, 2016 at 4:39 PM, Sean Busbey <[email protected]> wrote: > How about if we make a script for determining flakey status, based on > rate of failing in either post-commit or in the pre-existing branch > check for precommit. The script could live in dev-support and work by > effectively grabbing the test reports from those jobs. > > We can then run that script in a job and save the results as a build > artifact. > > We can then build the changes in your original email: ignoring the > list entries in our normal builds and rechecking just them in a > dedicated job. > > On Tue, Mar 29, 2016 at 6:30 PM, Sean Busbey <[email protected]> wrote: > > Where/How do we maintain the list of tests that are flagged as flakes? > > > > On Tue, Mar 29, 2016 at 6:25 PM, Apekshit Sharma <[email protected]> > wrote: > >> Proposal: > >> Maintain a list of flaky tests which can be ignored in main builds (for > >> patches) and setup a new job which will periodically run those flaky > tests. > >> > >> Benefits: > >> - Cleaner main builds. Less runs per patch, ideally no re-runs because > of > >> timeouts and other flakiness. Less frustration. Increased developer > >> productivity. > >> - Logs from various runs available when anyone tries to fix these tests. > >> > >> Possible con: We start demoting tests to flaky list which never get > fixed. > >> > >> How? > >> Internally, we have setup up two jenkins jobs, one for main build and > one > >> for flaky. In execute shell step, they curl to get the list of flaky > tests > >> and use surefire plugin flags to ignore/run particular tests. > >> I volunteer to set them up. However, not sure if this approach will work > >> since we have yetus upstream. > >> > >> - Appy > > > > > > > > -- > > busbey > > > > -- > busbey >
