On Tue, Mar 29, 2016 at 4:46 PM, Matteo Bertozzi <theo.berto...@gmail.com> wrote:
> don't we have already a flakey detector? HBASE-8018 > > https://github.com/apache/hbase/commit/4dc52261a19ed1055b837b136c97ea36e362ba6e > > I remember that script. It used to work for me but had forgotten about it. St.Ack > On Tue, Mar 29, 2016 at 4:39 PM, Sean Busbey <bus...@cloudera.com> wrote: > > > How about if we make a script for determining flakey status, based on > > rate of failing in either post-commit or in the pre-existing branch > > check for precommit. The script could live in dev-support and work by > > effectively grabbing the test reports from those jobs. > > > > We can then run that script in a job and save the results as a build > > artifact. > > > > We can then build the changes in your original email: ignoring the > > list entries in our normal builds and rechecking just them in a > > dedicated job. > > > > On Tue, Mar 29, 2016 at 6:30 PM, Sean Busbey <bus...@cloudera.com> > wrote: > > > Where/How do we maintain the list of tests that are flagged as flakes? > > > > > > On Tue, Mar 29, 2016 at 6:25 PM, Apekshit Sharma <a...@cloudera.com> > > wrote: > > >> Proposal: > > >> Maintain a list of flaky tests which can be ignored in main builds > (for > > >> patches) and setup a new job which will periodically run those flaky > > tests. > > >> > > >> Benefits: > > >> - Cleaner main builds. Less runs per patch, ideally no re-runs > because > > of > > >> timeouts and other flakiness. Less frustration. Increased developer > > >> productivity. > > >> - Logs from various runs available when anyone tries to fix these > tests. > > >> > > >> Possible con: We start demoting tests to flaky list which never get > > fixed. > > >> > > >> How? > > >> Internally, we have setup up two jenkins jobs, one for main build and > > one > > >> for flaky. In execute shell step, they curl to get the list of flaky > > tests > > >> and use surefire plugin flags to ignore/run particular tests. > > >> I volunteer to set them up. However, not sure if this approach will > work > > >> since we have yetus upstream. > > >> > > >> - Appy > > > > > > > > > > > > -- > > > busbey > > > > > > > > -- > > busbey > > >
