On Wednesday 28 January 2015 12:27:06 Jan Kundrát wrote: > On Wednesday, 28 January 2015 10:08:54 CEST, Ben Cooksley wrote: > > 11) We actually do use some of Jenkins advanced features, and it > > offers quite a lot more than just a visual view of the last failure. > > > > As a quick overview: > > a) Tracked history for tests (you can determine if a single test > > > > is flaky and view a graph of it's pass/fail history). > > Please see section 3.3.2 which discusses possible ways on how to deal with > flaky tests. IMHO, the key feature and our ultimate goal is "let's handle > flaky tests efficiently", not "let's have a graph of failing tests" (how > would that work with a non-linear history of pre-merge CI?). > > > b) Log parsing to track the history of compiler warnings and other > > > > matters of significance (this is fully configurable based on regexes) > > That's in section 3.3.3. One solution for using this is making the build > warning-free on one well-known platform, and enforcing -Werror in there.
I realise that the CI stuff is in some ways a side-issue, but I should point out that what these features really offer is *metrics* and, to a lesser extent, *motivation*. The motivation aspect is simple: it's nice to see "bad" numbers going down and "good" numbers going up, and for some people (myself included), that's a good motivator. And motivation is important in open source projects. The metrics aspect is a bit more nebulous, but it can be useful to see this information when making decisions - if a lot of tests have just recently started failing, that's a different situation than if they've been failing for a long time. Alex
