Hi, I think it's a good idea and +1 to run the job during the night. I hope it won't be too long/difficult to take care of this during an XS release though :).
Thanks, Adel On Wed, Oct 3, 2018 at 10:56 AM Vincent Massol <[email protected]> wrote: > > Hi devs, > > We started discussing a first global test coverage strategy in > https://markmail.org/message/grphwta63pp5p4l7 > > I’d like to propose some updates and tuning now that we have a Clover Jenkins > pipeline working (brainstormed with Simon): > > Here’s the new proposal: > > * We run the Clover Jenkins pipeline every night (between 11PM-8AM) > * The pipeline sends an email whenever the new report has modules lowering > the global TPC score compared to the baseline report (negative contribution > per module) > * The baseline report is the report generated just after each XS release. > This means that we keep the same baseline during a XS release > ** Technically it means that the pipeline will update the latest.txt file > (which contains the clover report timestamp) when it notices a version change > * We add a step in the Release Plan Template to have the report passing > before we can release. > * The RM is in charge of a release from day 1 to the release day (already the > case), and is also in charge of making sure that the global coverage job > failures get addressed before the release day so that we’re ready on the > release day. > > Options: > * Make it easier and only fail the pipeline job when the global TPC value is > lower than the baseline (vs failing whenever a module has negative > contribution) > > WDYT? > > Thanks > -Vincent >
