> On 14 May 2018, at 13:48, Marius Dumitru Florea > <[email protected]> wrote:
[snip] >>>>> >>>>>> ** Find new packages introduced that have a TPC < the average computed >>>> TPC >>>>>> >>>>> >>>>> What is the average computed TPC currently? >>>> >>>> It’s about 70%, see for example: >>>> http://maven.xwiki.org/site/clover/20180511/clover- >>>> commons+rendering+platform-20180511-0147/dashboard.html >>>> >>>> >>> Requiring 70% TPC for new packages is a nice goal but I find it hard to >>> achieve in practice. >> >> Two points on this: >> > > >> * We should have above 80-90%+ in practice for new modules, not 70%. If we >> have 70% or less we can be sure you did a bad job on quality and that >> you’re impacting the overall quality of XWiki. >> > > *should* but I don't think it happens in practice, for various reasons, > mainly due to limited time. Did you check the TPC of the last 10 new > packages? Is it above 70%? I’ll check it. Thanks -Vincent > > >> * For some modules, it makes less sense or it’s harder to have unit tests >> and integration tests are better. We’re not talking about 70% of unit test >> coverage but 70%+ coverage of overall testing (unit, integration, >> functional). >> >> Now we’ll need to put in place to see it in action and verify if there are >> cases where this can be hard to achieve. But I’d prefer that we consider >> those cases as exceptional and handle them in an ad-hoc manner. >> >>> >>>>> >>>>>> ** Find packages and/or files having a TPC lower than the previous TPC >>>>>> ** Find removed packages that had a TPC higher than the average >> computed >>>>>> TPC >>>>>> >>>>> >>>>> I would add: >>>>> >>>>> ** find the packages that have the TPC higher than what is declared in >>>> the >>>>> pom (because we don't always update the TPC value declared in the pom >>>> when >>>>> we refactor the code or when we add new tests) >>>> >>>> Yes I agree. We should do that but not in the pipeline for the global >>>> coverage. We should have another pipeline for this and update the >> pom.xml >>>> files in it. >>>> >>>>> >>>>>> * Save a report in the directory for the Clover report at >>>>>> http://ci.xwiki.org/view/Tools/job/Clover/ >>>>>> * For all failures, send an email to [email protected] with >>>> details >>>>>> and a link to the saved report >>>>>> * Ideally, and if we can do it, call the github API to find the >> authors >>>> of >>>>>> commits for those packages and add them in the report. Examples of >> APIs >>>> we >>>>>> could use: >>>>>> ** https://api.github.com/repos/xwiki/xwiki-platform/commits? >>>>>> since=2018-05-07T00:00:00Z&until=2018-05-10T00:00:00Z (there’s a path >>>>>> parameter that could be used to filter but I don’t think it’ll work >>>>>> ** https://github.com/xwiki/xwiki-platform/compare/master@ >>>>>> %7B2018-05-07%7D...master@%7B2018-05-10%7D (the committers/authors >> need >>>>>> to be extracted from the HTML which is a bit fragile) >>>>>> * Add a step in the Release process to ensure that the global TPC has >>>> not >>>>>> been lowered. This would be a way to ensure we pay attention to that >> and >>>>>> fix it when we lower it. We would need to tune this to find something >>>> that >>>>>> helps keep the TPC increasing while not putting too much pressure at >> the >>>>>> same time on the release date. It doesn’t have to be in the release >>>> process >>>>>> but we need some checkpoint to make sure we look at it and that all >> devs >>>>>> fix the tests when they lower the global TPC, or at the very least >> that >>>> an >>>>>> analysis is done in case where it’s hard to keep the TPC (for example, >>>>>> removing existing code that had a lot of tests will lower the TPC ;)). >>>>>> >>>>>> WDYT? >>>>>> >>>>>> As a dev, would you be willing to pay attention to not lower the >> global >>>>>> TPC and work to fix it when it happens? >>>> >>>> @Marius: what about this question? Would you be ok with it? :) >>>> >>> >>> Sure. >> >> ok cool >> >> Thanks >> -Vincent >> >>> >>> >>>> >>>> Thanks >>>> -Vincent >>>> >>>>>> >>>>>> Thanks >>>>>> -Vincent
