On Aug 21, 2012, at 9:52 AM, Vincent Massol <[email protected]> wrote:
> > On Aug 20, 2012, at 9:28 PM, Sergiu Dumitriu wrote: > >> On 08/20/2012 11:17 AM, Vincent Massol wrote: >>> Hi devs, >>> >>> Each java module has a current TPC level (Test Percentage Coverage), be it >>> 5% of 90%. >>> >>> As part of improving XWiki's quality, the idea is that future commits in >>> this module shouldn't lower the module's quality. >>> >>> One easy way to ensure this is to measure the current TPC level and make >>> the build fail whenever the build is under this TPC threshold. >>> >>> This means that if you commit code that has less tests average than what >>> currently exists then the build will fail. >>> >>> Here's one way to implement it: >>> >>> <!-- Fail the build if the test coverage is below a given value. >>> Note: Since this takes a bit of time to execute we only run this when >>> the integration-tests profile is active --> >>> <profiles> >>> <profile> >>> <id>integration-tests</id> >>> <build> >>> <plugins> >>> <plugin> >>> <groupId>com.atlassian.maven.plugins</groupId> >>> <artifactId>maven-clover2-plugin</artifactId> >>> <configuration> >>> <targetPercentage>86.9%</targetPercentage> >>> </configuration> >>> <executions> >>> <execution> >>> <id>clover-check</id> >>> <phase>verify</phase> >>> <goals> >>> <goal>instrument-test</goal> >>> <goal>check</goal> >>> </goals> >>> </execution> >>> </executions> >>> </plugin> >>> </plugins> >>> </build> >>> </profile> >>> </profiles> >>> >>> So this proposal is about the following strategy: >>> * Allow committers to modify module's pom.xml to fail the build when then >>> TPC is under the current threshold >>> * Since oldcore is really difficult to test, maybe we should not set build >>> failing for oldcore. OTOH oldcore is an important module. WDYT? >>> * If for some reason a committer wants to lower the TPC threshold for a >>> module he needs to get the agreement from the other committers (vote). >>> There might be valid reasons like fixing an important bug quickly for a >>> release and writing a test is just too complex so the writing of the test >>> is to be delayed for after the release, etc. >>> * I'd like to consider this as an experiment and see how it goes >>> >>> Note: This is about unit test TPC which is different from the TPC that >>> includes functional tests. >>> >>> WDYT? >>> >>> Here's my +1 to try this out and see how it goes. >> >> +1. >> >> I'm not sure that a [vote] is required, since that is usually something that >> is too formal. By the rules, a vote should last 3 days, it should have at >> least 3 +1, and no -1. But IMO a +1 means strong support, the voter >> understands and agrees with the matter, and lowering the TPC is not >> something on which people have a strong opinion. And the example that you >> give requires a quick decision, the 3 days required to pass a vote should be >> enough to write the missing tests. >> >> So, I'd rather trust developers not to lower the threshold without a valid >> reason, and we can always signal a change that we don't agree with >> afterwards. > > ok, makes sense. Actually I've experimented a valid use case this morning. When you add some code you have 3 choices for testing it: unit test, integration tests or functional tests. Unit tests and integration tests will count for the TPC threshold but not functional tests ATM (we don't measure it in the build). Thus when I add my code the build failed since my TPC % decreased. I could have written a functional tests and then I'd have had to lower the TPC threshold value in the pom.xml. This is valid. Thanks -Vincent _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
