> From: Sébastien Fauvel <sfau...@gmail.com> > Sent: 07 September 2022 00:33 > > To quickly create non-regression tests on cases with high combinatorics, > I use approvaltests. You execute code with all cases and for each, you write > the value in a file. You just have to check that this file is always the same > after a test execution to make the test passed. > > I made an example you can see here: > https://github.com/sfauvel/freecol/blob/test_with_approvals/test/src/net/sf/freecol/common/model/ColonyProductionApprovalsTest.java > > [...] > > I'm not sure about all the cases you want to test and how to generate them > but If you think this approach could help, I can explain in more detail or > complete those tests to cover more cases..
Sorry for the late reply -- I thought I had already responded, but obviously not. The problem is that it's not entirely clear what the actual production should be in the classic ruleset, and just looking at the current production (and approving it) will probably not discover any more bugs with the current implementation. However, using approval tests for regression testing might be a really good idea. A solution like this is especially interesting when making changes to the specification (mods), since it can be used to both quickly determine if a change has the desired outcome and if any regressions have happened. There's already a documentation generator for the rules (GenerateDocumentation). We could use this approach for making the documentation better as well. For example, we could make a utility for both generating the documentation and verifying the specification changes: verifyRulesAndDocument --output=build/my-mod data/default data/rules/classic data/mods/my-mod mv build/my-mod docs/my-mod verifyRulesAndDocument --expected=docs/my-mod --output=build/my-mod data/default data/rules/classic data/mods/my-mod The tool would take the TC, rule and a list of mods as input ... and use that for generating the documentation + files with the outputs we want to verify. These files can then later be used as the expected values. We can also make a list of differences highlighting the changes that happens after applying the mod. That is something that might even be interesting to display in the mod selection dialog. Anyone who wants to work on a solution like this? Best wishes, Stian Grenborgen _______________________________________________ Freecol-developers mailing list Freecol-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freecol-developers
