Niclas Hedhman a écrit : > I just realized that even automated, we can't test all permutations. > Already at 2600 combinations, then take that times 4 seconds, and we are > taking 3 hours. When I add one more feature (say JMX), we then hit 6 hours, > 12 hours and so on. Quite soon it will take a week to test it all...
Well, it depends :-) We could run tests in parallel, have a "quick test" task that only triggers a selected list of permutations, CI could run those tests at some point in time etc... > On Sat, Mar 11, 2017 at 6:14 PM, Paul Merlin <[email protected]> wrote: > >> Hi Niclas, >> >> That sounds great, I'll have a look later to see how to automate the >> tests, we really don't want to tests all the permutations by hand. >> >> Cheers >> >> >> Niclas Hedhman a écrit : >>> Hi, >>> Last night I made a big improvement on the generator-polygene, making a >> lot >>> easier to work with the codebase (modular, creation-by-convention,...). >>> Still room for improvement in that area, but it is now miles better. >>> >>> I am toying with the idea of generating ALL possible projects in the >>> integration tests, to ensure that no conflict exists, or that something >> is >>> missed in some combination. But my Groovy-Fu is just not "there", and I >>> don't want to look "silly" by having a Java program with Groovy syntax. >>> >>> In essence, generate 2688 JSON documents with all possible combinations, >>> and pass them one by one to a command line. >>> >>> Why so many? >>> >>> 1 of 14 entitiy stores >>> x >>> 1 of 4 indexing >>> x >>> 1 of 2 serialization >>> x >>> 1 of 3 caching >>> x >>> 1 of 2 metrics >>> x >>> each of 2 features (4 combos) >>> >>> And with each other feature I add, the number doubles, so perhaps not >>> doable in the long run, as I intend to include the following features >>> eventually; >>> >>> 'rest api' >>> ,'security' >>> // ,'version migration' >>> // ,'logging' >>> // ,'jmx' >>> // ,'circuit breakers' >>> // ,'file transactions' >>> // ,'spring integration' >>> // ,'servlet deployment' >>> // ,'osgi support' >>> // ,'alarms' >>> // ,'scheduling' >>> // ,'groovy mixins' >>> // ,'javascript mixins' >>> >>> >>> Other things that I am working on are; >>> * Support for adding Domain Layer modules with entities, values, >>> objects.... >>> * Support for different application types; Rest, command line, WAR, >>> Docker(?) >>> * Post creation-time, creation of new modules with entities, values, >>> objects.... >>> * and hopefully, post creation-time addition of features. >>> >>> The last two are of course very tricky, since humans may have been >>> tinkering with files, so perhaps that is a pipe-dream. >>> >>> Anyway, if anyone could help out with the tests, then this could be >> wrapped >>> up pretty quickly. Meanwhile, I will continue on the other features. >>> >>> >>> Cheers > > >
