Brad, Whilst I am not really doing much with Chapel these days, a few thoughts in case they help. It may just express things people already know, in which case apologies for "statin' the bleedin' obvious".
The JUnit3 JUnit4 approach to testing in Java, which is based on sUnit from Smalltalk, assumes the entire language is object-based with objects being instances of classes. I haven't used JUnit5 much but I suspect it is the same. TestNG improved on JUnit3 and JUnit4 by using annotations before JUnit4 existed and by separating test suites using an external XML system. Spock Framework improved on everything by using Groovy in a very special way to provide an excellent test specification DSL. Now whilst sUnit/JUnit3 caused a revolution in testing, trying to impose the object/class/reflection model caused lots of problems in languages that didn't really support that model, most particularly C++. It wasn't really till post C++11 that C++ test frameworks worked in harmony with the language rather than being a Java model imposed on C++. For me the current market leader is Catch2, but there are other header-only frameworks. I am not entirely up-to-date as I have stopped using C++. Of course in the Haskell world there arose QuickCheck and the whole property-based testing revolution to test frameworks. Again there was a fashion for trying to port this framework to other languages. Often badly, either because the language wasn't suitable, or because the port was a poor one. Names will not be mentioned. However Rust's QuickCheck works nicely. During all this period Python had unittest, a port of JUnit3 to Python. Python supports all the object/class/refelction and so unittest worked quite nicely, indeed became to de facto and de jure standard. However it was class based and many found this irritating for a language in which functions did not have to exist inside classes. (Sound familiar?) This gave rise to pytest which for me beats unittest hands down. pytest makes writing unit and functional tests, and indeed benchmarks, for code really easy and indeed quite enjoyable. And then there is Hypothesis: this brings a re-engineering of the QuickCheck property- based testing idea appropriate to the Python language – as opposed to being a port of QuickCheck. Hypothesis works hand-in-hand with pytest and all it's plugins to make a truly magnificent testing toolkit. As Chapel has picked up Cargo to lead to Mason (I've been away from Chapel a while hopefully this is still the case), Chapel should pick the best for a test framework – even though there is already a system for testing the compiler, runtime system, and standard library. For me pytest and hypothesis are the market leaders, but they are founded on Python's reflection capability. Rust of course has enshrined unit testing within modules using a built- in runner. D went this route long before. Both avoid any need for manual suite construction. But this is for unit testing alone. Rust and D have a convention for functional tests in a separate directory hierarchy with a built-in runner and hence avoiding manual building of suites. Rust has the quickcheck crate which is rather wonderful, and shows that property based testing is feasible within the Rust unit testing framework. D hasn't got this as yet, but it does have the compile time features needed. D is very different from Rust though in how the compile time features work. Missing from this is a compare and contrast of the Rust and D situation with that of Chapel. This may undermine somewhat the above commentary as input to the future of testing in Chapel. When I looked for a testing framework for Chapel applications the one I found (I can't remember the name offhand, sorry) was like C++Unit, JUnit3 based with manual construction of suites. Given the situation in Python and Rust, this was so backward looking I gave up with it immediately. Whilst pytest and hypothesis may not be feasible in Chapel, anything Rust and D can do, I am sure Chapel can. So integrated unit testing per module with a property-based framework with shrinking, and a convention for handlng functional tests would be order of the day. -- Russel. =========================================== Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
signature.asc
Description: This is a digitally signed message part
_______________________________________________ Chapel-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/chapel-developers
