Hi Shahzeb,
On 07/03/2017 17:31, Siddiqui, Shahzeb wrote:
Hello,
I have been working on creating a testing framework to automatically
generate test cases for applications that work with EasyBuild in a
HMNS module layout. I couldn’t seem to find any tool that provides
test cases for HPC apps that work properly with a module environment
so I decided to create me own.
I have pushed some test cases for intel, gcc, openmpi, mpich as
examples. If anyone is interested in developing a testing framework,
check out https://github.com/shahzebsiddiqui/testgen-HPC and maybe we
can collaborate on a standard for basic test
cases for each application.
I haven't looked into this in detail, but it sure looks interesting.
To be honest, I'm not a big fan of CMake myself, but I'm not familiar
with ctest so I'll give it the benefit of doubt here. ;-)
A major thing missing right now is a proper open source license.
This could be a major help: https://choosealicense.com/, although in
your case you probably also have company approval to worry about?
One other thing I think you should do early on is to define some utility
functions in bash to remove some of the heavy copy-pasting.
For example, you should have 'load_module' function where you load a
specified module, and check the exit code, and maybe also check that the
module indeed appears in 'module list'.
You should also have a function like 'run_and_check' that runs a
specified command, captures stdout/stderr, and checks the exit code.
Also try to avoid copy-pasting tests (cfr. OpenMPI), but maybe have some
way of defining both version-agnostic and version-specific tests.
These are just some random thoughts, I like the approach overall.
Maybe this could grow out to be a much used testing framework....
regards,
Kenneth
Regards,
Shahzeb Siddiqui
HPC Linux Engineer
B2220-447.2
Groton, CT
