Hello Wouter, testing CMake code is indeed very important. This is why kitware does it as well. Did you check the CMake code testing infrastructure in "Tests/RunCMake" in the sources? It is a very flexible concept which makes adding tests easy enough for everyone contribute (IMO). At the beginning it might be a bit confusing how expected results etc. are handled, but once you understand how it works it's really nice.
Another (maybe the main) advantage: you have tons of examples and the RunCMake infrastructure is maintained so you don't have to it all on your own. best regards, Michael > -----Ursprüngliche Nachricht----- > Von: cmake-developers [mailto:cmake-developers-boun...@cmake.org] Im > Auftrag von Wouter Klouwen > Gesendet: Dienstag, 29. Mai 2018 18:36 > An: CMake Developers > Betreff: [cmake-developers] Unit testing CMake modules > > Hi all, > > We have a rather large amount of CMake code (4k+ lines) in various > modules/functions. They contain the common logic for many of our projects. > This makes it quite an important base for everything our team does. > > Rather shamefully at present these modules are rather poorly tested. I'd like > to improve this, but the current way of testing CMake code is typically to run > a trimmed project, and to verify whether certain invocations produce a > certain output or file hierarchy. > This involves a bit of infrastructure and can be a bit cumbersome maintain, to > diagnose when tests fail, and it requires a separate run of the tests. > The overall cumbersomeness of the setup in turn discourages in our team, > including myself, from adding tests. > > I'd like a more integrated approach that makes running at least some basic > tests part of the build progress, and a more direct way of reporting failures. > > In other programming environments, testing often involves some kind of > mocking environment, and CMake helpfully allows the overriding of CMake > built in functions, though this is typically discouraged. > > In my ideal world it would be possible to save the state of the current > function set up, then call a function with a certain given number of > parameters, and expect a certain sequence of events, such as functions to > be called, environment variables to be set, etc. > Something akin to CMakePushCheckState, except for built in functions. > > Then a module could provide for some functions that would set up > expectations and verify them, within a run of cmake, or possibly some other > commands could be added, to give some syntactic glossy coat to it. > > As it wouldn't actually trigger any of the expensive generating functions, it > would be lightweight, quick to run and give pretty direct errors in terms of > failed expectations, reducing debug time. > > If it was done with CMake commands, I might imagine it to look something > like: > > function(foobar) > # function to test, does something non trivial > if ("FOO" IN_LIST ARGV) > install(FILES foo DESTINATION foo_dir) > else("BAR" IN_LIST ARGV) > message(FATAL_ERROR "Some error") > else("BAZ" IN_LIST ARGV) > set(BAZ True PARENT_SCOPE) > endif() > endfunction() > > test(foobar) > expect(WITH "FOO" CALL install FILES foo DESTINATION foo_dir) > expect(WITH "BAR" CALL message FATAL_ERROR "Some error") > expect(WITH "BAZ" ENVIRONMENT BAZ True) > endtest(foobar) > > What do people think? Is this crazy? Is there a quicker way to get somewhere > close? Should I put some effort into making this into an actual > proposal/working code? > > Thanks in advance, > W > > > This transmission contains information that may be confidential and contain > personal views which are not necessarily those of YouView TV Ltd. YouView > TV Ltd (Co No:7308805) is a limited liability company registered in England > and > Wales with its registered address at YouView TV Ltd, 3rd Floor, 10 Lower > Thames Street, London, EC3R 6YT. For details see our web site at > http://www.youview.com > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > https://cmake.org/mailman/listinfo/cmake-developers -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake-developers
