On Tue, Aug 23, 2016 at 08:00:09 +0200, Rolf Eike Beer wrote:
> Am Dienstag, 23. August 2016, 10:06:01 schrieb Craig Scott:
> > So how would you want the feature to work? I'd suggest an initial set of
> > requirements something like the following:
> >
> > - Need to support the ability to define multiple setup and/or tear down
> > tasks.
> > - It should be possible to specify dependencies between setup tasks and
> > between tear down tasks.
> > - Individual tests need to be able to indicate which setup and/or tear
> > down tasks they require, similar to the way DEPENDS is used to specify
> > dependencies between test cases.
> > - When using ctest --rerun-failed, ctest should automatically invoke any
> > setup or tear down tasks required by the test cases that will be re-run.
> > - Setup or tear down tasks which reference executable targets should
> > substitute the actual built executable just like how add_custom_command()
> > does.
>
> -need a way to mark if 2 tests with the same setup/teardown can share those
> or
> if they need to run for every of them
Proposal:
add_test(NAME setup-foo ...)
set_tests_properties(setup-foo PROPERTIES
SETUP_GROUP foo
SETUP_STEP SETUP_PER_TEST) # Also SETUP_ONCE.
add_test(NAME use-foo ...)
set_tests_properties(use-foo PROPERTIES
SETUP_GROUP foo) # implicit depends on all SETUP_GROUP foo / SETUP_STEP
SETUP_* tests.
add_test(NAME use-foo2 ...)
set_tests_properties(use-foo2 PROPERTIES
SETUP_GROUP foo)
add_test(NAME teardown-foo2 ...)
set_tests_properties(teardown-foo2 PROPERTIES
SETUP_GROUP foo
SETUP_STEP TEARDOWN) # implicit depends on all non-TEARDOWN steps
Multiple setup/teardown steps could be done with DEPENDS between them.
> -the default for each test is "no s/t", which means it can't be run with any
> of the above in parallel (especially for compatibillity)[1]
> -need a way to tell if a test doesn't care about those
Making RESOURCE_LOCK a rwlock rather than a mutex might make sense here.
SETUP_STEP bits have a RESOURCE_LOCK_WRITE group_${group}, otherwise it
is RESOURCE_LOCK_READ group_${group}.
> 1) think of a database connector test: the test that will check what happens
> if no DB is present will fail if the setup step "start DB" was run, but not
> the teardown
RESOURCE_LOCK on that group_${group} can fix that I think.
> > Some open questions:
<snip>
I agree with what Eike said.
--Ben
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers
