On Fri, Sep 9, 2016 at 5:14 AM, Daniel Pfeifer <dan...@pfeifer-mail.de> wrote:
> On Thu, Sep 8, 2016 at 5:52 PM, Brad King <brad.k...@kitware.com> wrote: > > I think if we introduce the notion of tests requiring other tests > > then a new model of test selection and enablement needs to be > > designed. Some kind of test DAG could be defined with various > > roots and subgraphs being selectable an causing all reachable > > tests to be included. > > This could be expanded even further. If "tests requiring other tests" > is generalized to "tests requiring X", wouldn't this allow incremental > testing? Say you change one file in your project. You rebuild only > the parts of the project that are affected by this change. Then you > rerun only the tests that are affected by the change. This really has > to be carefully thought out. > Interesting idea, but yes potentially a big body of work. That said, the fixtures implementation up for review already gets you most of the way there I think. One could potentially define a fixture that covers the tests for functionality a developer is working on. A test is able to require multiple fixtures, so developers could define fixtures to whatever granularity they wanted. We could add option(s) to the ctest command line which allow selection of tests based on fixture names in the same way we currently do with labels and test names. This would allow developers to run just the set of tests related to a particular functionality. One could argue that labels already get you most of the way there now too, but since fixtures would come with clean handing of setup/cleanup dependencies, they may be more useful to developers. They also feel a bit closer to the implementation than what labels are likely to cover and so fixtures might be a better fit for the scenario you mentioned. I'd also be skeptical whether we could make it easy/convenient/robust for CMake to work out which tests to re-run based on which files had to be rebuilt. Asking developers to pick which fixture(s) to retest instead might be a good compromise due to the simplicity of both use and implementation. Even aside from whether the above would satisfy the proposed scenario, maybe it's worth considering adding test selection based on fixture names regardless. I think this probably should be a separate branch that follows from the current one though so it doesn't turn into a monster of a merge. ;) I'd be happy to look at that idea once the current branch is done and in, since I have a concrete use case driving me to get that one completed first. -- Craig Scott Melbourne, Australia http://crascit.com
-- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers