On Tue, Dec 15, 2015 at 10:33:38 -0800, Alan W. Irwin wrote: > On 2015-12-15 11:20-0500 Ben Boeckel wrote: > > I think, instead, that --min-timeout and --max-timeout options might be > > better which allow you to say "this machine is slow; tests may take > > longer (max(property, option))" or "this machine is fast, clamp down the > > timeout (min(property, option))". Alternatively, a --timeout-scale > > option to multiply all timeouts in properties might be better. > > I think we are stuck with --timeout since it has been available as a > ctest option for a long time. So the first priority is that option > should override anything set by the project, i.e., anything set by > the project should be considered to be a default value.
Changing --timeout's behavior isn't going to help you since old CMake versions won't work as intended. I don't think changing its behavior is best. For example, in our Buildbot infrastructure, we set the default timeout to 3 minutes because the default timeout for CTest causes tests to take *way* too long (I think it seems to be 25 minutes?). Some tests, however, *do* take longer than that on even reasonable hardware, so those should ignore the lower default timeout we set. Breaking this gets a *big* -1 from me. If you want compatibility between ctest versions, write a script to add arguments to a list based on the version that is running the script (any new command line arguments should get corresponding ctest_test() keywords). > That said, most cases including the ones I discussed above are scaling > ones so I agree it would be worthwhile to implement an additional > overriding --timeout-scale option for ctest which multiplies all > default timeouts set by the project. If the user specifies both > --timeout and --timeout-scale then one of them should be ignored in a > well-documented way (probably --timeout since --timeout-scale is a bit > more sophisticated). I think it would be "apply the scaling, but if there is no value, use the --timeout value". Whether that is pre- or post-scaling is arguable (one makes more sense reading, the other makes it easier to script). --Ben -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers