2018-01-12 10:58 GMT+01:00 Alan W. Irwin <ir...@beluga.phys.uvic.ca>:
> I am looking into the practicality of using clusters containing ~5 > cheap ARM 8-core computers (such as the Banana Pi M3) to rapidly build > and test software (since even with ccache I am currently spending a > lot of my time waiting for tests to complete as I develop my > software). Such clusters would give you ~40 cores which is a lot of > computer power for ~$500 or so. But, of course, the issue with > clusters is how to use them efficiently? > > For one 8-core M3 you would expect > > make -j<NNN> target > > to scale linearly with NNN up to the saturation limit of roughly NNN=8 > (where target depends on ~hundreds of different and mostly independent > tests). But my understanding from recent Google searching is that the > -j option for GNU Make is not MPI-aware, i.e., only works for the > cores for a single computer (node) in a cluster. So in a cluster you > would have to assign the above task to just one node and simply accept > that you cannot scale NNN up to the total number of cores in that > cluster. > > AFAIU what you want is "distributed ctest" not really distributed build right? For distributed build you may have a look at distcc (http://distcc.org/) which seems already used by CMake-enabled project like ccache is ( https://stackoverflow.com/questions/32752446/using-compiler-prefix-commands-with-cmake-distcc-ccache/34317588 ) Now I'm not quite sure to understand why you speak of MPI-awareness ? Are your test including MPI which would mean that some executable are already eating up more than one core? Or do you have a big collection of mono-processor tests you want to run on the cluster? -- Eric
-- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
