Todd Lipcon has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/9503 )
Change subject: build: adjust the number of processors for various tests ...................................................................... build: adjust the number of processors for various tests Many of our tests are multithreaded and can utilize many cores worth of CPU during execution. Other tests are somewhat timing-sensitive and might time out if run in coordination with a stress test, particularly on a small machine which might only have four CPUs total. This patch adjusts the core count property for a bunch of tests based on some empirical analysis using a new 'perf-stat' wrapper capability provided in run-test.sh. The wrapper produces periodic samples of the CPU usage for each test. Measurements are written into log files which can then be analyzed with a command like the following: for x in build/latest/test-logs/*.perf.txt ; do echo -n $x grep task-clock $x | sort -nk2 | tail -1 done | sort -nk3 This resulted in a list of tests along with their peak CPU usage (from their busiest 1000ms period). I used this as a rough guide to find tests with inappropriate settings in CMakeLists.txt. The net effect of this change is that 'ctest -j' might run slightly longer, but that we'll have fewer flaky tests in such a run configuration because we don't have stress tests perturbing timing of unrelated non-stress tests. Change-Id: I922031af69178b98459d0d59287442b692425afa Reviewed-on: http://gerrit.cloudera.org:8080/9503 Tested-by: Kudu Jenkins Reviewed-by: Adar Dembo <a...@cloudera.com> --- M CMakeLists.txt M build-support/run-test.sh M src/kudu/cfile/CMakeLists.txt M src/kudu/client/CMakeLists.txt M src/kudu/clock/CMakeLists.txt M src/kudu/consensus/CMakeLists.txt M src/kudu/integration-tests/CMakeLists.txt M src/kudu/mini-cluster/CMakeLists.txt M src/kudu/rpc/CMakeLists.txt M src/kudu/security/CMakeLists.txt M src/kudu/tablet/CMakeLists.txt M src/kudu/tools/CMakeLists.txt M src/kudu/tserver/CMakeLists.txt M src/kudu/util/CMakeLists.txt 14 files changed, 80 insertions(+), 48 deletions(-) Approvals: Kudu Jenkins: Verified Adar Dembo: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/9503 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I922031af69178b98459d0d59287442b692425afa Gerrit-Change-Number: 9503 Gerrit-PatchSet: 4 Gerrit-Owner: Todd Lipcon <t...@apache.org> Gerrit-Reviewer: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon <t...@apache.org>
