This is an automated email from the ASF dual-hosted git repository. alexey pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/kudu.git
The following commit(s) were added to refs/heads/master by this push: new 1474380f5 KUDU-3580 Fix the crash caused when binaries run on older CPU machines 1474380f5 is described below commit 1474380f5ccfd2f7e78756488d12eb52d2664132 Author: Yingchun Lai <laiyingc...@apache.org> AuthorDate: Thu Apr 11 12:06:52 2024 +0800 KUDU-3580 Fix the crash caused when binaries run on older CPU machines After Kudu linking rocksdb, the Kudu binaries may crash with error "Illegal instruction" when running on machines which don't support newer CPU instruction (e.g. AVX512) but were built on a machine which supports. This patch enables the PORTABLE [1] option when building librocksdb to fix the issue. It should be noted that portable libraries may cause a slight performance degradation, it's recommend to disable portable option (by setting PORTABLE environment variable to OFF when build Kudu thirdparties) if there is no port requirements. The PORTABLE option only takes effect on librocksdb currently, the following content shows the comparation of the 'db_bench' tool of RocksDB with the '-DPORTABLE' option enabled and disabled benchmark results: - The test is similar to Kudu use case, random write and sequential read, key and value size is about 40 bytes. - The tests ran 3 times. - The binaries are built and run on the same machine which supports newer CPU instruction (e.g. AVX512). PORTABLE: $ ./db_bench -benchmarks=fillrandom,readseq -num=10000000 -key_size=40 -value_size=40 1. fillrandom : 5.237 micros/op 190954 ops/sec 52.369 seconds 10000000 operations; 14.6 MB/s readseq : 0.448 micros/op 2231382 ops/sec 2.833 seconds 6322271 operations; 170.2 MB/s 2. fillrandom : 5.236 micros/op 190981 ops/sec 52.361 seconds 10000000 operations; 14.6 MB/s readseq : 0.444 micros/op 2252646 ops/sec 2.806 seconds 6321658 operations; 171.9 MB/s 3. fillrandom : 5.182 micros/op 192960 ops/sec 51.824 seconds 10000000 operations; 14.7 MB/s readseq : 0.444 micros/op 2252317 ops/sec 2.807 seconds 6323209 operations; 171.8 MB/s NON-PORTABLE: $ ./db_bench -benchmarks=fillrandom,readseq -num=10000000 -key_size=40 -value_size=40 1. fillrandom : 5.190 micros/op 192676 ops/sec 51.900 seconds 10000000 operations; 14.7 MB/s readseq : 0.391 micros/op 2560051 ops/sec 2.470 seconds 6322786 operations; 195.3 MB/s 2. fillrandom : 5.156 micros/op 193945 ops/sec 51.561 seconds 10000000 operations; 14.8 MB/s readseq : 0.404 micros/op 2477956 ops/sec 2.551 seconds 6320644 operations; 189.1 MB/s 3. fillrandom : 5.527 micros/op 180940 ops/sec 55.267 seconds 10000000 operations; 13.8 MB/s readseq : 0.407 micros/op 2458297 ops/sec 2.571 seconds 6320885 operations; 187.6 MB/s 1. https://github.com/facebook/rocksdb/blob/v7.7.3/CMakeLists.txt#L248 Change-Id: Id30ae995c41a592fccbdb822bc1f457c5e6878ac Reviewed-on: http://gerrit.cloudera.org:8080/21287 Tested-by: Alexey Serbin <ale...@apache.org> Reviewed-by: Alexey Serbin <ale...@apache.org> --- thirdparty/build-definitions.sh | 1 + thirdparty/build-thirdparty.sh | 6 ++++++ 2 files changed, 7 insertions(+) diff --git a/thirdparty/build-definitions.sh b/thirdparty/build-definitions.sh index 4e31d9b3c..1199ecafc 100644 --- a/thirdparty/build-definitions.sh +++ b/thirdparty/build-definitions.sh @@ -1198,6 +1198,7 @@ build_rocksdb() { CXXFLAGS="$EXTRA_CXXFLAGS -fPIC" \ cmake \ -DROCKSDB_BUILD_SHARED=ON \ + -DPORTABLE=$PORTABLE \ -DWITH_SNAPPY=ON \ -Dsnappy_ROOT_DIR=$PREFIX \ -DUSE_RTTI=ON \ diff --git a/thirdparty/build-thirdparty.sh b/thirdparty/build-thirdparty.sh index 937beeda7..ae32a4f3b 100755 --- a/thirdparty/build-thirdparty.sh +++ b/thirdparty/build-thirdparty.sh @@ -35,6 +35,9 @@ # * EXTRA_LIBS - additional libraries to link. # * EXTRA_MAKEFLAGS - additional flags passed to make. # * PARALLEL - parallelism to use when compiling (defaults to number of cores). +# * PORTABLE - whether to build portable libraries, otherwise build native libraries. Portable +# libraries may cause a slight performance degradation, it's recommend to disable +# portable option if there is no port requirements. (defaults to ON). set -ex @@ -204,6 +207,9 @@ else exit 1 fi +### Build portable libraries by default. +PORTABLE=${PORTABLE:-"ON"} + ### Detect and enable 'ninja' instead of 'make' for faster builds. if which ninja-build > /dev/null ; then NINJA=ninja-build