This is an automated email from the ASF dual-hosted git repository.
alexey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
The following commit(s) were added to refs/heads/master by this push:
new 1474380f5 KUDU-3580 Fix the crash caused when binaries run on older
CPU machines
1474380f5 is described below
commit 1474380f5ccfd2f7e78756488d12eb52d2664132
Author: Yingchun Lai <laiyingc...@apache.org>
AuthorDate: Thu Apr 11 12:06:52 2024 +0800
KUDU-3580 Fix the crash caused when binaries run on older CPU machines
After Kudu linking rocksdb, the Kudu binaries may
crash with error "Illegal instruction" when running
on machines which don't support newer CPU
instruction (e.g. AVX512) but were built on a
machine which supports.
This patch enables the PORTABLE [1] option when building
librocksdb to fix the issue.
It should be noted that portable libraries may cause
a slight performance degradation, it's recommend to
disable portable option (by setting PORTABLE environment
variable to OFF when build Kudu thirdparties) if there
is no port requirements.
The PORTABLE option only takes effect on librocksdb
currently, the following content shows the comparation
of the 'db_bench' tool of RocksDB with the '-DPORTABLE'
option enabled and disabled benchmark results:
- The test is similar to Kudu use case, random write and
sequential read, key and value size is about 40 bytes.
- The tests ran 3 times.
- The binaries are built and run on the same machine
which supports newer CPU instruction (e.g. AVX512).
PORTABLE:
$ ./db_bench -benchmarks=fillrandom,readseq -num=10000000 -key_size=40
-value_size=40
1.
fillrandom : 5.237 micros/op 190954 ops/sec 52.369 seconds 10000000
operations; 14.6 MB/s
readseq : 0.448 micros/op 2231382 ops/sec 2.833 seconds 6322271
operations; 170.2 MB/s
2.
fillrandom : 5.236 micros/op 190981 ops/sec 52.361 seconds 10000000
operations; 14.6 MB/s
readseq : 0.444 micros/op 2252646 ops/sec 2.806 seconds 6321658
operations; 171.9 MB/s
3.
fillrandom : 5.182 micros/op 192960 ops/sec 51.824 seconds 10000000
operations; 14.7 MB/s
readseq : 0.444 micros/op 2252317 ops/sec 2.807 seconds 6323209
operations; 171.8 MB/s
NON-PORTABLE:
$ ./db_bench -benchmarks=fillrandom,readseq -num=10000000 -key_size=40
-value_size=40
1.
fillrandom : 5.190 micros/op 192676 ops/sec 51.900 seconds 10000000
operations; 14.7 MB/s
readseq : 0.391 micros/op 2560051 ops/sec 2.470 seconds 6322786
operations; 195.3 MB/s
2.
fillrandom : 5.156 micros/op 193945 ops/sec 51.561 seconds 10000000
operations; 14.8 MB/s
readseq : 0.404 micros/op 2477956 ops/sec 2.551 seconds 6320644
operations; 189.1 MB/s
3.
fillrandom : 5.527 micros/op 180940 ops/sec 55.267 seconds 10000000
operations; 13.8 MB/s
readseq : 0.407 micros/op 2458297 ops/sec 2.571 seconds 6320885
operations; 187.6 MB/s
1. https://github.com/facebook/rocksdb/blob/v7.7.3/CMakeLists.txt#L248
Change-Id: Id30ae995c41a592fccbdb822bc1f457c5e6878ac
Reviewed-on: http://gerrit.cloudera.org:8080/21287
Tested-by: Alexey Serbin <ale...@apache.org>
Reviewed-by: Alexey Serbin <ale...@apache.org>
---
thirdparty/build-definitions.sh | 1 +
thirdparty/build-thirdparty.sh | 6 ++++++
2 files changed, 7 insertions(+)
diff --git a/thirdparty/build-definitions.sh b/thirdparty/build-definitions.sh
index 4e31d9b3c..1199ecafc 100644
--- a/thirdparty/build-definitions.sh
+++ b/thirdparty/build-definitions.sh
@@ -1198,6 +1198,7 @@ build_rocksdb() {
CXXFLAGS="$EXTRA_CXXFLAGS -fPIC" \
cmake \
-DROCKSDB_BUILD_SHARED=ON \
+ -DPORTABLE=$PORTABLE \
-DWITH_SNAPPY=ON \
-Dsnappy_ROOT_DIR=$PREFIX \
-DUSE_RTTI=ON \
diff --git a/thirdparty/build-thirdparty.sh b/thirdparty/build-thirdparty.sh
index 937beeda7..ae32a4f3b 100755
--- a/thirdparty/build-thirdparty.sh
+++ b/thirdparty/build-thirdparty.sh
@@ -35,6 +35,9 @@
# * EXTRA_LIBS - additional libraries to link.
# * EXTRA_MAKEFLAGS - additional flags passed to make.
# * PARALLEL - parallelism to use when compiling (defaults to number of
cores).
+# * PORTABLE - whether to build portable libraries, otherwise build native
libraries. Portable
+# libraries may cause a slight performance degradation, it's
recommend to disable
+# portable option if there is no port requirements. (defaults
to ON).
set -ex
@@ -204,6 +207,9 @@ else
exit 1
fi
+### Build portable libraries by default.
+PORTABLE=${PORTABLE:-"ON"}
+
### Detect and enable 'ninja' instead of 'make' for faster builds.
if which ninja-build > /dev/null ; then
NINJA=ninja-build
