During our tests with OVSDB we found out that memory allocation is
used intensively by the server, so we thought that using a different
memory allocator could increase the performance. We tried jemalloc
and the performance gain was between 20% and 40%.
This patch would allow anyone to enable jemalloc on their systems
in case you want to try out this memory allocator.
The patch adds the --enable-jemalloc flag to link ovsdb-server with jemalloc
instead of the default memory allocator. This can improve the OVSDB
Server performance under certains loads.
Signed-off-by: Esteban Rodriguez Betancourt <esteb...@hpe.com>
---
.travis.yml | 3 +++
DESIGN.md | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
INSTALL.md | 4 ++++
configure.ac | 14 ++++++++++++++
ovsdb/automake.mk | 4 ++++
5 files changed, 76 insertions(+)
diff --git a/.travis.yml b/.travis.yml
index 52c9362..fa7cefa 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -10,6 +10,8 @@ addons:
- gcc-multilib
- libssl-dev
- llvm-dev
+ - libjemalloc1
+ - libjemalloc-dev
before_install: ./.travis/prepare.sh
@@ -33,6 +35,7 @@ env:
- KERNEL=3.4.110
- KERNEL=3.2.76
- KERNEL=2.6.32.70
+ - OPTS="--enable-jemalloc"
script: ./.travis/build.sh $OPTS
diff --git a/DESIGN.md b/DESIGN.md
index 6865d47..85d4f11 100644
--- a/DESIGN.md
+++ b/DESIGN.md
@@ -1082,6 +1082,57 @@ against desired actions in a bytewise fashion:
Please report other discrepancies, if you notice any, so that we can
fix or document them.
+jemalloc as Default Memory Manager for OVSDB-Server
+===================================================
+
+Background
+----------
+
+OVSDB-Server performs a lot of memory-related operations. Many of those are
related to
+allocating small pieces of memory. It requests and releases memory continually
for creating
+and destroying objects like dynamic strings, json objects, etc.
+
+Tests revealed that by replacing the GLibC memory allocator with the [jemalloc]
+(http://www.canonware.com/jemalloc) memory allocator, OVSDB-Server's
performance can be
+significantly improved, and also, in some cases, can reduce the memory usage.
+
+## Performance Improvements
+
+Several tests (insertion, updates, pubsub and transaction size) were run in
order to
+compare the GLibC performance and memory usage along both jemalloc and
tcmalloc memory
+allocators.
+
+The results showed favorable results for jemalloc library.
+
+- Update 1: Each of 10 parallel workers do 25000 updates over 1000 rows.
+- Update 2: Each of 10 parallel workers do 25000 updates over 200000 rows.
+- Insert: Each of 10 parallel workers insert 25000 rows.
+- Message Queue Simulation: One producer waits for requests (requested by 10
parallel workers).
+ For each request the producer updates 512 rows, composed of one map column
with 256 elements.
+- Transaction Size: The program inserts 100, 1000, 10000, 100000 and 500000
rows, alternating
+ between inserting all the rows in the same transaction, or inserting one row
per transaction.
+
+| Test | GLibC | jemalloc | tcmalloc |
+|----------|-----------|------------|------------|
+| Update 1 | 28s | 24s | 31s |
+| Update 2 | 57s | 38s | 33s |
+| Insert | 38s | 32s | 27s |
+| Queue | 4s | 4s | 4s |
+| Size | 129.92s | 119.87s | 114.21s |
+| Duration (seconds) |
+
+
+| Test | GLibC | jemalloc | tcmalloc |
+|----------|-----------|------------|------------|
+| Update 1 | 7.35 | 18.92 | 171.42 |
+| Update 2 | 1090.7 | 82.88 | 111.08 |
+| Insert | 66.66 | 57.86 | 127.56 |
+| Queue | 1094.40 | 28.13 | 270.47 |
+| Size | 410.17 | 116.50 | 114.21 |
+| Memory Usage (RSS, megabytes) |
+
+The results show a consistent better results of jemalloc over GLibC memory
allocator.
+TCMalloc was faster in some benchmarks, but uses a lot more of RAM than
jemalloc.
Suggestions
===========
diff --git a/INSTALL.md b/INSTALL.md
index dc688ad..51d2228 100644
--- a/INSTALL.md
+++ b/INSTALL.md
@@ -257,6 +257,10 @@ Here is an example:
`% mkdir _gcc && (cd _gcc && ../configure CC=gcc)`
`% mkdir _clang && (cd _clang && ../configure CC=clang)`
+To use jemalloc as the memory allocator for OVSDB Server, add
+--enable-jemalloc, which links OVSDB Server with jemalloc (the library
+must be installed on the system). For the rationale about this flag
+refer to DESIGN.md.
Building the Sources
--------------------
diff --git a/configure.ac b/configure.ac
index 49aa182..4e0369c 100644
--- a/configure.ac
+++ b/configure.ac
@@ -43,6 +43,20 @@ AC_SYS_LARGEFILE
LT_INIT([disable-shared])
m4_pattern_forbid([LT_INIT]) dnl Make autoconf fail if libtool is missing.
+AC_ARG_ENABLE([jemalloc],
+ [AS_HELP_STRING([--enable-jemalloc], [use jemalloc as the memory manager
+ instead of GLibC] @@)],
+ [enable_jemalloc=${enableval}],
+ [enable_jemalloc=no])
+
+AS_IF([test "x$enable_jemalloc" != xno],
+ [
+ AC_SEARCH_LIBS([malloc], [jemalloc],
+ [AM_CONDITIONAL(USE_JEMALLOC, true)],
+ [AM_CONDITIONAL(USE_JEMALLOC, false)])
+ ],
+ [AM_CONDITIONAL(USE_JEMALLOC, false)])
+
# The following explanation may help to understand the use of the
# version number fields: current, revision, and age.
#
diff --git a/ovsdb/automake.mk b/ovsdb/automake.mk
index 7db6fea..6a9cc8a 100644
--- a/ovsdb/automake.mk
+++ b/ovsdb/automake.mk
@@ -66,6 +66,10 @@ MAN_ROOTS += ovsdb/ovsdb-client.1.in
sbin_PROGRAMS += ovsdb/ovsdb-server
ovsdb_ovsdb_server_SOURCES = ovsdb/ovsdb-server.c
ovsdb_ovsdb_server_LDADD = ovsdb/libovsdb.la lib/libopenvswitch.la
+if USE_JEMALLOC
+ovsdb_ovsdb_server_LDADD += -ljemalloc
+endif
+
# ovsdb-server.1
man_MANS += ovsdb/ovsdb-server.1
DISTCLEANFILES += ovsdb/ovsdb-server.1
--
1.9.1
_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev
