Hi David,
David Sommerseth wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 20/04/11 17:25, Jan Just Keijser wrote:
Hi *,
copying in the openvpn-devel list as they might be interested in this memory
usage analysis as well....
Ralf Hildebrandt wrote:
* Ralf Hildebrandt <ralf.hildebra...@charite.de>:
* Fredrik Kers <fredrik.k...@gmail.com>:
I measure the memory usage by checking the VmRSS (Resident Set Size) in
/proc/[pid]/status before and after the clients connect (about 1000 of
them).
So it's the same method (because that's what top and htop do).
I'm using a 64 bit system, maybe that will cause OpenVPN to consume
more memory per client then your setup
Yes, that could be. It's 32Bit here.
pmap -d pid_of_openvpn
is showing:
# pmap -d 1607
1607: /usr/sbin/openvpn --writepid /var/run/openvpn.server.pid --syslog
ovpn-server --cd /etc/openvpn --config /etc/openvpn/server.conf
Address Kbytes Mode Offset Device Mapping
08048000 500 r-x-- 0000000000000000 068:00005 openvpn
080c5000 4 rwx-- 000000000007d000 068:00005 openvpn
080c6000 24 rwx-- 0000000000000000 000:00000 [ anon ]
* 08b51000 41396 rwx-- 0000000000000000 000:00000 [ anon ]
b7270000 132 rwx-- 0000000000000000 000:00000 [ anon ]
...
bfef8000 132 rw--- 0000000000000000 000:00000 [ stack ]
mapped: 46240K writeable/private: 42304K shared: 0K
The line starting with * shows the biggest chunk of memory in use.
I'm running openvpn 2.1.4 on a 64bit system (CentSO 5.5 with openssl 0.9.8e).
when the first client connects the openvpn server
gets "hit" by ~ 600 Kb (as seen using pmap). The size of the 'hit' most likely
depends on the encryption cipher used.
I recompiled openvpn using
./configure --with-mem-check=valgrind
and then ran the server again using
valgrind --tool=massif .../openvpn
this results in a nice memory report which you can view using
ms_print massif.out.22218
turns out that most memory is allocated by the openssl libraries : CRYPTO_malloc
the memory trace is too large to show here, but there's tons of stuff in that
'ms_print' output.
Note that compression was NOT used in this example, apparently the openssl libs
call deflateInit internally?
Timestep 49 is when the client connects: you see the memory usage jump from
327,520 bytes to 934,688.
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
48 36,072,833 327,520 276,102 51,418 0
49 36,341,762 934,688 883,126 51,562 0
94.48% (883,126B) (heap allocation functions) malloc/new/new[], --alloc-fns,
etc.
Again, thanks a lot for wonderful reports. It is easy to shoot OpenSSL
here. But I'm not going to do that just yet. There are some work in
progress of modularising the SSL layer in OpenVPN (if patches are cleaned
up in time and get a decent review, we might have this in OpenVPN 2.3
already). This work will also add support for PolarSSL.
When we see those patches, it might be easier to see if there's something
which can be improved in OpenVPN or if it is something which is completely
into the hands of OpenSSL. So running this test with the PolarSSL module
instead will be very interesting.
I generally think that the crypto context needs to stay in memory for some
time, especially in UDP mode, to avoid re-negotiating of the connections in
case of connectivity issues between client and server. But when the
session is defined as closed by the OpenVPN server, this memory pool should
be released. For TCP mode, the session is closed (afair) when the client
explicitly closes the connection. In UDP mode it's a timeout, unless
- --explicit-exit is used. This freeing of context memory needs to be
verified in the code.
for the record: I did not mean to "shoot OpenSSL" here; I use OpenSSL a
lot for my regular work and I know how difficult it is to get it to
behave correctly when it comes to memory usage.
The reason I started digging was that I added a very dumb 'printf'
statement every time gc_alloc or ALLOC_OBJ was called (in openvpn
buffer.c) ; I then grabbed the output after a client connected and added
up all the mallocs : this did not even add up to 75 KB (and I'm probably
double counting too). So then I went on a hunt to find out what _IS_
causing the 600 KB hit, which led to the ms_print report.
I'm not too worried about a 600 KB per client hit for an OpenVPN server
- it means a 1,000 clients will eat up less than 1 GB of RAM . Unless
you're running on tiny hardware (dd-wrt etc) this should not be an
issue, but I would not recommend to connect a 1,000 clients to a single
dd-wrt box anyway.
It will be interesting to see if PolarSSL is actually better in terms of
memory consumed.
share and enjoy,
JJK
