Send kea-dev mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.isc.org/mailman/listinfo/kea-dev
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of kea-dev digest..."
Today's Topics:
1. Suggestions for DHCP packets logs (Chaigneau, Nicolas)
2. Re: Suggestions for DHCP packets logs (Chaigneau, Nicolas)
3. Re: Suggestions for DHCP packets logs (Angelo Failla)
4. Re: Suggestions for DHCP packets logs (Francis Dupont)
----------------------------------------------------------------------
Message: 1
Date: Fri, 12 Jun 2015 09:11:33 +0000
From: "Chaigneau, Nicolas" <[email protected]>
To: Kea Dev List <[email protected]>
Subject: [kea-dev] Suggestions for DHCP packets logs
Message-ID:
<ab94b0b675bdf14189cd5a861db36c841954e...@de-cm-mbx26.corp.capgemini.com>
Content-Type: text/plain; charset="us-ascii"
Hello,
The following are recommendations for implementing the logging of DHCP packets.
These originate from functional requirements, and from performance tests I've
run (on our implementation of packets logging, handled through a hook).
Discussion is welcome as always!
1) don't use log4cplus
There is a very significant performance penalty when logging a large volume of
data through log4cplus.
Packets should not be logged through log4cplus.
Packets should be written directly through a fstream. (I've measured about 30%
improvement in DORA/s capacity, as compared to using log4cplus).
2) enabling packets logs
A parameter should allow to enable or disable the packets logs.
3) flushing of packets logs
There is also a performance benefit in allowing to disable explicit flushing
(ie each time a line is logged) of the log file.
In this case, automatic flushing is handled by the system and happens whenever
the output buffer is full (or when the file is closed).
This is not necessarily desirable for everyone.
A parameter should be provided to control whether packets logs are flushed
explicitly or automatically.
4) handle packets logs rotation
Rotation of the packets logs should not be delegated to an external task such
as logrotate.
This should be handled directly by Kea.
The performance test results are unequivocal, even though this was not really
expected (as logrotate is run by another CPU).
When logrotate works in copytruncate mode on the packets logs, the following
happens while the log is being processed:
- Kea CPU usage spikes to 100%
- DHCP packets are being dropped (from the client point of view)
As a consequence, users cannot get leases when logrotate is processing the
packets logs (and this takes some time because the files are huge - about 30 to
40 seconds for a 4.5 GB file).
This is clearly not acceptable.
A solution is to have Kea directly handle the file rotation.
This can be done on several criteria, for instance:
- on a time frame basis (eg. hourly packets log files),
- when file size exceeds a given limit,
- or a number of logged packets, ...
I've chosen the first approach, which is to log to a timestamped hourly file
such as "log-packets-<YYYYMMDD><HH>".
Kea closes the file and opens a new one whenever the file name (according to
current time) changes.
The rotation criteria and file name format should be configurable.
I need to rotate files hourly (otherwise the files are way too big and are very
hard to handle subsequently), but some people could prefer to do so daily or
not at all depending on the traffic they're dealing with.
Regards,
Nicolas.
This message contains information that may be privileged or confidential and is
the property of the Capgemini Group. It is intended only for the person to whom
it is addressed. If you are not the intended recipient, you are not authorized
to read, print, retain, copy, disseminate, distribute, or use this message or
any part thereof. If you receive this message in error, please notify the
sender immediately and delete all copies of this message.
------------------------------
Message: 2
Date: Fri, 12 Jun 2015 09:18:24 +0000
From: "Chaigneau, Nicolas" <[email protected]>
To: "Chaigneau, Nicolas" <[email protected]>, Kea Dev
List <[email protected]>
Subject: Re: [kea-dev] Suggestions for DHCP packets logs
Message-ID:
<ab94b0b675bdf14189cd5a861db36c841954e...@de-cm-mbx26.corp.capgemini.com>
Content-Type: text/plain; charset="us-ascii"
> WARNING: Kindly be aware the Sender Address on this mail may be forged. It
> appears to be from capgemini.com but the message has been received from a
> server outside Capgemini Group perimeter. User discretion is necessary before
> performing actions mentioned in this mail.
That's weird, I didn't have this message before.
Do you know if it is added by the mailing list, or by my company mail server ?
This message contains information that may be privileged or confidential and is
the property of the Capgemini Group. It is intended only for the person to whom
it is addressed. If you are not the intended recipient, you are not authorized
to read, print, retain, copy, disseminate, distribute, or use this message or
any part thereof. If you receive this message in error, please notify the
sender immediately and delete all copies of this message.
------------------------------
Message: 3
Date: Fri, 12 Jun 2015 09:21:06 +0000
From: Angelo Failla <[email protected]>
To: "Chaigneau, Nicolas" <[email protected]>, Kea Dev
List <[email protected]>
Subject: Re: [kea-dev] Suggestions for DHCP packets logs
Message-ID: <[email protected]>
Content-Type: text/plain; charset="utf-8"
From: Chaigneau, Nicolas
<[email protected]><mailto:[email protected]>
Date: June 12, 2015 at 10:11:42 AM
To: Kea Dev List <[email protected]>><mailto:[email protected]>
Subject: [kea-dev] Suggestions for DHCP packets logs
1) don't use log4cplus
There is a very significant performance penalty when logging a large volume of
data through log4cplus.
Packets should not be logged through log4cplus.
Packets should be written directly through a fstream. (I've measured about 30%
improvement in DORA/s capacity, as compared to using log4cplus).
can?t you use Google?s GLOG framework? it supports log rotation and other
things. it?s what C++ backend services at Facebook are using.
We don?t use the log rotation facility in GLOG, we just let applications write
to stderr/stdout and we use `runit` (http://smarden.org/runit/), which handles
application supervision and log file rotation via the `svlogd` daemon
(http://smarden.org/runit/svlogd.8.html). that?s how we also deployed KEA
internally.
-Angelo
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://lists.isc.org/pipermail/kea-dev/attachments/20150612/aa96280d/attachment-0001.html>
------------------------------
Message: 4
Date: Fri, 12 Jun 2015 11:13:34 +0000
From: Francis Dupont <[email protected]>
To: "Chaigneau, Nicolas" <[email protected]>
Cc: Kea Dev List <[email protected]>
Subject: Re: [kea-dev] Suggestions for DHCP packets logs
Message-ID: <[email protected]>
Content-Type: text/plain; charset="us-ascii"
BTW I am not against to save the packets in binary, i.e., in pcap or
any common/standardized formats that tools like tcpdump or wireshark
handle.
Regards
Francis Dupont <[email protected]>
> The following are recommendations for implementing the logging of
> DHCP packets.
=> 0. the term logging is not appropriate for the content for packets.
I suggest "dumping".
> 3) flushing of packets logs
=> direct (i.e., write vs fwrite) I/O?
> 4) handle packets logs rotation
=> as you suggest no intermediate process or even library this seems to
be clearly required.
> When logrotate works in copytruncate mode on the packets logs, the following
> happens while the log is being processed:
> - Kea CPU usage spikes to 100%
> - DHCP packets are being dropped (from the client point of view)
=> for the AFTR which is an one process tool I used a lot of small tricks
to avoid this kind of issue, i.e., never perform a "long" task without
slicing it with the packet handling. Note one can get the same issue
in a multi-process code when a critical lock is handled by a "long"
task...
Regards
[email protected]
------------------------------
_______________________________________________
kea-dev mailing list
[email protected]
https://lists.isc.org/mailman/listinfo/kea-dev
End of kea-dev Digest, Vol 15, Issue 3
**************************************
