Re: [systemd-devel] [RFC] [PATCH 0/3] journal: Add deferred log processing to reduce synchonous IO overhead

2013-12-16 Thread Karol Lewandowski
On 12/14/2013 05:12 PM, Zbigniew Jędrzejewski-Szmek wrote: On Fri, Dec 13, 2013 at 10:16:16PM +0100, Karol Lewandowski wrote: One of the problems I see, though, is that no matter how deep I make the queue (`max_dgram_qlen') I still see process sleeping on send() way earlier that configured

Re: [systemd-devel] [RFC] [PATCH 0/3] journal: Add deferred log processing to reduce synchonous IO overhead

2013-12-16 Thread Karol Lewandowski
On 12/14/2013 04:47 AM, Lennart Poettering wrote: On Fri, 13.12.13 22:16, Karol Lewandowski (lmc...@gmail.com) wrote: On Fri, Dec 13, 2013 at 03:45:36PM +0100, Lennart Poettering wrote: On Fri, 13.12.13 12:46, Karol Lewandowski (k.lewando...@samsung.com) wrote: One of the problems I see,

Re: [systemd-devel] [RFC] [PATCH 0/3] journal: Add deferred log processing to reduce synchonous IO overhead

2013-12-16 Thread Lennart Poettering
On Mon, 16.12.13 15:36, Karol Lewandowski (k.lewando...@samsung.com) wrote: On 12/14/2013 04:47 AM, Lennart Poettering wrote: On Fri, 13.12.13 22:16, Karol Lewandowski (lmc...@gmail.com) wrote: On Fri, Dec 13, 2013 at 03:45:36PM +0100, Lennart Poettering wrote: On Fri, 13.12.13 12:46,

Re: [systemd-devel] [RFC] [PATCH 0/3] journal: Add deferred log processing to reduce synchonous IO overhead

2013-12-16 Thread Lennart Poettering
On Mon, 16.12.13 16:54, Lennart Poettering (lenn...@poettering.net) wrote: That's it. While journal code tries to set buffer size via SO_SNDBUF/SO_RCVBUF options to 8MB, kernel limits these to wmem_max/rmem_max. On machines I've tested respective values are quite small - around

Re: [systemd-devel] [RFC] [PATCH 0/3] journal: Add deferred log processing to reduce synchonous IO overhead

2013-12-15 Thread David Timothy Strauss
On Sat, Dec 14, 2013 at 8:12 AM, Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl wrote: Are you sure that the sysctl is set early enough, before the listening socket is created? Perhaps this is why your suggestion [1] for the journal bootup issue didn't help. [1]

Re: [systemd-devel] [RFC] [PATCH 0/3] journal: Add deferred log processing to reduce synchonous IO overhead

2013-12-14 Thread Zbigniew Jędrzejewski-Szmek
On Fri, Dec 13, 2013 at 10:16:16PM +0100, Karol Lewandowski wrote: One of the problems I see, though, is that no matter how deep I make the queue (`max_dgram_qlen') I still see process sleeping on send() way earlier that configured queue depth would suggest. Are you sure that the sysctl is set

Re: [systemd-devel] [RFC] [PATCH 0/3] journal: Add deferred log processing to reduce synchonous IO overhead

2013-12-13 Thread Karol Lewandowski
On 12/12/2013 07:05 PM, Lennart Poettering wrote: On Thu, 12.12.13 17:42, Karol Lewandowski (lmc...@gmail.com) wrote: Hmm, but this wouldn't be any different from old syslog, right? I mean, old syslog also uses sendv() and recv() on AF_UNIX SOCK_DGRAM sockets... That's true, however, we

Re: [systemd-devel] [RFC] [PATCH 0/3] journal: Add deferred log processing to reduce synchonous IO overhead

2013-12-13 Thread Lennart Poettering
On Fri, 13.12.13 12:46, Karol Lewandowski (k.lewando...@samsung.com) wrote: From 2.6.29 it's in mailine kernel under drivers/staging/android/logger.c But if the the socket layer is really that much slower than android logging, then that's kinda sad, and the kernel socket stuff

Re: [systemd-devel] [RFC] [PATCH 0/3] journal: Add deferred log processing to reduce synchonous IO overhead

2013-12-13 Thread Karol Lewandowski
On Fri, Dec 13, 2013 at 03:45:36PM +0100, Lennart Poettering wrote: On Fri, 13.12.13 12:46, Karol Lewandowski (k.lewando...@samsung.com) wrote: Well, are you suggesting that the AF_UNIX/SOCK_DGRAM code actually hands off the timeslice to the other side as soon as it queued something in? If by

[systemd-devel] [RFC] [PATCH 0/3] journal: Add deferred log processing to reduce synchonous IO overhead

2013-12-12 Thread Karol Lewandowski
Folks, We are trying to apply journald as default (and only) logging system for our ARM-based handhelds. There is one major problem, though - performance. We have found that journald can consume considerable amount of CPU time under moderate load (10-20% of CPU per, for say 100-300msg/s). We

Re: [systemd-devel] [RFC] [PATCH 0/3] journal: Add deferred log processing to reduce synchonous IO overhead

2013-12-12 Thread Lennart Poettering
On Thu, 12.12.13 14:12, Karol Lewandowski (k.lewando...@samsung.com) wrote: Folks, We are trying to apply journald as default (and only) logging system for our ARM-based handhelds. There is one major problem, though - performance. We have found that journald can consume considerable

Re: [systemd-devel] [RFC] [PATCH 0/3] journal: Add deferred log processing to reduce synchonous IO overhead

2013-12-12 Thread Karol Lewandowski
On Thu, Dec 12, 2013 at 03:51:05PM +0100, Lennart Poettering wrote: On Thu, 12.12.13 14:12, Karol Lewandowski (k.lewando...@samsung.com) wrote: (OP here from private email.) Folks, We are trying to apply journald as default (and only) logging system for our ARM-based handhelds. There

Re: [systemd-devel] [RFC] [PATCH 0/3] journal: Add deferred log processing to reduce synchonous IO overhead

2013-12-12 Thread Lennart Poettering
On Thu, 12.12.13 17:42, Karol Lewandowski (lmc...@gmail.com) wrote: Hmm, but this wouldn't be any different from old syslog, right? I mean, old syslog also uses sendv() and recv() on AF_UNIX SOCK_DGRAM sockets... That's true, however, we aren't migrating from syslog, but from Android