On Sun, 24 Jan 2016, Andre wrote:
David,
Thanks for the reply.
most of the time you have a fairly good idea of how many files you will be
creating at once, make the cachesize comfortably larger than the number
of files that you expect to be creating.
Indeed. That was my approach as well. Don't go ballistic but don't chose a
small number otherwise the results will be disappointing.
one common source of loss with udp is DNS lookups, try disabling DNS
(this will make fromhost be the same as fromhost-ip), and see what
happens.
In my case, it did not help as much as I needed. Don't ask me why but I
tried disabling DNS lookups and the results improved, but not enough to
counter the push back from the log users terrified from having to remember
IPs. :-)
In any case, I continued troubleshooting and ended up changing
DynaFileCacheSize and "oy vay"! What a difference!
a low size will result in a lot of system calls to close/open files,
which will
result in a lot of cpu time and low throughput.
the pstats output is going to be key to tracking down what is going on.
This is the great puzzle. When testing I see not a large CPU utilisation
and the throughput was good but packets were being lost at 1% rate (40k
EPS).
The CPU was so low that I simply concluded something like: nah, the CPU
usage is so low and UDP netstat counters so stable that I'm not sure if a
change to DynaFileCacheSize is warranted.... (Oopsie...)
it's making a huge number of syscalls, and flushing output to disk, so I guess
this would show as iowait time, not cpu time for the program.
As I mentioned, stats outside and inside rsyslog looked ok:
1. "netstat -anus" would display no increase in "packet receive errors"
while "packet received" seemed to match the expected results.
2. I could not see a change in discarded.full, discarded.nf within the
pstats output.
Wouldn't dropped UDP packets and discarded due to full buffers be discarded
and the counter of "packet receive errors" increased?
no, there is another queue you are not thinking of
the OS has it's UDP buffers, if things overflow those buffers, rsyslog never
knows the packet existed, so it can't account for them. There is a way in
netstat to show when the buffer was full, but it doesn't count as a recieve
error because UDP is allowed to be dropped if there is congestion.
David Lang
If that is the case, then I would conclude the packet loss occurred within
rsyslog but yet no sign of that was explicitly written on my pstats output.
Is that expected behaviour?
Cheers
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
