Hi David, thanks for your message and interest. This is very welcome as I would really like to get hold of more insight into rsyslog's performance in real-world and extreme scenarios. So far, I have unfortunately been unable to do that, because I had no funding for the required hardware and time to conduct the testing. I would *really* appreciate if you could help with that and I would be very willing to tweak the code for optimum performance (actually, I am always very concerned about high performance and I was a bit sad about the fact that I could not ensure it).
Having said that, I'd first of all would like to have a look at your rsyslog.conf, so that we are on common ground. Also, I think, this can become quite lengthy and also of interest for others. May it be useful to discuss this on the web forum, so that we can easier keep track of things. If you think this is useful, I recommend this one here: http://kb.monitorware.com/general-f34.html some more comments inline: On Mon, 2008-09-15 at 07:23 +0200, [EMAIL PROTECTED] wrote: > I've got a couple projects I'm working on that rsyslog is looking like > a > good answer to. I'm going to send questions about them in seperate > messages. > > the first project is a traditional syslog server. I've been testing > various syslog implementations to find out what sort of performance > they > can sustain. > > I've gotten the (almost) standard sysklogd to sustail almost 30,000 > messages/sec (udp) before it starts loosing significant numbers of > logs > > rsyslog is able to handle bursts of around 150,000 messages/sec, but > it > doesn't seem to write them out very fast, and over time seems to be > limited to ~22,000 messages/sec > > the hardware I am running this test on is insanely overpowered (four > dual-core opterons, 16G of ram, battery-backed cache on a raid card > with > high-speed SAS drives). in production I will be working with slower > hardware, but I'm trying to find the limits of the software before I > start > introducing lower hardware limits. > > what can I look at doing to speed up the writing of data from the > queues? > does rsyslog write the messages one at a time, or does it have an > option > for writing them in batches (pulling a bunch off of the queue and > sending > them all at once)? The design is that each individual message is pushed to the output. The output than writes the message. There has been discussion about a lazy write, but nothing in this regard has yet been implemented. Note that this is a good time to request new things. I am finished with the last major thing (TLS) support and in front of the next one (enhanced scripting), but scripting, I think, could be moved to a later time in favour of something more important (like performance ;)). > > some sort of batch mode would be critical for writes to a database > where > you can frequently get 1000x speedups if you do the inserts as a > single > transaction as opposed to individual transactions. I am not (any longer) so much a database guy. Can I really do a batch insert with them? Can you point me to an API? Looking forward to your reply! Rainer _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog
