> -----Original Message----- > From: [email protected] [mailto:rsyslog- > [email protected]] On Behalf Of [email protected] > Sent: Thursday, November 04, 2010 11:26 PM > To: rsyslog-users > Subject: Re: [rsyslog] One Queue for multiple Actions in 4.6.4? > > On Thu, 4 Nov 2010, Marc Schiffbauer wrote: > > >> On Thu, 4 Nov 2010, Marc Schiffbauer wrote: > >> > >>> Will the global MainMsgQueue maintain message order? > >> > >> I believe that it will, but keep in mind that the settings needed to > >> try > >> and maintain message order are very different than the settings that > >> you > >> need to use to get best performance. > >> > >> what sort of message rate are you talking about using? > >> > > > > About up to 3000 Messages/s > > at that data rate you will have a very hard time doing things the way > you > are trying. > > at the database level, doing 3000 inserts/sec requires _very_ high-end > hardware. rsyslog supports inserting multiple messages in one command, > and > with that you could probably handle 3000 messages/sec on very low-end > hardware (because rsyslog could insert messages in batches of 100 or > more > if needed). unfortunantly, when you enable this, you don't maintain the > message order. > > at the rsyslog level, doing disk based queues with everything tuned to > come as close to minimizing the chance of data loss is very hard to do > at > that traffic level. I ran tests last year on a 8 core box with 64G of > ram > and a fusion-io SSD pci card, and depending on the filesystem I used, I > was able to get from 2K to 8K messages/sec where I wasn't trying to do > anything more than write the message to a log file.
disk based queues *could* work much faster, but it requires a full rewrite. Some details already in the design paper, but no point in discussing that right now: I can't do this right at this time. Rainer _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
