----- Ursprüngliche Mail -----
> oh, one thing I forgot: you mentioned disk queues in some mail. You
> can not
> use disk-assisted queues if you totally insist on ordering, because
> there is
> some inherent re-ordering possible when starting up disk mode. So you
> need to
> use pure disk queues or pure memory queues.
>
Hi Rainer,
I think if I set up matermarks and queue sizes in a way that disk-assisted
queues
will only use the disk files very seldom so that some single message re-order
events
when it starts using disk files could be tolerated.
I could use some catch-all mechanism to find such occurences.
Or will re-ordering take place all the time while it is using disk files?
But I have another question about $Rulsets:
When I have a ruleset with its own mainqueue and several actions within the
ruleset:
Where do I put the settings for that mainqueue?
$RuleSet dbAction
$RulesetCreateMainQueue on
# <--- is this the place to put ruleset mainqueue params ($ActionQueue...)?
# rulset action 1
if $syslogtag startswith
# rulset action 2
if $syslogtag startswith
#switch back to default ruleset
$ruleset RSYSLOG_DefaultRuleset
$ActionOmrulesetRulesetName dbAction
# <------------------------------------------- or here?
mail.* :omruleset:
> Rainer
>
> > -----Original Message-----
> > From: [email protected] [mailto:rsyslog-
> > [email protected]] On Behalf Of Marc Schiffbauer
> > Sent: Thursday, November 04, 2010 11:52 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] One Queue for multiple Actions in 4.6.4?
> >
> > > >
> > > > About up to 3000 Messages/s
> >
> > To be more precise: That will be the expected peak. So the average
> > will
> > be
> > much lower.
> >
> > And we have decent hardware to do this.
> >
> > >
> > > at that data rate you will have a very hard time doing things the
> > > way
> > > you
> > > are trying.
> > >
> > > at the database level, doing 3000 inserts/sec requires _very_
> > > high-
> > end
> > > hardware.
> > > rsyslog supports inserting multiple messages in one command,
> > > and
> > > with that you could probably handle 3000 messages/sec on very
> > > low-end
> > > hardware (because rsyslog could insert messages in batches of 100
> > > or
> > > more
> > > if needed). unfortunantly, when you enable this, you don't
> > > maintain
> > > the
> > > message order.
> > >
> > > at the rsyslog level, doing disk based queues with everything
> > > tuned
> > to
> > > come as close to minimizing the chance of data loss is very hard
> > > to
> > do
> > > at
> > > that traffic level. I ran tests last year on a 8 core box with 64G
> > > of
> > > ram
> > > and a fusion-io SSD pci card, and depending on the filesystem I
> > > used,
> > > I
> > > was able to get from 2K to 8K messages/sec where I wasn't trying
> > > to
> > do
> > > anything more than write the message to a log file.
> > >
> >
> > Thanks for the numbers.
> >
> > > you are going to have to think very hard about how critical it
> > > really
> > > is
> > > for you to maintain the same message order and what type of
> > > reliability
> > > you really need for your messages.
> > >
> > > it's counter-intuitive, but it may be that you end up with better
> > > overall
> > > reliability with a much faster configuration that has worse
> > > 'worst-case'
> > > data loss, but is fast enough that under normal conditions
> > > everything
> > > is
> > > processed really fast than you would under a configuration that is
> > > much
> > > slower all the time, but has a better worst-case data loss.
> >
> > Yes, I will keep that in mind, thanks.
> >
> >
> > >
> > > David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
