> -----Original Message----- > From: [email protected] [mailto:rsyslog- > [email protected]] On Behalf Of [email protected] > Sent: Monday, October 01, 2012 6:40 PM > To: rsyslog-users > Subject: Re: [rsyslog] omoracle does not write all log events to > database > > On Mon, 1 Oct 2012, Damian Burloke wrote: > > > I have working rsyslog (v5.8.10) with simple omoracle configuration > pushing > > data to a single Oracle 11g database table. The configuration is > working > > but I'm a bit puzzled about the behaviour of the omoracle plugin > where I > > only ever see the second last log event written to the database table > > ...however, *all* log events (including last one) are written to > local > > messages file immediately > > > > In my testing, with "$OmoracleBatchSize" option set to lowest > possible > > value of "1" (I discovered setting to '0' is *not* a good idea), the > > omoracle plugin processes one log message at a time however the last > log > > event is not written to the database ...unless another arrives or > rsyslogd > > is shut down. In the latter case, all outstanding log events, > including > > the shut down messages, are written to the database. > > > > Is this (description above) the expected behaviour or should all log > > messaages be completely processed and written to database as they > arrive > > same as for local messages file? > > > > I had a look at the omoracle code but, sadly, my skills in that area > are > > poor. I have searched for information on the behaviour I'm seeing > but no > > luck finding any answers hence my decision to pose a question on this > list > > - any feedback greatly appreciated. > > the omoracle code was contributed, so it's hard to answer questions > about > why this is happening. > > But your expectations that it should write everything to the database > as > it arrives are what I would expect as well > > enabling batching should not change this as rsyslog does not wait for a > batch to be full before processing it, instead the batch is the maximum > number of messages it will pull off the message queue to process at > once.
IIRC, omoracle does it's own kind of batching, so it is disjunkt from rsyslog batching. Not sure, though. If omoracle does not work, would it make sense to try omlibdbi? In theory it should work with oracle, but I have not test environment at all to try this out. Rainer > > when you have larger batch sizes, and a high message volume, so you see > all but the last message? or all but the last message batch? > > It sounds almost like omoracle is not closing it's transaction until > the > next one starts. > > > how batches work in rsyslog: > > If you have a batch size of 100, but there is only one message in the > queue, rsyslog will process that one message and pass it on to the > output > module. > > The idea is that this reduces the latency in processing all messages as > much as possible, and if the overhead of a single message is > significant, > then the next cycle through the message queue will find a lot more > messages. > > > or, putting it another way, if it takes T=3 to process a batch of > messages, and a message arrives for every T > > it doesn't to > > T=1 1 message arrives, Q=1 > T=2 1 message arrives, Q=2 > T=3 1 message arrives, Q=3, start processing batch #1 of 3 messages > T=4 1 message arrives, Q=1, batch processing contines > T=5 1 message arrives, Q=2, batch processing finishes > T=6 1 message arrives, Q=3, start processing batch #2 of 3 messages > > instead it does > > T=1 1 message arrives, Q=1, start processing batch #1 of 1 message > T=2 1 message arrives, Q=1, batch processing contines > T=3 1 message arrives, Q=2, batch processing finishes > T=4 1 message arrives, Q=3, start processing batch #2 of 3 messages > > David Lang > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST > if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
