On Mon, 1 Oct 2012, Damian Burloke wrote:
I have working rsyslog (v5.8.10) with simple omoracle configuration pushing
data to a single Oracle 11g database table. The configuration is working
but I'm a bit puzzled about the behaviour of the omoracle plugin where I
only ever see the second last log event written to the database table
...however, *all* log events (including last one) are written to local
messages file immediately
In my testing, with "$OmoracleBatchSize" option set to lowest possible
value of "1" (I discovered setting to '0' is *not* a good idea), the
omoracle plugin processes one log message at a time however the last log
event is not written to the database ...unless another arrives or rsyslogd
is shut down. In the latter case, all outstanding log events, including
the shut down messages, are written to the database.
Is this (description above) the expected behaviour or should all log
messaages be completely processed and written to database as they arrive
same as for local messages file?
I had a look at the omoracle code but, sadly, my skills in that area are
poor. I have searched for information on the behaviour I'm seeing but no
luck finding any answers hence my decision to pose a question on this list
- any feedback greatly appreciated.
the omoracle code was contributed, so it's hard to answer questions about
why this is happening.
But your expectations that it should write everything to the database as
it arrives are what I would expect as well
enabling batching should not change this as rsyslog does not wait for a
batch to be full before processing it, instead the batch is the maximum
number of messages it will pull off the message queue to process at once.
when you have larger batch sizes, and a high message volume, so you see
all but the last message? or all but the last message batch?
It sounds almost like omoracle is not closing it's transaction until the
next one starts.
how batches work in rsyslog:
If you have a batch size of 100, but there is only one message in the
queue, rsyslog will process that one message and pass it on to the output
module.
The idea is that this reduces the latency in processing all messages as
much as possible, and if the overhead of a single message is significant,
then the next cycle through the message queue will find a lot more
messages.
or, putting it another way, if it takes T=3 to process a batch of
messages, and a message arrives for every T
it doesn't to
T=1 1 message arrives, Q=1
T=2 1 message arrives, Q=2
T=3 1 message arrives, Q=3, start processing batch #1 of 3 messages
T=4 1 message arrives, Q=1, batch processing contines
T=5 1 message arrives, Q=2, batch processing finishes
T=6 1 message arrives, Q=3, start processing batch #2 of 3 messages
instead it does
T=1 1 message arrives, Q=1, start processing batch #1 of 1 message
T=2 1 message arrives, Q=1, batch processing contines
T=3 1 message arrives, Q=2, batch processing finishes
T=4 1 message arrives, Q=3, start processing batch #2 of 3 messages
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
