On Mon, 23 Jul 2012, Xavier Fustero wrote:
Hi,
first I want to clarify that my initial issue was fixed adding another
queue for the second templates. I read queue stuff quickly so I missed you
need a queue for every action. Thanks Rainer.
Second, let's discuss this queue a little bit. I replied in-line.
On 20 July 2012 18:15, <[email protected]> wrote:
On Fri, 20 Jul 2012, Rainer Gerhards wrote:
$ActionQueueType LinkedList # use asynchronous processing
$ActionQueueFileName srvrfwd # set file name, also enables disk mode
$ActionResumeRetryCount -1 # infinite retries on insert failure
$ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts
A question on this.
do you really need to use linkedlist to get async processing? or is it
just the fact that you have defined a queue type for the action?
To be honest I didn't pay attention on the different type of queues. I
tested it quickly to check if the queue system worked or not (it did). My
scenario is clients that uses two templates to send logs to a server. I
want having a queue in front of each template to avoid losing any message
in case something breaks in between ( I am using RELP as well). I am not
using queue to send data to a database.
After reading the doc more carefully, I read that In-memory queues don't
write in disk. It says they don't survice any program abort. As you know, I
am using Linkedlist mode (I didn't know it has process overheat). I did
some tests writing a loop to send 1000 messages and aborting the
communication in between by firewall, aborting rsyslog, stunnel in the
server to check that messages are delivered at the end. My surprise was to
see that my client writes messages in a file while it can't send them to
the server.
I am getting confused as I understood In-memory won't write anything in
disk but I see it creates a file with all messages queued in the working
directory. I though that will do a Disk-assisted queue. Can someone clarify
the difference between them?
I've seen linkedlist be significantly slower than the default memory array
(especially if there are lots of messages queued), so I'm wondering if this
is an option.
In case I choose a In-memory queue, do you suggest me to go for the
FixedArray? Is the only problem with this type that we can lose message in
case we accumulate more than N pending messages?
My question was primarily directed at Rainer, I don't know the answers.
My understanding is that both FixedArray and LinkedList are in-memory
queues, it's only if you add the ActionQueueFileName tag that it shifts to
being a disk assited queue and overflowing to disk.
My understanding is that even with LinkedList, you will first allocate the
max memory for the message queue in RAM before you start putting extra
messages on disk.
If my understanding is correct, you should be able to use FixedArray for a
disk assisted queue as well.
note that once you start overflowing to disk, your performance drops
significantly, so in that mode, the performance difference between
FixedArray and LinkedList no longer matters.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
