Hi rsyslog
I have been thinking about how to build a more robust failover mechanism.
So Is it necessary to support a queue-compression in rsyslog?
There are two major situations:
1. the backend action suspend.
When backend(e.g. a database) suspend, action will dectect it.
A failover action will be executed if we configure
action.execOnlyWhenPreviousIsSuspended="on".
The failover target could be another redundant backend
or simply(finally) write it into file(omfile) directly.
In this case the queue won't full(If action.resumeRetryCount = "0"),
because of a failover action.
But the logs to failover action or files could be too big to keep disk
space free.
If the failover action is omfile, we can compress files with gzip
seting, to delay disk consumption.
But when disk is full, queue-size will increase too, if it's ongoing,
queue will full or discard message.
That's we don't want to occur.
2. the backend action slowdown or "resume retry" actived.
When backend is just slowdown, but not suspend.There is no suspend
status detected.
Or resume(e.g action.resumeInterval = "30"
action.resumeRetryCount="-1") actived, worker thread will sleep for resume.
It means the worker thread will do nothing, until
resumeInterval condition reached.
In those case above, the queue-size could increase If consumption
speed is less than production speed.
But, we can do nothing until a queue-full occur. unless, the backend
is speedup as before.
A compression is necessary for queue, to delay a queue-full so that
make message lost
I think it's necessary in some case, especially in a busy system.
Like virtual memory compression in OS(e.g. MacOS, Windows10), or the
compression support in File System(e.g. ZFS, btrfs)
A queue compression can be optional in many cases.
Thanks
--
eSX
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
