That would be an interesting feature, but we need someone who is willing to implement it. Maybe it's not that hard, as the driver already supports compression.
Rainer 2018-08-03 6:22 GMT+02:00 eSX via rsyslog <[email protected]>: > Hi rsyslog > I have been thinking about how to build a more robust failover mechanism. > > So Is it necessary to support a queue-compression in rsyslog? > > There are two major situations: > 1. the backend action suspend. > When backend(e.g. a database) suspend, action will dectect it. > A failover action will be executed if we configure > action.execOnlyWhenPreviousIsSuspended="on". > The failover target could be another redundant backend > or simply(finally) write it into file(omfile) directly. > In this case the queue won't full(If action.resumeRetryCount = "0"), > because of a failover action. > But the logs to failover action or files could be too big to keep disk > space free. > If the failover action is omfile, we can compress files with gzip > seting, to delay disk consumption. > But when disk is full, queue-size will increase too, if it's ongoing, > queue will full or discard message. > That's we don't want to occur. > > > 2. the backend action slowdown or "resume retry" actived. > When backend is just slowdown, but not suspend.There is no suspend > status detected. > Or resume(e.g action.resumeInterval = "30" > action.resumeRetryCount="-1") actived, worker thread will sleep for resume. > It means the worker thread will do nothing, until > resumeInterval condition reached. > In those case above, the queue-size could increase If consumption > speed is less than production speed. > But, we can do nothing until a queue-full occur. unless, the backend > is speedup as before. > A compression is necessary for queue, to delay a queue-full so that > make message lost > > I think it's necessary in some case, especially in a busy system. > Like virtual memory compression in OS(e.g. MacOS, Windows10), or the > compression support in File System(e.g. ZFS, btrfs) > A queue compression can be optional in many cases. > > Thanks > > -- > eSX > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
