I was trying to make use of kafka redundancy
As Kafka implements a safe mechanisms (copying the message into multiple
nodes), I was wondering if theres a relayed-delivery mode
ie: rsyslog acks the message once kafka has ack'ed the message.
On 04/10/17 09:31, Rainer Gerhards wrote:
2017-10-04 9:29 GMT+02:00 mostolog--- via rsyslog <rsyslog@lists.adiscon.com>:
Thanks Rainer and David
About this: is there any option/planned feature to ensure message has been
delivered before marking it as processed?
eg: tell source message has been acknowledged when target has stored the
event
source--->relp--->rsyslog--->kafka
This would be "as easy as" not moving the reading cursor from files for
imfile status. (devil in details)
as David said, you can achieve this by using the "direct" queue mode.
Of course, performance is bad, but that's always the case if you go to
half-duplex operations.
HTH
Rainer
On 03/10/17 19:38, David Lang wrote:
and if the system crashes, or issues a kill -9 to rsyslog, there will not
be a chance to save the memory queue and some messages will be lost.
you would have to use a disk queue, or disable queues entirely (direct
mode) to avoid any chance of log loss. This will cripple your performance
(In a test several years ago, with a high-end PCI SSD, this resulted in a
1000x slowdown)
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
