Ralph, could you elaborate on your response, please? AFAIK, Logstash and Filebeat provide guaranteed delivery, if configured correctly. As a matter of fact they have docs (here <https://www.elastic.co/guide/en/logstash/current/persistent-queues.html> and here <https://www.elastic.co/guide/en/beats/filebeat/current/how-filebeat-works.html#at-least-once-delivery>) explaining how to do it – actually, there are several ways on how to do it. What makes you think they don't provide guaranteed delivery?
I have implemented two different types of logging pipelines with guaranteed delivery: 1. Using a Google Cloud BigQuery appender 2. Using a Redis appender (Redis queue is ingested to Elasticsearch through Logstash) I want to learn where I can potentially violate the delivery guarantee. On Thu, Nov 30, 2023 at 5:54 AM Ralph Goers <ralph.go...@dslextreme.com> wrote: > Fluentbit, Fluentd, Logstash, and Filebeat are the main tools used for log > forwarding. While they all have some amount of plugability none of the are > as flexible as Flume. In addition, as I have mentioned before, none of them > provide guaranteed delivery so I would never recommend them for forwarding > audit logs. >
