Hello, I'm brand new to Heka so my answers might not be really helpful. Anyway I'd be very interested to hear suggestions for your use case too.
On Mon, Nov 3, 2014 at 2:04 PM, Gerhard Häring <[email protected]> wrote: > Hello there, > > I'm just trying to set up a Heka log delivery solution as an alternative > to the Logstash based one I currently have. The web nodes are connected via > TcpOutput to es.my.domain, which points to the healthy hekad/ElasticSearch > receiver hosts. I tested wether the node's hekad will switch over to a > healthy host when i stop the "receiver" hekad. This works. > > es.my.domain > A es01.my.domain (active/inactive via route53 health check) > A es02.my.domain (active/inactive via route53 health check) > A es03.my.domain (active/inactive via route53 health check) > > es01.my.domain es02.my.domain es03.my.domain > > > > +--+--------------+-+ +--+--------------+-+ > +--+--------------+-+ > | +--------------+ | | +--------------+ | | +--------------+ > | > | |hekad TcpInput| | | |hekad TcpInput| | | |hekad TcpInput| > | > | |TLS | | | |TLS | | | |TLS | > | > | +------+-------+ | | +------+-------+ | | +------+-------+ > | > | | | | | | | | > | > | | | | | | | | > | > | +------v-------+ | | +------v-------+ | | +------v-------+ > | > | |ElasticSearch | | | |ElasticSearch | | | |ElasticSearch | > | > | +--------------+ | | +--------------+ | | +--------------+ > | > +--+--------------+-+ +--+--------------+-+ > +--+--------------+-+ > > > My question is if you have any other suggestions on how to build a setup > that is resilient to one of the ElasticSearch/hekad hosts going down. > > I know that using RabbitMQ via the amqp output is an option, but are there > any examples for a RabbitMQ connection string for more than one host? > Looking at the Heka code, it doesn't seem possible to configure a list of RabbitMQ servers so I guess you're left with configuring a proxy cluster with VIP in front of your RabbitMQ cluster. Another option would be to leverage Kafka but the input/output plugins [1] aren't available in the dev branch yet. HTH Simon [1] https://github.com/mozilla-services/heka/tree/kafka_output/plugins/kafka > > I'd also prefer to avoid having to set up a RabbitMQ cluster if possible. > > -- Gerhard > > _______________________________________________ > Heka mailing list > [email protected] > https://mail.mozilla.org/listinfo/heka > >
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
