Re: [rsyslog] AMQP as log destination?

2012-12-07 Thread Fabio Sangiovanni

acronym alert, what does AMQP stand for?

It's a standard protocol to communicate with message queueing systems.
http://www.amqp.org/
http://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol

Message queueing system implementations that support AMQP: RabbitMQ, Apache QPid

persistent on-disk queues are already an option.

Yes, and I'd be happy to use them together with an AMQP output plugin for 
reliable massive log processing.

Fabio
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.


Re: [rsyslog] AMQP as log destination?

2012-12-07 Thread Fabio Sangiovanni

Il giorno 06/dic/2012, alle ore 20:47, Jerome Renard jerome.ren...@gmail.com 
ha scritto:

 Not AMQP, but maybe you will find omzmq3 useful [1]
 
 You can also use the omprog module [2] and from your program send logs
 to a RabbitMQ server (or anything else that support AMQP)
 
 'Hope that help :)
 
 1. 
 http://git.adiscon.com/?p=rsyslog.git;a=tree;f=plugins/omzmq3;h=6c9f8763a462af4756a6c4579dc3b27c82722b19;hb=HEAD
 2. http://www.rsyslog.com/doc/rsyslog_conf_modules.html/omprog.html
 


Hi,

unfortunately those are not options. We need to integrate with an AMQP system, 
and omprog module would mean too much perfomance loss and one more possibile 
point of failure.

Fabio

 -- 
 Jérôme
 ___
 rsyslog mailing list
 http://lists.adiscon.net/mailman/listinfo/rsyslog
 http://www.rsyslog.com/professional-services/
 What's up with rsyslog? Follow https://twitter.com/rgerhards
 NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
 sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
 LIKE THAT.

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.


Re: [rsyslog] AMQP as log destination?

2012-12-07 Thread Fabio Sangiovanni

Il giorno 06/dic/2012, alle ore 23:36, Radu Gheorghe radu0gheor...@gmail.com 
ha scritto:

 Hi Fabio,
 
 If you need AMPQ for integration with other apps, you can try Logstash with
 syslog input and AMPQ output.
 
 That said, Logstash needs AMPQ to have a persistency layer when shipping
 logs, since it has very limited queueing. As David pointed out, rsyslog has
 on disk and in memory queues which offer that out of the box. So if you
 need AMPQ for mass log shipping, I think you should consider using rsyslog
 directly. You can couple it directly to quite a lot of stuff, like
 Elasticsearch or MongoDB.
 

I know about logstash, but AMQP protocol is marked as unsupported; other than 
that, I'd really prefer not to use other software as relay, to keep the 
infrastructure as simple as possible. 

 And if you're looking for is missing (like AMPQ is), you can always develop
 input/output plugins or get some custom development from Adiscon to do that
 for you. I would assume this would be the good, clean solution in the long
 run.
 

That would be nice, I'll contact them :)

 Otherwise, like Jerome suggested, you can always hack a little script to do
 what you want and use omprog to pipe all logs to that script.
 
 Best regards,
 Radu
 

Fabio

___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.