On 25 Feb 2012, at 00:12, Devin Austin wrote: > Logging with RabbitMQ? Seems like a weird thing to do but I could be > wrong. I use Munin to keep track of stats.
Logging != stats, logging is a superset And you have N application servers, and you want to aggregate logs for them… Pushing your logs into a message queue gains you a crap tonne of stuff - you have automatic aggregation (as they're all in one place), and you can have N consumers - one writing to a file, one pushing messages into elastic search, one doing bulk-inserts into a DB for usage logging, you can trivially tail the logs for N servers in one command etc.. See also: http://logstash.net/ which should give you a nice idea of what you could do with this… Notably, I also emit a load of more structured logging from my applications - i.e. something like: $self->log('info', WebHit->new( uri => $c->req->uri, time_end => time(), time_taken => $req_time, memcache_hist => $req_memcache_hits, memcache_misses => $req_memcache_misses, … etc … ); When dumped to the console, these just stringify nicely (Log::Message::Structured::Stringify::Sprintf) The subset of messages which are specifically structured, as per above example (rather than just lines) then get pulled off RabbitMQ and stats come from them - so the same solution is nicely used for the stats collection and aggregation as well as general logging - by consuming a subset of the message queue (topic exchanges tfw!) I can draw graphs of memcache hit rate % aggregated by vhost and/or generally across all apps.. Cheers t0m _______________________________________________ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
