Hey JB, Thank you very much for the update! I'm already thrilled about that one :-)
Kind regards, Andreas On Fri Nov 14 2014 at 7:17:04 PM Achim Nierbeck <bcanh...@googlemail.com> wrote: > Hi JB, > > awesome. Thanks for the feedback :-) > > regards, Achim > > 2014-11-14 16:40 GMT+01:00 Jean-Baptiste Onofré <j...@nanthrax.net>: > > > Hi guys, > > > > just a quick update about Karaf Decanter. > > > > The INFRA created the git repository. I will cleanup the legal files, and > > add the latest features. I will push to karaf-decanter later today. > > > > Regards > > JB > > > > > > On 10/19/2014 09:18 PM, Jean-Baptiste Onofré wrote: > > > >> Hi all, > >> > >> this vote passed with only +1. > >> > >> I will push my latest changes on the github, request the git repo to > >> INFRA (to push there and remove the github one), and create a component > >> in Jira. > >> > >> Thanks all for your vote. > >> > >> Regards > >> JB > >> > >> On 10/14/2014 05:12 PM, Jean-Baptiste Onofré wrote: > >> > >>> Hi all, > >>> > >>> First of all, sorry for this long e-mail ;) > >>> > >>> Some weeks ago, I blogged about the usage of ELK > >>> (Logstash/Elasticsearch/Kibana) with Karaf, Camel, ActiveMQ, etc to > >>> provide a monitoring dashboard (know what's happen in Karaf and be able > >>> to store it for a long period): > >>> > >>> http://blog.nanthrax.net/2014/03/apache-karaf-cellar-camel- > >>> activemq-monitoring-with-elk-elasticsearch-logstash-and-kibana/ > >>> > >>> > >>> > >>> If this solution works fine, there are some drawbacks: > >>> - it requires additional middlewares on the machines. Additionally to > >>> Karaf itself, we have to install logstash, elasticsearch nodes, and > >>> kibana console > >>> - it's not usable "out of the box": you need at least to configure > >>> logstash (with the different input/output plugins), kibana (to create > >>> the dashboard that you need) > >>> - it doesn't cover all the monitoring needs, especially in term of SLA: > >>> we want to be able to raise some alerts depending of some events (for > >>> instance, when a regex is match in the log messages, when a feature is > >>> uninstalled, when a JMX metric is greater than a given value, etc) > >>> > >>> Actually, Karaf (and related projects) already provides most (all) data > >>> required for the monitoring. However, it would be very helpful to have > a > >>> "glue", ready to use and more user friendly, including a storage of the > >>> metrics/monitoring data. > >>> > >>> Regarding this, I started a prototype of a monitoring solution for > Karaf > >>> and the applications running in Karaf. > >>> The purpose is to be very extendible, flexible, easy to install and > use. > >>> > >>> In term of architecture, we can find the following component: > >>> > >>> 1/ Collectors & SLA Policies > >>> The collectors are services responsible of harvesting monitoring data. > >>> We have two kinds of collectors: > >>> - the polling collectors are invoked by a scheduler periodically. > >>> - the event driven collectors react to some events. > >>> Two collectors are already available: > >>> - the JMX collector is a polling collector which harvest all MBeans > >>> attributes > >>> - the Log collector is a event driven collector, implementing a > >>> PaxAppender which react when a log message occurs > >>> We can planned the following collectors: > >>> - a Camel Tracer collector would be an event driven collector, acting > as > >>> a Camel Interceptor. It would allow to trace any Exchange in Camel. > >>> > >>> It's very dynamic (thanks to OSGi services), so it's possible to add a > >>> new custom collector (user/custom implementation). > >>> > >>> The Collectors are also responsible of checking the SLA. As the SLA > >>> policies are tight to the collected data, it makes sense that the > >>> collector validates the SLA and call/delegate the alert to SLA > services. > >>> > >>> 2/ Scheduler > >>> The scheduler service is responsible to call the Polling Collectors, > >>> gather the harvested data, and delegate to the dispatcher. > >>> We already have a simple scheduler (just a thread), but we can plan a > >>> quartz scheduler (for advanced cron/trigger configuration), and another > >>> one leveraging the Karaf scheduler. > >>> > >>> 3/ Dispatcher > >>> The dispatcher is called by the scheduler or the event driven > collectors > >>> to dispatch the collected data to the appenders. > >>> > >>> 4/ Appenders > >>> The appender services are responsible to send/store the collected data > >>> to target systems. > >>> For now, we have two appenders: > >>> - a log appender which just log the collected data > >>> - a elasticsearch appender which send the collected data to a > >>> elasticsearch instance. For now, it uses "external" elasticsearch, but > >>> I'm working on an elasticsearch feature allowing to embed elasticsearch > >>> in Karaf (it's mostly done). > >>> We can plan the following other appenders: > >>> - redis to send the collected data in Redis messaging system > >>> - jdbc to store the collected data in a database > >>> - jms to send the collected data to a JMS broker (like ActiveMQ) > >>> - camel to send the collected data to a Camel direct-vm/vm endpoint of > a > >>> route (it would create an internal route) > >>> > >>> 5/ Console/Kibana > >>> The console is composed by two parts: > >>> - a angularjs or bootstrap layer allowing to configure the SLA and > >>> global settings > >>> - embedded kibana instance with pre-configured dashboard (when the > >>> elasticsearch appender is used). We will have a set of already created > >>> lucene queries and a kind of "Karaf/Camel/ActiveMQ/CXF" dashboard > >>> template. The kibana instance will be embedded in Karaf (not external). > >>> > >>> Of course, we have ready to use features, allowing to very easily > >>> install modules that we want. > >>> > >>> I named the prototype Karaf Decanter. I don't have preference about the > >>> name, and the location of the code (it could be as Karaf subproject > like > >>> Cellar or Cave, or directly in the Karaf codebase). > >>> > >>> Thoughts ? > >>> > >>> Regards > >>> JB > >>> > >> > >> > > -- > > Jean-Baptiste Onofré > > jbono...@apache.org > > http://blog.nanthrax.net > > Talend - http://www.talend.com > > > > > > -- > > Apache Member > Apache Karaf <http://karaf.apache.org/> Committer & PMC > OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/> Committer & > Project Lead > blog <http://notizblog.nierbeck.de/> > Co-Author of Apache Karaf Cookbook <http://bit.ly/1ps9rkS> > > Software Architect / Project Manager / Scrum Master >
