Re: Does anyone use Log4j2 with Logstash?
There exists a full-fledged Java API for Elasticsearch. Still, I am not sure it's the best idea to lump all the intricacies of efficient indexing into ES into a log4j2 appender. Logstash does this well - there is a reason it is part of the Elastic family. /David On 23 November 2015 at 18:32, Matt Sicker wrote: > ElasticSearch has a REST API for adding data which seems to be a rather > inefficient way to implement an appender. > > On 19 November 2015 at 13:36, Matt Sicker wrote: > > > I just spoke with my friend who's pretty familiar with both Solr and > > ElasticSearch, and he points out that Solr usage might be a bit more > > convoluted due to how rigidly defined search indexes are compared to ES. > > It'd be like the difference between SQL and NoSQL. > > > > On 19 November 2015 at 12:28, Ralph Goers > > wrote: > > > >> The only problem I have is with the number of integrations we could do. > >> ElasticSearch and Solr are similar, and Solr is an Apache project, so I > >> would expect we would want to support both if we support either. > >> > >> Ralph > >> > >> > On Nov 19, 2015, at 10:07 AM, Gary Gregory > >> wrote: > >> > > >> > Amazon has a pretty rich ecosystem of stuff. Should this be in a new > >> > log4j-amazon module or in the core module? Does it fit in no-sql? > >> > > >> > Gary > >> > On Nov 19, 2015 6:29 AM, "Matt Sicker" wrote: > >> > > >> >> An ElasticSearchAppender does sound useful. We use Log4j2 in all our > >> >> projects, so we don't really need to add middleware unless absolutely > >> >> necessary. > >> >> > >> >> On 19 November 2015 at 07:35, Mikael Ståldal < > >> mikael.stal...@magine.com> > >> >> wrote: > >> >> > >> >>> At my previous work, we first used Log4j 2 and Graylog, and Logstash > >> in > >> >>> between. But then I developed GelfLayout (now part of log4j-core) > and > >> we > >> >>> could configure Log4j 2 to send log events directly to Graylog, and > >> >>> Logstash became unnecessary. > >> >>> > >> >>> Can we do something similar with ELK (so that it become just "E-K")? > >> Can > >> >> we > >> >>> develop an ElasticSearchAppender for Log4j 2? > >> >>> > >> >>> > >> >>> On Thu, Nov 19, 2015 at 1:07 PM, Mikael Ståldal < > >> >> mikael.stal...@magine.com > >> > >> >>> wrote: > >> >>> > >> Is is even necessary to add Logstash to the mix? I think that > Log4j 2 > >> should be able to whatever is necessary by itself without either > >> Flume > >> >> or > >> Logstash. > >> > >> > >> On Wed, Nov 18, 2015 at 10:56 PM, David KOCH > >> wrote: > >> > >> > We use log4j2's SyslogAppender and a logstash UDP input source, > >> works > >> >> ok > >> > for us, no need to add Flume to the mix. > >> > > >> > /David > >> > > >> > On 18 November 2015 at 22:35, Matt Sicker > wrote: > >> > > >> >> Any experience with this? I was thinking of using Flume and the > >> >> Flume > >> >> appender to route all my log messages to an ELK stack, but I was > >> > wondering > >> >> if there were other ways of doing this. > >> >> > >> >> -- > >> >> Matt Sicker > >> >> > >> > > >> > >> > >> > >> -- > >> [image: MagineTV] > >> > >> *Mikael Ståldal* > >> Senior software developer > >> > >> *Magine TV* > >> mikael.stal...@magine.com > >> Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com > >> > >> Privileged and/or Confidential Information may be contained in this > >> message. If you are not the addressee indicated in this message > >> (or responsible for delivery of the message to such a person), you > >> may > >> >>> not > >> copy or deliver this message to anyone. In such case, > >> you should destroy this message and kindly notify the sender by > reply > >> email. > >> > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> [image: MagineTV] > >> >>> > >> >>> *Mikael Ståldal* > >> >>> Senior software developer > >> >>> > >> >>> *Magine TV* > >> >>> mikael.stal...@magine.com > >> >>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com > >> >>> > >> >>> Privileged and/or Confidential Information may be contained in this > >> >>> message. If you are not the addressee indicated in this message > >> >>> (or responsible for delivery of the message to such a person), you > may > >> >> not > >> >>> copy or deliver this message to anyone. In such case, > >> >>> you should destroy this message and kindly notify the sender by > reply > >> >>> email. > >> >>> > >> >> > >> >> > >> >> > >> >> -- > >> >> Matt Sicker > >> >> > >> > >> > >> > >> - > >> To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org > >> For additional commands, e-mail: log4j-user-h...@logging.apache.org > >> > >> > > > > > > -- > > Matt Sicker > > > > > > -- > Matt Sicker >
Re: Does anyone use Log4j2 with Logstash?
ElasticSearch has a REST API for adding data which seems to be a rather inefficient way to implement an appender. On 19 November 2015 at 13:36, Matt Sicker wrote: > I just spoke with my friend who's pretty familiar with both Solr and > ElasticSearch, and he points out that Solr usage might be a bit more > convoluted due to how rigidly defined search indexes are compared to ES. > It'd be like the difference between SQL and NoSQL. > > On 19 November 2015 at 12:28, Ralph Goers > wrote: > >> The only problem I have is with the number of integrations we could do. >> ElasticSearch and Solr are similar, and Solr is an Apache project, so I >> would expect we would want to support both if we support either. >> >> Ralph >> >> > On Nov 19, 2015, at 10:07 AM, Gary Gregory >> wrote: >> > >> > Amazon has a pretty rich ecosystem of stuff. Should this be in a new >> > log4j-amazon module or in the core module? Does it fit in no-sql? >> > >> > Gary >> > On Nov 19, 2015 6:29 AM, "Matt Sicker" wrote: >> > >> >> An ElasticSearchAppender does sound useful. We use Log4j2 in all our >> >> projects, so we don't really need to add middleware unless absolutely >> >> necessary. >> >> >> >> On 19 November 2015 at 07:35, Mikael Ståldal < >> mikael.stal...@magine.com> >> >> wrote: >> >> >> >>> At my previous work, we first used Log4j 2 and Graylog, and Logstash >> in >> >>> between. But then I developed GelfLayout (now part of log4j-core) and >> we >> >>> could configure Log4j 2 to send log events directly to Graylog, and >> >>> Logstash became unnecessary. >> >>> >> >>> Can we do something similar with ELK (so that it become just "E-K")? >> Can >> >> we >> >>> develop an ElasticSearchAppender for Log4j 2? >> >>> >> >>> >> >>> On Thu, Nov 19, 2015 at 1:07 PM, Mikael Ståldal < >> >> mikael.stal...@magine.com >> >> >>> wrote: >> >>> >> Is is even necessary to add Logstash to the mix? I think that Log4j 2 >> should be able to whatever is necessary by itself without either >> Flume >> >> or >> Logstash. >> >> >> On Wed, Nov 18, 2015 at 10:56 PM, David KOCH >> wrote: >> >> > We use log4j2's SyslogAppender and a logstash UDP input source, >> works >> >> ok >> > for us, no need to add Flume to the mix. >> > >> > /David >> > >> > On 18 November 2015 at 22:35, Matt Sicker wrote: >> > >> >> Any experience with this? I was thinking of using Flume and the >> >> Flume >> >> appender to route all my log messages to an ELK stack, but I was >> > wondering >> >> if there were other ways of doing this. >> >> >> >> -- >> >> Matt Sicker >> >> >> > >> >> >> >> -- >> [image: MagineTV] >> >> *Mikael Ståldal* >> Senior software developer >> >> *Magine TV* >> mikael.stal...@magine.com >> Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com >> >> Privileged and/or Confidential Information may be contained in this >> message. If you are not the addressee indicated in this message >> (or responsible for delivery of the message to such a person), you >> may >> >>> not >> copy or deliver this message to anyone. In such case, >> you should destroy this message and kindly notify the sender by reply >> email. >> >> >>> >> >>> >> >>> >> >>> -- >> >>> [image: MagineTV] >> >>> >> >>> *Mikael Ståldal* >> >>> Senior software developer >> >>> >> >>> *Magine TV* >> >>> mikael.stal...@magine.com >> >>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com >> >>> >> >>> Privileged and/or Confidential Information may be contained in this >> >>> message. If you are not the addressee indicated in this message >> >>> (or responsible for delivery of the message to such a person), you may >> >> not >> >>> copy or deliver this message to anyone. In such case, >> >>> you should destroy this message and kindly notify the sender by reply >> >>> email. >> >>> >> >> >> >> >> >> >> >> -- >> >> Matt Sicker >> >> >> >> >> >> - >> To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org >> For additional commands, e-mail: log4j-user-h...@logging.apache.org >> >> > > > -- > Matt Sicker > -- Matt Sicker
Re: StatusLogger
Perhaps you can use the FailoverAppender for this? http://logging.apache.org/log4j/2.x/manual/appenders.html#FailoverAppender On Fri, Nov 20, 2015 at 6:55 PM, Nicholas Duane wrote: > We're attempting to capture error, or info, events that our plugins > raise. For instance, we wrote a domain sockets appender. If that domain > sockets appender has trouble connecting to the domain socket we'd like to > know about it. In addition, we'd like to know about it centrally so that > we don't have to monitor each of the boxes our code is running on. We > therefore have a "logging" appender which writes to an http endpoint. The > log messages our plugins emit will get forwarded to this logging appender > (via the configuration) in hopes to get these issues to a central > location. Of course if the http appender has trouble communicating with > the http endpoint there's not much we can report on that, though I guess we > could write to the StatusLogger at that point. > > I hope I explained it well enough so that you understand what it is we're > trying to do. > > Thanks, > Nick > > > Subject: Re: StatusLogger > > From: ralph.go...@dslextreme.com > > Date: Fri, 20 Nov 2015 10:16:17 -0700 > > To: log4j-user@logging.apache.org > > > > What do you mean by “capture the events from our appenders”? The > StatusLogger is primarily used during configuration or to log errors that > occur in the appender. If you are trying to capture the events being logged > that sounds a bit odd as that is the purpose of an appender. > > > > If you want to capture all the Log4j status logger output you can > specify a destination on the configuration element. The output will then be > written to that location instead of to stdout. > > > > Ralph > > > > > On Nov 20, 2015, at 8:01 AM, Nicholas Duane wrote: > > > > > > The code happens to be a log4j2 appender, so it sounds like you're > saying we should be using the StatusLogger, correct? The issue is that we > want to capture the events from our appenders to a central location. > > > > > > Thanks, > > > Nick > > > > > >> Subject: Re: StatusLogger > > >> From: ralph.go...@dslextreme.com > > >> Date: Thu, 19 Nov 2015 19:01:45 -0700 > > >> To: log4j-user@logging.apache.org > > >> > > >> Yes, the StatusLogger is how Log4j logs things that happen within > Log4j itself. If you are writing plugins for Log4j those should also use > the StatusLogger as they effectively become part of Log4j. If the are > regular application code then they should not use the StatusLogger. > > >> > > >> Although the StatusLogger uses the same API as the Log4j API its > implementation is quite different and much more limited in what can be done > with the output. > > >> > > >> The StatusLogger implementation doesn’t have Appenders. Instead it > has StatusListeners that receive the events. The only listeners provided > with Log4j are the StatusConsoleListener, which writes events to stdout or > a PrintStream, and StatusLoggerAdmin, which makes events available over JMX. > > >> > > >> Ralph > > >> > > >> > > >> > > >>> On Nov 19, 2015, at 6:33 PM, Nicholas Duane wrote: > > >>> > > >>> I'm trying to get information on the StatusLogger. I've searched > and so far the log4j docs say: > > >>> > > >>> "Records events that occur in the logging system." > > >>> > > >>> There are also a bunch of articles related to people having problems > with the StatusLogger. I'm just looking to find out what it is. It > appears it's somewhat of an "internal" logger that log4j (log4j2) uses to > log internal events. One reason I'm looking into this is because I see > some code in one of our projects in which the class is logging to the > StatusLogger. I assume we shouldn't be doing this. > > >>> > > >>> Is the StatusLogger used in log4j2? In one post I read that the > "status" attribute controls the level. Can I set the appender for the > StatusLogger? > > >>> > > >>> Thanks, > > >>> Nick > > >>> > > >> > > >> > > >> > > >> - > > >> To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org > > >> For additional commands, e-mail: log4j-user-h...@logging.apache.org > > >> > > > > > > > > > > > - > > To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org > > For additional commands, e-mail: log4j-user-h...@logging.apache.org > > > > -- [image: MagineTV] *Mikael Ståldal* Senior software developer *Magine TV* mikael.stal...@magine.com Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com Privileged and/or Confidential Information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such a person), you may not copy or deliver this message to anyone. In such case, you should destroy this message and kindly notify the sender by reply email.
