Re: Elasticsearch logging

[Magnus Bäck]

On Friday, January 02, 2015 at 19:20 CET,
 Jesse Redl  wrote:

> When reviewing the logs generated by elasticsearch (1.4.x), a single
> log message is being  split across multiple lines?

Yes, that's normal for Java logs.

> Is this configurable within logging.yml? My undstanding is that
> elasticsearch is powered by log4j however, I'm not familiar with
> this product, nor can I find any decent documentation on the
> logging.yml file.

I suspect the reason you're asking is that you're considering
ingesting the Elasticsearch logs with Logstash, in which case
you should look into the JSONEventLayoutV1 Log4j layout
(https://github.com/logstash/log4j-jsonevent-layout) to get the
logs in JSON format. You should be able to drop that jar file
along with its dependencies (net.minidev:json-smart:1.1.1 and
commons-lang:commons-lang:2.6) into the Elasticsearch lib directory
(typically /usr/share/elasticsearch/lib) and adjust logging.yml
to use that layout. See also
https://github.com/elasticsearch/elasticsearch/issues/8786.

-- 
Magnus Bäck| Software Engineer, Development Tools
magnus.b...@sonymobile.com | Sony Mobile Communications

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/20150105095131.GA12719%40seldlx20533.corpusers.net.
For more options, visit https://groups.google.com/d/optout.

Re: Elasticsearch logging

[Jesse Redl]

For reference here is the configuration we are currently using:

  file:
> type: rollingFile
> file: ${path.logs}/${cluster.name}.log
> maxBackupIndex: 1
> maxFileSize: "100MB"
> layout:
>   type: pattern
>   conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"


On Friday, January 2, 2015 12:20:32 PM UTC-6, Jesse Redl wrote:
>
> Hey All,
>
> When reviewing the logs generated by elasticsearch (1.4.x), a single log 
> message is being  split across multiple lines? 
>
> Is this configurable within logging.yml? My undstanding is that 
> elasticsearch is powered by log4j however, I'm not familiar with this 
> product, nor can I find any decent documentation on the logging.yml file.  
>
>
> Thanks all.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/e56ec7a0-ab0f-4152-8a67-52cd8c407bb9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elasticsearch logging

[Nikolas Everett]

Logging.yml is a funky wrapper around log4j.properties style log4j
configuration so that is why you don't see as much documentation on it.

Do you see log lines smashed together and cut apart randomly?  That'd be a
bug.

Its customary for logs to be single lines except for stack traces which
contain multiple lines but all after the first start indented.
Elasticsearch inserts line breaks when it logs large things like thr
cluster state to make scanning the logs visually easier.

You could play with logging.yml to reformat the logs (probably possible to
make them json or something) or you could assume new log events start when
the first part of the new line is the date stamp.

Nik
On Jan 2, 2015 1:20 PM, "Jesse Redl"  wrote:

> Hey All,
>
> When reviewing the logs generated by elasticsearch (1.4.x), a single log
> message is being  split across multiple lines?
>
> Is this configurable within logging.yml? My undstanding is that
> elasticsearch is powered by log4j however, I'm not familiar with this
> product, nor can I find any decent documentation on the logging.yml file.
>
>
> Thanks all.
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/3371-bb65-41fc-986a-bffc3c473bb7%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAPmjWd0srkXorAkY2vBf7aLGu-_wZ0dnG5LKL%2Bcwc3thhm85sw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Elasticsearch logging

[Jesse Redl]

Hey All,

When reviewing the logs generated by elasticsearch (1.4.x), a single log 
message is being  split across multiple lines? 

Is this configurable within logging.yml? My undstanding is that 
elasticsearch is powered by log4j however, I'm not familiar with this 
product, nor can I find any decent documentation on the logging.yml file.  


Thanks all.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/3371-bb65-41fc-986a-bffc3c473bb7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.