Re: [rsyslog] Best practice for an application to get structured data to rsyslog

2015-04-16 Thread Brett Delle Grazie
On 15 April 2015 at 20:16, David Lang da...@lang.hm wrote: This is why I love opensource software, comment on a need and someone will point at a tool. Thanks! wq On Wed, 15 Apr 2015, Brian Knox wrote: We keep our logs in JSON format and don't find it to be a drawback. We have logs

Re: [rsyslog] Best practice for an application to get structured data to rsyslog

2015-04-16 Thread Ezell, Matthew A.
On 4/15/15, 12:43 AM, David Lang da...@lang.hm wrote: look at liblogging, it was started as part of lumberjack for exactly this purpose. liblogging does not currently support structured logging. I've opened an issue in GitHub to track that: https://github.com/rsyslog/liblogging/issues/22 On

Re: [rsyslog] Best practice for an application to get structured data to rsyslog

2015-04-15 Thread Dave Caplinger
On Apr 14, 2015, at 11:43 PM, David Lang da...@lang.hm wrote: On Wed, 15 Apr 2015, Ezell, Matthew A. wrote: Hello- What is the current best practice for a portable application to get structured data to rsyslog? ... gets JSON printed to the log. That may be undesirable in the common

Re: [rsyslog] Best practice for an application to get structured data to rsyslog

2015-04-15 Thread Radu Gheorghe
On Wed, Apr 15, 2015 at 6:25 PM, Dave Caplinger davecaplin...@solutionary.com wrote: On Apr 14, 2015, at 11:43 PM, David Lang da...@lang.hm wrote: On Wed, 15 Apr 2015, Ezell, Matthew A. wrote: [...] what I do is to take whatever message was output and then run mmjsonparse against

Re: [rsyslog] Best practice for an application to get structured data to rsyslog

2015-04-15 Thread David Lang
To: rsyslog-users rsyslog@lists.adiscon.com Subject: Re: [rsyslog] Best practice for an application to get structured data to rsyslog On Wed, Apr 15, 2015 at 6:25 PM, Dave Caplinger davecaplin...@solutionary.com wrote: On Apr 14, 2015, at 11:43 PM, David Lang da...@lang.hm wrote: On Wed, 15 Apr

Re: [rsyslog] Best practice for an application to get structured data to rsyslog

2015-04-15 Thread Brian Knox
We keep our logs in JSON format and don't find it to be a drawback. We have logs searchable in elasticsearch - and for working with logs on disk, have a small program that logs can be piped through that strips out everything but the json which makes it very easy to pipe logs to jq (a command line

Re: [rsyslog] Best practice for an application to get structured data to rsyslog

2015-04-14 Thread singh.janmejay
Have you looked at mmjsonparse? It solves the problem of de-serializing structured-messages handed-over to rsyslog in JSON-serialized form. For dual-mode: structured and unstructured, 2 common approaches exist. - Passing structured messages as JSON and optionally handling the differently on

[rsyslog] Best practice for an application to get structured data to rsyslog

2015-04-14 Thread Ezell, Matthew A.
Hello- What is the current best practice for a portable application to get structured data to rsyslog? Most modern syslog daemons seem to support some type of JSON format, but applications still tend to use the old syslog(3) function for logging. If an application emits CEE JSON directly to

Re: [rsyslog] Best practice for an application to get structured data to rsyslog

2015-04-14 Thread Ezell, Matthew A.
Sure, as a system administrator it's pretty clear how best to handle this. If there's CEE JSON data coming over the wire, use mmjsonparse. If it's unstructured traditional syslog(3) data, use mmnormalize to try to extract relevant fields based on rules I setup. Write the traditional message

Re: [rsyslog] Best practice for an application to get structured data to rsyslog

2015-04-14 Thread David Lang
On Wed, 15 Apr 2015, Ezell, Matthew A. wrote: Hello- What is the current best practice for a portable application to get structured data to rsyslog? Most modern syslog daemons seem to support some type of JSON format, but applications still tend to use the old syslog(3) function for logging.

Re: [rsyslog] Best practice for an application to get structured data to rsyslog

2015-04-14 Thread David Lang
On Wed, 15 Apr 2015, Ezell, Matthew A. wrote: Sure, as a system administrator it's pretty clear how best to handle this. If there's CEE JSON data coming over the wire, use mmjsonparse. If it's unstructured traditional syslog(3) data, use mmnormalize to try to extract relevant fields based on