On Fri, 22 May 2015, David Boles (dboles) wrote:
Okay, thanks for the explanation.
I suspect I'm mis-using rsyslog and that I will be better off piping messages
out omprog to a custom rewriter app and back in via a UNIX domain socket.
My original log messages come either from the Linux kernel via printk_emit or
in Lumberjack form via ul_syslog. Both of those systems let me add structured
data as key/value pairs. The rsyslog mmjsonparse finds the key/value data (as
does imkmsg) - but that data is itself a flattened representation of
hierarchical data. Neither printk_emit nor ul_syslog allow anything other than
key/value string pairs AFAIK.
try having it output a key as foo!bar, that's legal JSON as I understand it, but
I think rsyslog will interpret it and nest it accordingly.
I was trying to un-flatten that data during the rsyslog flow prior to putting
it into mongodb.
can you give an example to make it easier ot understand what you are trying to
do?
I very much understand the value in not creating a full-blown programming
language for the rsyslog config files. I would suggest that the use case I'm
describing will likely become more common over time. It might be possible to
extend mmjsonparse by adding a pair of new parameters. One would be a boolean
defaulting to "false" that indicates whether mmjsonparse should use the
extended semantics. The second would be an optional seperator specifier
defaulting to "!". The extended processing would expand the keys into the same
JSON hierarchy that we get when we define $!foo!bar in rsyslog.conf. That
would be powerful and require nothing more of the language.
I think your focus on mmjsonparse may be the problem, look at mmnormalize
(especially the items added in the last few weeks, including JSON parsing)
Effectively the daemon I intend to hang off of omprog will do this expansion
and then write it back to rsyslog. Does that strike anybody as wrong-headed? I
will ensure that there are no loops by gating off the message processing for
the altered stream to do nothing but write it out via ommongodb.
use a mm module, that takes the log item, manipulates it, and replaces the old
log item with the new version.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
