Hello, ----- Original Message ----- > In > rsyslog-6.4.0/plugins/ommongodb/README I found the line: > > "Note: currently templates are not supported. Ommongodb will pick a default > schema and use the message object content for that (templateless)."
I happen to be currently working on this, see the ommongodb branch at http://fedorapeople.org/cgit/mitr/public_git/rsyslog.git/ . I'd be interested to hear whether the proposed approach works for you. More detailed description is below. Mirek The ommongodb branch is more experimental - I have very little knowledge of rsyslog and the related plans and architecture and I'd be grateful for any comments, and I'll happily rewrite it in a different way if you think that would make more sense. It revolves around OMSR_TPL_AS_FIELDS, where the core is responsible for splitting the message into a set of name-value pairs, and the output module only handles format conversion and storage (following doc/dev_oplugins.html insisting on not exposing all of the message to plugins, and perhaps even using the "field" mode as an intermediate step for slightly editing JSON and formatting it again in plaintext.) A sample template is: > $template > MongoTemplate,"%hostname::::sys%%timereported::::time%%timegenerated::::time_rcvd%%msg%%syslogfacility-text::::syslog_fac%%syslogseverity-text::::syslog_server%%syslogtag::::syslog_tag%%programname:::rocid%%procid::::pid%%$!foo::::foo%%$!all-json%%$!abc:::optional-field:renamed%" - only properties are used; it's not possible to add a constant field (this may need to be added?) - escaping is automatic / left to the output plugin and should not be specified with an option - "$all-json" expands inline to all libee fields - An "optional-field" flag can be used for fields that should disappear instead of being set to an empty string if the corresponding libee property is missing. My next planned step is to prohibit field name repetition, with a "first use wins" semantics. This will allow writing templates like this: > %trusted_value_not_overridable_by_all_json::::tagname% %$!all-json% > %$!all-json% %default_value_used_if_missing_in_all_json:::tagname% Some questions: - I this approach reasonable? The problem with this is that the "field" treatment of the template is so different from other cases; there is a precedent with omoracledb's use of ...AS_ARRAY, but that's only a single module. - Why is the error reporting in ommongodb #ifdef-ed out? - Note that the first template above uses "%$!foo::::foo", because the field name defaults to the property name, i.e. "$!foo". Would it make sense to strip the "$!" when creating the property name, so that one could simply use "%$!foo% to copy a single input field to output? - What to do about non-string values? mongodb recognizes different types, and it would be good to use the native one (so that numbers instead of strings could be compared - note that there is no automatic type conversion when comparing different types in mongodb). Non-string types can be handled on output (by adding "format-as-int"/"format-as-date" options), but AFAICS on input / mmjsonparse everything is treated as strings. Is it at this point realistic to think about preserving the type of data as presented on input while reformatting it (e.g. by using mmjsonparse, and a template with $!all-json and some of the above-mentioned field "edits")? Or is rsyslog so fundamentally based on strings that this would take too much work? (There is always the option to preserve types simply by treating the JSON as unmodified plaintext). Thank you, Mirek _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards
