Thanks for the suggestion, Dave. I noticed that on the client side the log contained Hindi characters that got translated to "\u00E0\u00.4�\" which eventually caused the error. I'll give mmutf8fix plugin a try.
Thanks, Alec On Tue, Jun 28, 2016 at 3:24 PM, Dave Caplinger < [email protected]> wrote: > On Jun 28, 2016, at 4:04 PM, Alec Swan <[email protected]> wrote: > > > > I think the root cause of the problem is that there is an invalid UTF-8 > > sequence "\u00.4" in the value if the "message" field. In fact, I just > > confirmed that {"message":"\u00.4"} is not a valid JSON on > > http://jsonlint.com/. > > I've run into something similar where the original message source was > sending Windows-1252 or other character set. Rsyslog doesn't know the > incoming character set, so it doesn't know that it needs to be converted to > UTF-8. (That particular input would receive logs from various sources, so > the character set could vary per message). > > The fix we used was to add action(type="mmutf8fix") to the affected > ruleset prior to any JSON template use. This isn't strictly accurate > because you lose the 'invalid' character in the resulting string, but at > least that string is JSON-safe. In the ideal case you'd know what the > original character set was and explicitly convert it UTF-8, but that wasn't > practical in our use case. > > -- > Dave Caplinger | Director, Technical Product Management > Solutionary — An NTT Group Security Company > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
