Example i provided was from the head, don't know how ruby hashes look like. I don't know currently all incoming message formats. I think i will use more simple approach - rsyslog will just forward message as is to zmq server that will parse and transform message to appropriate format. That will be more simple to implement, yes it has extra overhead middle zmq server-parser but at the end this is just for legacy daemon that will migrate eventually. And besides all this endpoint message consumer expects message to be multipart with extra field in it.
Thanks all. 2012/8/29 Rainer Gerhards <[email protected]>: > > >> -----Original Message----- >> From: [email protected] [mailto:rsyslog- >> [email protected]] On Behalf Of David Kelly >> Sent: Wednesday, August 29, 2012 7:49 AM >> To: rsyslog-users >> Cc: rsyslog-users >> Subject: Re: [rsyslog] rsyslog + zmq >> >> I think you can use a parser, then a template associated with the >> output zmq plugin can format it in the json you want. >> >> We have been doing this quite a bit -- transforming the message between >> a zmq input and output plugin. That said, I'm not sure how to write the >> parser to parse ruby looking hashes with arbitrary keys - but I'll bet > > Is this a know standard format? Is it documented somewhere? IF so, I would > possibly (time permitting) be interested in writing a parser to get this over > to lumberjack format. > > Rainer >> others know more about that than I do. >> >> -d >> >> On Aug 28, 2012, at 1:53 AM, Evgeny Turnaev <[email protected]> >> wrote: >> >> > Thats great! >> > Thanks for answers, but how do i change message in the middle? >> > See a message sent to rsyslog will be application-structured itself. >> > Can i write a custom message parser to parse that message and change >> > it? >> > Simple example: If i got "foo => hello world" message in rsyslog i >> > want to send '{'foo' : 'hello world'}' into zmq. >> > >> > 2012/8/27 Rainer Gerhards <[email protected]>: >> >> >> >> >> >>> -----Original Message----- >> >>> From: [email protected] [mailto:rsyslog- >> >>> [email protected]] On Behalf Of Brian Knox >> >>> Sent: Monday, August 27, 2012 3:00 PM >> >>> To: rsyslog-users >> >>> Subject: Re: [rsyslog] rsyslog + zmq >> >>> >> >>> Oh nice! I can't wait to get 6.5 up in the lab. >> >> >> >> You'll probably like v7-devel (branch just created) even more ;) >> >> >> >> Rainer >> >>> >> >>> Brian >> >>> >> >>> On Mon, Aug 27, 2012 at 9:21 AM, Rainer Gerhards >> >>> <[email protected]>wrote: >> >>> >> >>>> >> >>>> >> >>>>> -----Original Message----- >> >>>>> From: [email protected] [mailto:rsyslog- >> >>>>> [email protected]] On Behalf Of Brian Knox >> >>>>> Sent: Monday, August 27, 2012 2:57 PM >> >>>>> To: rsyslog-users >> >>>>> Subject: Re: [rsyslog] rsyslog + zmq >> >>>>> >> >>>>> There is a new version of the zeromq rsyslog plugin that is >> >>> currently >> >>>>> included in the rsyslog source itself. It's on the head of the >> >>> master >> >>>>> branch - I don't know if it's been included in any 6.x beta >> >>> releases >> >>>>> yet. >> >>>> >> >>>> Not yet, but will be part of 6.5.0, which I hopefully release this >> >>> week. I >> >>>> am trying to fit in the new template stuff. >> >>>> >> >>>> Rainer >> >>>>> I highly recommend it over the code you linked, as there are a) a >> >>> lot >> >>>>> of >> >>>>> improvements over the first attempt we made and b) it's included >> in >> >>> the >> >>>>> rsyslog source itself as an official plugin now. The plugins are >> >>>>> "omzmq3" >> >>>>> and "imzmq3". >> >>>>> >> >>>>> The older plugins you linked to do not work with zeromq 3, do not >> >>> work >> >>>>> with >> >>>>> the new rsyslog 6 configuration system, and I know there are bugs >> >>> in >> >>>>> the >> >>>>> input module. >> >>>>> >> >>>>> Brian >> >>>>> >> >>>>> On Mon, Aug 27, 2012 at 8:32 AM, Evgeny Turnaev >> >>> <[email protected]> >> >>>>> wrote: >> >>>>> >> >>>>>> Hello. >> >>>>>> I have an intention to use zmq system as a message transport >> >>> layer >> >>>>>> for daemons that use syslog logging. >> >>>>>> The massage must also be modified before passing into zmq. >> >>>>>> Simplified flow: >> >>>>>> * 3rd parity old daemon calls syslog("123 Foo Bar") >> >>>>>> * rsyslog catches message parse it and produces json message >> >>> that >> >>>>>> get passed to >> >>>>>> * zmq output system >> >>>>>> >> >>>>>> I think i have 2 choices: >> >>>>>> 1) Use syslog daemon and write zmq plugin >> >>>>>> I googled a little and found >> >>>>>> https://github.com/aggregateknowledge/rsyslog-zeromq >> >>>>>> Is it going to be included in rsyslog as official plugin? >> >>>>>> Install how-to of this plugin mentioned patching rsyslog >> >>> and >> >>>>>> that worry me a little. >> >>>>>> Also in this scenario: how do i make fast message >> >>> modification? >> >>>>>> is it possible to do in output module code? >> >>>>>> >> >>>>>> 2) Write my own pseudo-syslog daemon that will receive >> >>> messages in >> >>>>>> syslog format and send changed messages into zmq system. >> >>>>>> As i know old legacy format is pretty simple but i dont >> >>> want >> >>>>> to >> >>>>>> write my own >> >>>>>> syslog format parser conforming to all rfc and broken >> >>> reality. >> >>>>>> Where do i look in rsyslog sources for syslog message >> >>> parser? >> >>>>>> Is it possible by rsyslog licence to use part of rsyslog code? >> >>> How do >> >>>>>> i integrate with my own code? Maybe there are others open source >> >>>>>> syslog message parsers? >> >>>>>> >> >>>>>> >> >>>>>> Any suggestions? >> >>>>>> >> >>>>>> -- >> >>>>>> -------------------------------------------- >> >>>>>> Турнаев Евгений Викторович >> >>>>>> +7 906 875 09 43 >> >>>>>> -------------------------------------------- >> >>>>>> _______________________________________________ >> >>>>>> rsyslog mailing list >> >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >>>>>> http://www.rsyslog.com/professional-services/ >> >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >> >>>>>> >> >>>>> _______________________________________________ >> >>>>> rsyslog mailing list >> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >>>>> http://www.rsyslog.com/professional-services/ >> >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >> >>>> _______________________________________________ >> >>>> rsyslog mailing list >> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >>>> http://www.rsyslog.com/professional-services/ >> >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >> >>>> >> >>> _______________________________________________ >> >>> rsyslog mailing list >> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >>> http://www.rsyslog.com/professional-services/ >> >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> http://www.rsyslog.com/professional-services/ >> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> > >> > >> > >> > -- >> > -------------------------------------------- >> > Турнаев Евгений Викторович >> > +7 906 875 09 43 >> > -------------------------------------------- >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards -- -------------------------------------------- Турнаев Евгений Викторович +7 906 875 09 43 -------------------------------------------- _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards
