Rainer - so we are saying that if the plugin is configured to "connect" the zeromq socket, we will allow multiple instance mode, and if it is configured to "bind" the zeromq socket, we will restrict to single instance mode? This sounds reasonable to me.
This evening (GMT-5 evening) I'll add the configuration parameter for choosing between binding and connecting the socket to the new plugin code and get it up on github and we can go from there! BRian -----Original Message----- From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of Rainer Gerhards Sent: Monday, February 10, 2014 10:00 AM To: rsyslog-users Subject: Re: [rsyslog] Plugin Questions Folks, with Brian's permission I reply to our conversation on the rsyslog mailing list. Brian is updating the 0mq plugins to v8 and we are discussing some design issues. I think this is probably of interest for some others, so I asked that we carry on on the list. Context is provided in the mail text below... On Mon, Feb 10, 2014 at 12:52 PM, Brian Knox <taote...@gmail.com> wrote: > In ZeroMQ, the main thing binding on the output plugin would allow is > easier load balancing in some cases. > > Let's say you have an omzmq output on tcp://127.0.0.1:7777, that you > bind it, and that it is a ZMQ_PUSH socket. > > If I now connect say.. 4 ZMQ_PULL sockets to the bound output socket, > then ZeroMQ will round robin load balance across the four connections. > > So, if you have some process that does some work that has a > significant cost to rsyslog output.. you bind an output, and connect > multiple workers to it to load balance. > > However from what you are saying, this will now keep us from having > multiple worker instances on the output. > OK, I think I finally understand. Let's see if we can nail this down with a simple solution. First let me repeat how I see the situation: If you bind an output, than you expect that the output does not have a performance problem. The performance problem is on the "receivers". So in this case a single instance of omzmq is considered fast enough to push messages into the queue. If so, we could effectively work with a single action "worker instance", as described on slide 10 of this presentation: http://www.slideshare.net/rainergerhards1/the-rsyslog-v8-engine-developers-view It would just be that we do it because we want it that way and not because of we are unable to do it any other way. Even further, we could have a config parameter for the bind port. If given, we'd use a single "instance", if not, then multiple. IN ANY CASE this will only affect the omzmq code. Even in "single instance mode", the rsyslog core engine would do message filtering, formating, etc on multiple workers -- just like it did in v7. It's even a bit faster than in v7 because we have these optimized interface callbacks. If I am right with my assumption, I'd say that's the way to go. I would be happy to help with the plumbing for this "single instance mode", if you have you code on github or somewhere else I can pull it from, I'd happy to simply add it (so that you can review). Rainer > > It's not a huge deal.. it just means to do that sort of pattern I > would need an external zmq proxy that rsyslog connects to, and then > the proxy would be responsible for work distribution outside of > zeromq. I could write such a thing and release it open source (I'm > the original author and maintainer of the zproxy proxy code in czmq so I'm > very familiar with it). > > I have another idea for a solution that would work within the output > plugin itself, but it is more complex and I'd rather keep things > simple for the initial port to 8. > > My proposal at this point is: > > 1) I finish the new omzmq output for 8. It supports the latest > release of czmq. I will only allow "connects" to keep things simple. > I will add in support for CURVE authentication and encryption ( > http://rfc.zeromq.org/spec:26). > > 2) I create a new version of imzmq. The refactor will include code > cleanup, and reducing the complexity of the options so that imzmq only > supports "binds". I add in support on the input side for CURVE as well. > This lets use connect together rsyslog instances over zeromq, with > certificate based authentication and encryption. > > 3) The new plugins will be "imzmq" and "omzmq" rather than "imzmq3" and > "omzmq3". We can deprecate the 3 versions of the plugins. > > How's that sound? > Brian > > > > > On Mon, Feb 10, 2014 at 5:56 AM, Rainer Gerhards > <rgerha...@hq.adiscon.com > > wrote: > >> >> On Sun, Feb 9, 2014 at 9:23 PM, Brian Knox <taote...@gmail.com> wrote: >> >>> The reason I ask is - with zeromq, connections are stateless and >>> either side may "bind" or "connect". A transport address >>> (tcp://127.0.0.1:5556, ipc:///tmp/myipsock, etc) can be -connected- >>> to many times but only -bound- one time. This effects how things like fan >>> out an fan in patterns work. >>> >>> I would like to allow -binds- on the omzmq side. However, if each >>> thread spins up, creates a zeromq socket and tries to bind to the >>> same address, this won't work. >>> >>> >> I thought I was too tired yesterday to fully understand. >> Unfortunately, I still don't grasp in full what that means. What is >> the advantage of being "bound" as an output. I have sockets on my mind, >> maybe that's the problem. >> If I bind an output socket to a specific port, I can obviously have >> only one instance of that socket in the system. Is that what you want? >> >> We can of course ensure that each action will "activate" only one >> worker instance at a time, but is that what we want? That would mean >> we could not really us the multi-instance feature we introduduced. >> And so I conclude I misunderstand something ;) Can you help me get >> this straight so that I can provide some real advise on the plugin? >> >> Rainer >> >> >>> However - if this is the case - I can spin up one additional "proxy" >>> thread, that would -bind- to the address going out of rsyslog (say, >>> tcp:// >>> 127.0.0.1:5556) externally, and -bind- internally on zeromq's inproc >>> transport. Then, each worker thread that spins up connects to the >>> inproc side of the proxy. >>> >>> The main reason behind this is - this would allow me to add in >>> support for the curve based encryption and authentication in zeromq >>> 4. This lets us do key based auth + encryption between rsyslog instances >>> via zeromq. >>> >>> I have the output working now with connects - if I get a little time >>> this week I'll fiddle around with ideas for using the proxy thread >>> to still allow bound outputs. >>> >>> The 8 changes look great! >>> >>> Brian >>> >>> >>> On Sun, Feb 9, 2014 at 3:15 PM, Rainer Gerhards < >>> rgerha...@hq.adiscon.com> wrote: >>> >>>> In essence yes, but you need to init the settings in the >>>> createwrkrinstance entry points. The core does not know what you need. >>>> >>>> Sent from phone, thus brief. >>>> Am 09.02.2014 20:26 schrieb "Brian Knox" <taote...@gmail.com>: >>>> >>>> I have the new output plugin fully working. I have more >>>> configuration >>>>> options to add to it but the transition to the new api is up and >>>>> running at least. I have some fun questions now. >>>>> >>>>> Given this: >>>>> >>>>> >>>>> >>>>> typedef struct _instanceData { >>>>> zctx_t *ctx; /* zeromq context */ >>>>> const char *endPoint; /* zeromq endpoint */ >>>>> int zocketType; /* zeromq socket type */ >>>>> uchar *tplName; /* template name */ >>>>> } instanceData; >>>>> >>>>> typedef struct wrkrInstanceData { >>>>> instanceData *pData; >>>>> void *zocket; /* zeromq socket */ } wrkrInstanceData_t; >>>>> >>>>> I have my zeromq context in instance data - the context is thread >>>>> safe, so this should be fine. >>>>> >>>>> I have my zeromq socket in wrkrInstanceData - as the sockets must >>>>> not be shared across threads. >>>>> >>>>> Assuming this is correct - if I have a zeromq endpoint defined >>>>> (tcp:// >>>>> 127.0.0.1:5556) - and more than one worker thread starts - am I >>>>> correct in assuming that for each thread, I will get a new zeromq >>>>> socket, and each of these sockets will connect to the endpoint? >>>>> >>>>> >>>>> >>> >> > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
