Re: [zeromq-dev] Reliable pub sub on c++ linux
Do you have the code where you set the hwms? On 27 Apr 2015 21:08, "Peter Krey" wrote: > I have HWM set to zero on recv and pub. I am keeping track of sequence > numbers recved on the sub socket which are sent out by the pub socket. Here > is an example output. > > The pub socket is publishing a uint64_t seqNumber. If i change the socket > types to pair, no seqNumbers are ever missed. > > > seqNumber missed 2301000 > seqNumber missed 2303206 > seqNumber missed 2305000 > seqNumber missed 2306820 > seqNumber missed 2309353 > seqNumber missed 2311575 > seqNumber missed 2314514 > seqNumber missed 2316767 > seqNumber missed 2318000 > seqNumber missed 2319924 > seqNumber missed 2321730 > seqNumber missed 2323618 > seqNumber missed 2325000 > seqNumber missed 2326963 > seqNumber missed 2329000 > seqNumber missed 2330664 > seqNumber missed 2333000 > seqNumber missed 2334997 > seqNumber missed 2336000 > seqNumber missed 2338000 > seqNumber missed 234 > seqNumber missed 2343000 > seqNumber missed 2344933 > seqNumber missed 2346401 > seqNumber missed 2349000 > seqNumber missed 2351000 > seqNumber missed 2352309 > seqNumber missed 2354198 > seqNumber missed 2356000 > seqNumber missed 2357645 > > On Mon, Apr 27, 2015 at 12:56 PM, Pieter Hintjens wrote: > >> You can increase the HWM on sender and receiver to match your >> expectations. >> >> If you set the HWM to zero there will never be any message loss, which >> also means your publisher will explode if the subscriber stops >> reading. >> >> >> >> On Mon, Apr 27, 2015 at 9:03 PM, Peter Krey wrote: >> > Hi, >> > >> > What is the best way to get guaranteed in order delivery over pub-sub >> > framework in zmq using c++ on linux? >> > >> > I have a test server and client running zmq pub and sub sockets. The pub >> > pushes sequence numbers as fast as possible in a tight loop. The sub >> socket >> > misses around one in every 10k messages. >> > >> > Thanks >> > >> > ___ >> > zeromq-dev mailing list >> > zeromq-dev@lists.zeromq.org >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> > >> ___ >> zeromq-dev mailing list >> zeromq-dev@lists.zeromq.org >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> > > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zproto and Malamute/Majordomo
Lucas, It may well be that code generation is a good choice for solving your problem but zproto (one specific set of code generators) is not. You can always use GSL to write your own code generator scripts that are verbose and descriptive. Codecs generated by zproto are not all verbose or descriptive. On Mon, Apr 27, 2015 at 7:54 PM, Lucas Russo wrote: > Pieter Hintjens imatix.com> writes: > > > > > My personal view on this is that zproto is ideal for underlying > > protocols, whereas application protocols (such as you'd carry over > > Malamute) should use a cheaper text format. > > > > This is the Cheap and Nasty discussion in the Guide. Zproto being the > 'nasty'. > > > > On Mon, Apr 27, 2015 at 7:13 PM, Pieter Hintjens imatix.com> > wrote: > > > It's been done, and removed from the basic codec generator. You can > > > see how this was done in zproto_codec_c_v1.gsl. > > > > > > There has been talk of adding this encode/decode support back in to the > C codec. > > > > > > On Mon, Apr 27, 2015 at 3:56 PM, Lucas Russo gmail.com> > wrote: > > >> Hello everybody, > > >> > > >> I'm taking a look at zproto project, in order to better structurize > and > > >> document an application protocol. > > >> > > >> However, is there a way to use zproto on top of, for instance, > malamute or > > >> majordomo API? Meaning defining my application protocol with zproto, > but > > >> sending/receiving messages using malamute or majordomo? > > >> > > >> As far as I understood zproto uses the regular czmq API to send/recv > > >> messages, but malamute and majordomo both have different APIs to > send/recv, > > >> like mlm_sendx () and mdp_client_send (). > > >> > > >> Regards, > > >> > > >> Lucas > > >> > > >> > > >> > > >> ___ > > >> zeromq-dev mailing list > > >> zeromq-dev lists.zeromq.org > > >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > >> > > > > Hi Pieter, Thanks for your answer, > > I see. But without zproto support for this kind of thing, wouldn't my only > option be to document, structurize and serialize the message myself? > > Also, from the guide, the 'cheap' part of the pattern should be > "synchronous, verbose, descriptive, and flexible". Isn't this a good use > case for a tool to generate codecs for us automatically? So, we would end > up having a very descriptive and verbose protocol? > > Maybe I misunderstood something, but zproto seemed like a suitable choice > to my needs. Is there something very flawed in this reasoning? > > Thanks and Regards, > > Lucas > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zproto and Malamute/Majordomo
Pieter Hintjens imatix.com> writes: > > My personal view on this is that zproto is ideal for underlying > protocols, whereas application protocols (such as you'd carry over > Malamute) should use a cheaper text format. > > This is the Cheap and Nasty discussion in the Guide. Zproto being the 'nasty'. > > On Mon, Apr 27, 2015 at 7:13 PM, Pieter Hintjens imatix.com> wrote: > > It's been done, and removed from the basic codec generator. You can > > see how this was done in zproto_codec_c_v1.gsl. > > > > There has been talk of adding this encode/decode support back in to the C codec. > > > > On Mon, Apr 27, 2015 at 3:56 PM, Lucas Russo gmail.com> wrote: > >> Hello everybody, > >> > >> I'm taking a look at zproto project, in order to better structurize and > >> document an application protocol. > >> > >> However, is there a way to use zproto on top of, for instance, malamute or > >> majordomo API? Meaning defining my application protocol with zproto, but > >> sending/receiving messages using malamute or majordomo? > >> > >> As far as I understood zproto uses the regular czmq API to send/recv > >> messages, but malamute and majordomo both have different APIs to send/recv, > >> like mlm_sendx () and mdp_client_send (). > >> > >> Regards, > >> > >> Lucas > >> > >> > >> > >> ___ > >> zeromq-dev mailing list > >> zeromq-dev lists.zeromq.org > >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >> > Hi Pieter, Thanks for your answer, I see. But without zproto support for this kind of thing, wouldn't my only option be to document, structurize and serialize the message myself? Also, from the guide, the 'cheap' part of the pattern should be "synchronous, verbose, descriptive, and flexible". Isn't this a good use case for a tool to generate codecs for us automatically? So, we would end up having a very descriptive and verbose protocol? Maybe I misunderstood something, but zproto seemed like a suitable choice to my needs. Is there something very flawed in this reasoning? Thanks and Regards, Lucas ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Reliable pub sub on c++ linux
I have HWM set to zero on recv and pub. I am keeping track of sequence numbers recved on the sub socket which are sent out by the pub socket. Here is an example output. The pub socket is publishing a uint64_t seqNumber. If i change the socket types to pair, no seqNumbers are ever missed. seqNumber missed 2301000 seqNumber missed 2303206 seqNumber missed 2305000 seqNumber missed 2306820 seqNumber missed 2309353 seqNumber missed 2311575 seqNumber missed 2314514 seqNumber missed 2316767 seqNumber missed 2318000 seqNumber missed 2319924 seqNumber missed 2321730 seqNumber missed 2323618 seqNumber missed 2325000 seqNumber missed 2326963 seqNumber missed 2329000 seqNumber missed 2330664 seqNumber missed 2333000 seqNumber missed 2334997 seqNumber missed 2336000 seqNumber missed 2338000 seqNumber missed 234 seqNumber missed 2343000 seqNumber missed 2344933 seqNumber missed 2346401 seqNumber missed 2349000 seqNumber missed 2351000 seqNumber missed 2352309 seqNumber missed 2354198 seqNumber missed 2356000 seqNumber missed 2357645 On Mon, Apr 27, 2015 at 12:56 PM, Pieter Hintjens wrote: > You can increase the HWM on sender and receiver to match your expectations. > > If you set the HWM to zero there will never be any message loss, which > also means your publisher will explode if the subscriber stops > reading. > > > > On Mon, Apr 27, 2015 at 9:03 PM, Peter Krey wrote: > > Hi, > > > > What is the best way to get guaranteed in order delivery over pub-sub > > framework in zmq using c++ on linux? > > > > I have a test server and client running zmq pub and sub sockets. The pub > > pushes sequence numbers as fast as possible in a tight loop. The sub > socket > > misses around one in every 10k messages. > > > > Thanks > > > > ___ > > zeromq-dev mailing list > > zeromq-dev@lists.zeromq.org > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Reliable pub sub on c++ linux
You can increase the HWM on sender and receiver to match your expectations. If you set the HWM to zero there will never be any message loss, which also means your publisher will explode if the subscriber stops reading. On Mon, Apr 27, 2015 at 9:03 PM, Peter Krey wrote: > Hi, > > What is the best way to get guaranteed in order delivery over pub-sub > framework in zmq using c++ on linux? > > I have a test server and client running zmq pub and sub sockets. The pub > pushes sequence numbers as fast as possible in a tight loop. The sub socket > misses around one in every 10k messages. > > Thanks > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Paranoid Pirate pattern with priority queuing
If your requirement is to allow urgent traffic you can use multiple sockets connected to a single endpoint. If you want to explicitly stop lower-priority traffic, you have to have fully independent reading sockets, and poll them in order of priority. On Mon, Apr 27, 2015 at 8:46 PM, James & Veri wrote: > ok, absent that feature, does the approach I described for prioritized PPP > sound about right? > >> From: p...@imatix.com >> Date: Mon, 27 Apr 2015 19:11:54 +0200 >> To: zeromq-dev@lists.zeromq.org >> Subject: Re: [zeromq-dev] Paranoid Pirate pattern with priority queuing >> >> We moved the resource functionality to RFC 37 (draft), as it didn't >> get into ZeroMQ 4.0. >> > > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZBeacon multiple interface support
Keep in mind that on Linux an IP address is assigned to a host, not an interface. Even if you think you're adding an IP address "to an interface," an ARP request for that IP address will by default be responded to on all interfaces. There is a way to override this behavior -- refer to the arp_ignore documentation in Documentation/networking/ip-sysctl.txt in the kernel source. Patrick On Saturday, April 11, 2015 02:03:28 PM Pieter Hintjens wrote: > On Fri, Apr 10, 2015 at 10:21 AM, Arnaud Loonstra wrote: > > > I'm not sure if I get what you mean but I think there exist many > > networking solutions which support multiple interfaces. > > It's fairly simply to listen on multiple interfaces. That's not the > hard thing. You can open multiple sockets and read each one, or run > multiple instances of the beacon actor. However when you have two > networks and you want to run Zyre then you have two groups of nodes > that don't see each other unless every single machine is listening on > both interfaces. > > E.g. A, B, C, where A and C are on different networks and B is on > both. A and C can't see each other, whereas B can see A and C. So B > has to forward messages between A and C. > > If you solve this issue then you can force interface selection > manually as we do now ZSYS_INTERFACE. > > -Pieter > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Reliable pub sub on c++ linux
Hi, What is the best way to get guaranteed in order delivery over pub-sub framework in zmq using c++ on linux? I have a test server and client running zmq pub and sub sockets. The pub pushes sequence numbers as fast as possible in a tight loop. The sub socket misses around one in every 10k messages. Thanks ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Paranoid Pirate pattern with priority queuing
ok, absent that feature, does the approach I described for prioritized PPP sound about right? > From: p...@imatix.com > Date: Mon, 27 Apr 2015 19:11:54 +0200 > To: zeromq-dev@lists.zeromq.org > Subject: Re: [zeromq-dev] Paranoid Pirate pattern with priority queuing > > We moved the resource functionality to RFC 37 (draft), as it didn't > get into ZeroMQ 4.0. > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zproto and Malamute/Majordomo
My personal view on this is that zproto is ideal for underlying protocols, whereas application protocols (such as you'd carry over Malamute) should use a cheaper text format. This is the Cheap and Nasty discussion in the Guide. Zproto being the 'nasty'. On Mon, Apr 27, 2015 at 7:13 PM, Pieter Hintjens wrote: > It's been done, and removed from the basic codec generator. You can > see how this was done in zproto_codec_c_v1.gsl. > > There has been talk of adding this encode/decode support back in to the C > codec. > > On Mon, Apr 27, 2015 at 3:56 PM, Lucas Russo wrote: >> Hello everybody, >> >> I'm taking a look at zproto project, in order to better structurize and >> document an application protocol. >> >> However, is there a way to use zproto on top of, for instance, malamute or >> majordomo API? Meaning defining my application protocol with zproto, but >> sending/receiving messages using malamute or majordomo? >> >> As far as I understood zproto uses the regular czmq API to send/recv >> messages, but malamute and majordomo both have different APIs to send/recv, >> like mlm_sendx () and mdp_client_send (). >> >> Regards, >> >> Lucas >> >> >> >> ___ >> zeromq-dev mailing list >> zeromq-dev@lists.zeromq.org >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zproto and Malamute/Majordomo
It's been done, and removed from the basic codec generator. You can see how this was done in zproto_codec_c_v1.gsl. There has been talk of adding this encode/decode support back in to the C codec. On Mon, Apr 27, 2015 at 3:56 PM, Lucas Russo wrote: > Hello everybody, > > I'm taking a look at zproto project, in order to better structurize and > document an application protocol. > > However, is there a way to use zproto on top of, for instance, malamute or > majordomo API? Meaning defining my application protocol with zproto, but > sending/receiving messages using malamute or majordomo? > > As far as I understood zproto uses the regular czmq API to send/recv > messages, but malamute and majordomo both have different APIs to send/recv, > like mlm_sendx () and mdp_client_send (). > > Regards, > > Lucas > > > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Paranoid Pirate pattern with priority queuing
We moved the resource functionality to RFC 37 (draft), as it didn't get into ZeroMQ 4.0. On Mon, Apr 27, 2015 at 3:29 PM, James & Veri wrote: > Assume you have a requirement to support reliable priority queues whereby, > if there is ever a message sent by a client with a higher priority (say 3 > queues, high/medium/low) workers will always see those requests before they > see lower priority requests. I'm currently working with jeromq and netmq, so > the solution should be based on zmq 3.2.5. > > I have thought of doing this in the proxy with an endpoint per priority on > the front end and one endpoint for the back end. I'm thinking the > prioritization would come from the proxy by organizing its poller's inputs > such that each iteration through its polling loop checks for input in > descending order from highest to lowest and delivers from the first frontend > socket available for reading. Does this sound right? I assume the logic at > the top of the poller loop still hold (only poll for input on all of the > priority endpoints if there's at least one 'ready' worker), right? > > Is this the simplest approach? It seems that it gets pretty port-hungry at > the proxy as the number of such prioritized queues grows ( (# of priorities > + 1) x (# of distinct prioritized queues) ). I'm thinking one poller thread > per distinct prioritized queue. > > Also, I seem to remember somewhere that versions of zeromq supporting zmtp > 4.0 were expected to alleviate this by supporting ipc/tcp endpoints that can > specify different resources at the same endpoint, ala > tcp://*:/queue-name/priority-value. Looking through the 'what's new' > notes for the latest zmq 4.0 builds though, I didn't seem to find anything > like this. Is this still planned (or already available)? If so, is dev work > in progress for the managed implementations (jeromq/netmq)? > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] zproto and Malamute/Majordomo
Hello everybody, I'm taking a look at zproto project, in order to better structurize and document an application protocol. However, is there a way to use zproto on top of, for instance, malamute or majordomo API? Meaning defining my application protocol with zproto, but sending/receiving messages using malamute or majordomo? As far as I understood zproto uses the regular czmq API to send/recv messages, but malamute and majordomo both have different APIs to send/recv, like mlm_sendx () and mdp_client_send (). Regards, Lucas ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Paranoid Pirate pattern with priority queuing
Assume you have a requirement to support reliable priority queues whereby, if there is ever a message sent by a client with a higher priority (say 3 queues, high/medium/low) workers will always see those requests before they see lower priority requests. I'm currently working with jeromq and netmq, so the solution should be based on zmq 3.2.5. I have thought of doing this in the proxy with an endpoint per priority on the front end and one endpoint for the back end. I'm thinking the prioritization would come from the proxy by organizing its poller's inputs such that each iteration through its polling loop checks for input in descending order from highest to lowest and delivers from the first frontend socket available for reading. Does this sound right? I assume the logic at the top of the poller loop still hold (only poll for input on all of the priority endpoints if there's at least one 'ready' worker), right? Is this the simplest approach? It seems that it gets pretty port-hungry at the proxy as the number of such prioritized queues grows ( (# of priorities + 1) x (# of distinct prioritized queues) ). I'm thinking one poller thread per distinct prioritized queue. Also, I seem to remember somewhere that versions of zeromq supporting zmtp 4.0 were expected to alleviate this by supporting ipc/tcp endpoints that can specify different resources at the same endpoint, ala tcp://*:/queue-name/priority-value. Looking through the 'what's new' notes for the latest zmq 4.0 builds though, I didn't seem to find anything like this. Is this still planned (or already available)? If so, is dev work in progress for the managed implementations (jeromq/netmq)? ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zproxy and curve
No problem
Alan
-Original Message-
From: zeromq-dev-boun...@lists.zeromq.org
[mailto:zeromq-dev-boun...@lists.zeromq.org] On Behalf Of Pieter Hintjens
Sent: 27 April 2015 09:51
To: ZeroMQ development list
Subject: Re: [zeromq-dev] zproxy and curve
This is very kind of you. I've posted the code here:
https://gist.github.com/anonymous/6fb336d4cc7e5cf76eed
Hope that is OK.
-Pieter
On Mon, Apr 27, 2015 at 10:25 AM, Alan Ward wrote:
> Hi Pieter,
> Sorry for the delay, but I work only part-time on this.
>
> My overall architecture is based on the Asynchronous Client/Server pattern
> from the Guide.
>
> I have N (configurable) backend worker processes to provide solution scaling.
> They are processes rather than threads as they use postgres.
>
> Until recently I was using zmq_proxy to handle the distribution of work to
> these workers and the retrieval of their replies. I found I had to deal with
> the curve security before using the proxy, but that is OK in my case.
>
> When I moved to zsock, etc; zmq_proxy stopped working (as discussed
> previously) and I found there was no way of using zproxy with curve.
>
> I was already using a zpoller solution in my client, so I transferred that
> logic to the server to handle the workers. This also enabled me to clean up
> my close-down as this code was able to action the STOP message.
>
> As requested I have included the code I use to handle all this. [I
> have simplified it slightly and not tested it]
>
> --
> -
>
> {
> char worker_endpoint [128], worker_endpoints [1024];
> int process_instance;
>
> worker_endpoints [0] = 0;
>
> // start async worker processes
> for ( process_instance = 0; process_instance < CFG_Backend_Threads;
> process_instance++ ) {
> pid_t pid = fork();
>
> if ( pid == 0 ) { /* now in the forked process*/
> server_worker ( process_instance );
> _exit (0);
> }
> else
> // build string for worker IPC connections
> sprintf ( worker_endpoint, "ipc:///tmp/async%d.ipc", process_instance );
> strcat ( worker_endpoints, worker_endpoint );
> if ( process_instance < CFG_Backend_Threads - 1 ) strcat (
> worker_endpoints, "," );
> }
>
> // Connect frontend socket that talks to clients
> zsock_t *frontend = zsock_new ( ZMQ_ROUTER );
> assert ( frontend );
>
> // set up 2-way curve security
> secure_connection ( frontend );
>
> int rc = zsock_bind (frontend, "tcp://*:%s", CFG_Server_Async_Port);
> assert ( rc != -1 );
>
> // connect socket for async backend workers
> zsock_t *backend = zsock_new ( ZMQ_DEALER );
> assert ( backend );
> rc = zsock_attach ( backend, worker_endpoints, 1 );
> assert ( rc != -1 );
>
> // Set up poller to handle 2-way traffic
> zpoller_t *poller = zpoller_new ( frontend,
> backend,
> NULL );
> assert (poller);
>
> while (1) {
>
> void *which = zpoller_wait (poller, -1 );
>
> if ( which == frontend ) {
>
> // get incoming client message
> zmsg_t *msg = zmsg_recv ( frontend );
> assert (msg);
>
> // test for STOP command
> zframe_t *content = zmsg_last (msg);
> assert (content);
> char *message = zframe_data (content);
> if (streq (message, "STOP") ) {
>
> // send STOP message to each worker, before exiting
> for ( process_instance = 0; process_instance <
> CFG_Backend_Threads; process_instance++ ) {
>
> // need to copy message as it is destroyed by send
> zmsg_t *msg_copy = zmsg_dup ( msg );
> rc = zmsg_send ( &msg_copy, backend );
> }
>
> zmsg_destroy ( &msg );
>
> // wait for workers to get message before destroying socket
> sleep (1);
> zsock_destroy ( &frontend );
> zsock_destroy ( &backend );
> return;
> }
>
> // not STOP, send message to worker process
> rc = zmsg_send ( &msg, backend );
> assert ( rc != -1 );
> }
>
> else if ( which == backend ) {
>
> // get reply mesage from backend process and send to client
> zmsg_t *msg = zmsg_recv ( backend );
> assert (msg);
> rc = zmsg_send ( &msg, frontend );
> assert ( rc != -1 );
> }
>
> else if ( zpoller_terminated ) break;
> }
> }
>
> --
>
>
> Regards,
> Alan
>
> -Original Message-
> From: zeromq-dev-boun...@lists.zeromq.org
> [mailto:zeromq-dev-boun...@lists.zeromq.org] On Behalf Of Pieter
> Hintjens
> Sent: 22 April 2015 14:36
> To: ZeroMQ development list
> Subject: Re: [zeromq-dev] zproxy and curve
>
> Nice. if you want to publish that piece of code, others may enjoy seeing how
> you did it.
>
> On Wed, Apr
Re: [zeromq-dev] zproxy and curve
This is very kind of you. I've posted the code here:
https://gist.github.com/anonymous/6fb336d4cc7e5cf76eed
Hope that is OK.
-Pieter
On Mon, Apr 27, 2015 at 10:25 AM, Alan Ward wrote:
> Hi Pieter,
> Sorry for the delay, but I work only part-time on this.
>
> My overall architecture is based on the Asynchronous Client/Server pattern
> from the Guide.
>
> I have N (configurable) backend worker processes to provide solution scaling.
> They are processes rather than threads as they use postgres.
>
> Until recently I was using zmq_proxy to handle the distribution of work to
> these workers and the retrieval of their replies. I found I had to deal with
> the curve security before using the proxy, but that is OK in my case.
>
> When I moved to zsock, etc; zmq_proxy stopped working (as discussed
> previously) and I found there was no way of using zproxy with curve.
>
> I was already using a zpoller solution in my client, so I transferred that
> logic to the server to handle the workers. This also enabled me to clean up
> my close-down as this code was able to action the STOP message.
>
> As requested I have included the code I use to handle all this. [I have
> simplified it slightly and not tested it]
>
> ---
>
> {
> char worker_endpoint [128], worker_endpoints [1024];
> int process_instance;
>
> worker_endpoints [0] = 0;
>
> // start async worker processes
> for ( process_instance = 0; process_instance < CFG_Backend_Threads;
> process_instance++ ) {
> pid_t pid = fork();
>
> if ( pid == 0 ) { /* now in the forked process*/
> server_worker ( process_instance );
> _exit (0);
> }
> else
> // build string for worker IPC connections
> sprintf ( worker_endpoint, "ipc:///tmp/async%d.ipc", process_instance );
> strcat ( worker_endpoints, worker_endpoint );
> if ( process_instance < CFG_Backend_Threads - 1 ) strcat (
> worker_endpoints, "," );
> }
>
> // Connect frontend socket that talks to clients
> zsock_t *frontend = zsock_new ( ZMQ_ROUTER );
> assert ( frontend );
>
> // set up 2-way curve security
> secure_connection ( frontend );
>
> int rc = zsock_bind (frontend, "tcp://*:%s", CFG_Server_Async_Port);
> assert ( rc != -1 );
>
> // connect socket for async backend workers
> zsock_t *backend = zsock_new ( ZMQ_DEALER );
> assert ( backend );
> rc = zsock_attach ( backend, worker_endpoints, 1 );
> assert ( rc != -1 );
>
> // Set up poller to handle 2-way traffic
> zpoller_t *poller = zpoller_new ( frontend,
> backend,
> NULL );
> assert (poller);
>
> while (1) {
>
> void *which = zpoller_wait (poller, -1 );
>
> if ( which == frontend ) {
>
> // get incoming client message
> zmsg_t *msg = zmsg_recv ( frontend );
> assert (msg);
>
> // test for STOP command
> zframe_t *content = zmsg_last (msg);
> assert (content);
> char *message = zframe_data (content);
> if (streq (message, "STOP") ) {
>
> // send STOP message to each worker, before exiting
> for ( process_instance = 0; process_instance < CFG_Backend_Threads;
> process_instance++ ) {
>
> // need to copy message as it is destroyed by send
> zmsg_t *msg_copy = zmsg_dup ( msg );
> rc = zmsg_send ( &msg_copy, backend );
> }
>
> zmsg_destroy ( &msg );
>
> // wait for workers to get message before destroying socket
> sleep (1);
> zsock_destroy ( &frontend );
> zsock_destroy ( &backend );
> return;
> }
>
> // not STOP, send message to worker process
> rc = zmsg_send ( &msg, backend );
> assert ( rc != -1 );
> }
>
> else if ( which == backend ) {
>
> // get reply mesage from backend process and send to client
> zmsg_t *msg = zmsg_recv ( backend );
> assert (msg);
> rc = zmsg_send ( &msg, frontend );
> assert ( rc != -1 );
> }
>
> else if ( zpoller_terminated ) break;
> }
> }
>
> --
>
> Regards,
> Alan
>
> -Original Message-
> From: zeromq-dev-boun...@lists.zeromq.org
> [mailto:zeromq-dev-boun...@lists.zeromq.org] On Behalf Of Pieter Hintjens
> Sent: 22 April 2015 14:36
> To: ZeroMQ development list
> Subject: Re: [zeromq-dev] zproxy and curve
>
> Nice. if you want to publish that piece of code, others may enjoy seeing how
> you did it.
>
> On Wed, Apr 22, 2015 at 12:43 PM, Alan Ward wrote:
>> Hi Pieter,
>>
>> Thanks for the replies.
>> In the end I cooked my own proxy using a zpoller loop.
>> This also allows me to handle close down more cleanly.
>>
>> Thanks,
>> Alan
>>
>> -Original Message-
>> From: zero
Re: [zeromq-dev] zproxy and curve
Hi Pieter,
Sorry for the delay, but I work only part-time on this.
My overall architecture is based on the Asynchronous Client/Server pattern from
the Guide.
I have N (configurable) backend worker processes to provide solution scaling.
They are processes rather than threads as they use postgres.
Until recently I was using zmq_proxy to handle the distribution of work to
these workers and the retrieval of their replies. I found I had to deal with
the curve security before using the proxy, but that is OK in my case.
When I moved to zsock, etc; zmq_proxy stopped working (as discussed previously)
and I found there was no way of using zproxy with curve.
I was already using a zpoller solution in my client, so I transferred that
logic to the server to handle the workers. This also enabled me to clean up my
close-down as this code was able to action the STOP message.
As requested I have included the code I use to handle all this. [I have
simplified it slightly and not tested it]
---
{
char worker_endpoint [128], worker_endpoints [1024];
int process_instance;
worker_endpoints [0] = 0;
// start async worker processes
for ( process_instance = 0; process_instance < CFG_Backend_Threads;
process_instance++ ) {
pid_t pid = fork();
if ( pid == 0 ) { /* now in the forked process*/
server_worker ( process_instance );
_exit (0);
}
else
// build string for worker IPC connections
sprintf ( worker_endpoint, "ipc:///tmp/async%d.ipc", process_instance );
strcat ( worker_endpoints, worker_endpoint );
if ( process_instance < CFG_Backend_Threads - 1 ) strcat (
worker_endpoints, "," );
}
// Connect frontend socket that talks to clients
zsock_t *frontend = zsock_new ( ZMQ_ROUTER );
assert ( frontend );
// set up 2-way curve security
secure_connection ( frontend );
int rc = zsock_bind (frontend, "tcp://*:%s", CFG_Server_Async_Port);
assert ( rc != -1 );
// connect socket for async backend workers
zsock_t *backend = zsock_new ( ZMQ_DEALER );
assert ( backend );
rc = zsock_attach ( backend, worker_endpoints, 1 );
assert ( rc != -1 );
// Set up poller to handle 2-way traffic
zpoller_t *poller = zpoller_new ( frontend,
backend,
NULL );
assert (poller);
while (1) {
void *which = zpoller_wait (poller, -1 );
if ( which == frontend ) {
// get incoming client message
zmsg_t *msg = zmsg_recv ( frontend );
assert (msg);
// test for STOP command
zframe_t *content = zmsg_last (msg);
assert (content);
char *message = zframe_data (content);
if (streq (message, "STOP") ) {
// send STOP message to each worker, before exiting
for ( process_instance = 0; process_instance < CFG_Backend_Threads;
process_instance++ ) {
// need to copy message as it is destroyed by send
zmsg_t *msg_copy = zmsg_dup ( msg );
rc = zmsg_send ( &msg_copy, backend );
}
zmsg_destroy ( &msg );
// wait for workers to get message before destroying socket
sleep (1);
zsock_destroy ( &frontend );
zsock_destroy ( &backend );
return;
}
// not STOP, send message to worker process
rc = zmsg_send ( &msg, backend );
assert ( rc != -1 );
}
else if ( which == backend ) {
// get reply mesage from backend process and send to client
zmsg_t *msg = zmsg_recv ( backend );
assert (msg);
rc = zmsg_send ( &msg, frontend );
assert ( rc != -1 );
}
else if ( zpoller_terminated ) break;
}
}
--
Regards,
Alan
-Original Message-
From: zeromq-dev-boun...@lists.zeromq.org
[mailto:zeromq-dev-boun...@lists.zeromq.org] On Behalf Of Pieter Hintjens
Sent: 22 April 2015 14:36
To: ZeroMQ development list
Subject: Re: [zeromq-dev] zproxy and curve
Nice. if you want to publish that piece of code, others may enjoy seeing how
you did it.
On Wed, Apr 22, 2015 at 12:43 PM, Alan Ward wrote:
> Hi Pieter,
>
> Thanks for the replies.
> In the end I cooked my own proxy using a zpoller loop.
> This also allows me to handle close down more cleanly.
>
> Thanks,
> Alan
>
> -Original Message-
> From: zeromq-dev-boun...@lists.zeromq.org
> [mailto:zeromq-dev-boun...@lists.zeromq.org] On Behalf Of Pieter
> Hintjens
> Sent: 20 April 2015 12:25
> To: ZeroMQ development list
> Subject: Re: [zeromq-dev] zproxy and curve
>
> Maybe I should wait another 5 minutes so you can answer the question
> :-)
>
> Indeed, there is no support for configuring CURVE security on the zproxy
> sockets.
>
> Two options. A is to add
