Re: [zeromq-dev] using majordomo broker with asynchronous clients
Vishal,
Please see the freelance model; a broker design won't scale to 1M
messages a second without significant extra work.
-Pieter
On Fri, Dec 12, 2014 at 7:57 AM, Vishal Ahuja vahu...@gmail.com wrote:
Andre, thank you for your help! I changed the client to send 10k requests at
once, and then send two requests for every reply that it receives. I chose
10k because that allowed me to run up to ten clients (each sending 100k
messages) in parallel without any packet loss. Here is the client code:
#include ../include/mdp.h
#include time.h
int main (int argc, char *argv [])
{
int verbose = (argc 1 streq (argv [1], -v));
mdp_client_t *session = mdp_client_new (argv[1], verbose);
//mdp_client_t *session = mdp_client_new(tcp://localhost:,
verbose);
int count1, count2;
struct timeval start,end;
gettimeofday(start, NULL);
for (count1 = 0; count1 1; count1++) {
zmsg_t *request = zmsg_new ();
zmsg_pushstr (request, Hello world);
mdp_client_send (session, echo, request);
}
printf(sent all\n);
for (count1 = 0; count1 45000; count1++) {
zmsg_t *reply = mdp_client_recv (session,NULL,NULL);
if (reply)
{
zmsg_destroy (reply);
zmsg_t *request = zmsg_new ();
zmsg_pushstr (request, Hello world);
mdp_client_send (session, echo, request);
request = zmsg_new ();
zmsg_pushstr (request, Hello world);
mdp_client_send (session, echo, request);
}
else
break; // Interrupted by Ctrl-C
}
/* receiving the remaining 55k replies */
for(count1 = 45000; count1 10; count1++)
{
zmsg_t *reply = mdp_client_recv (session,NULL,NULL);
if (reply)
{
zmsg_destroy (reply);
}
else
break;
}
gettimeofday(end, NULL);
long elapsed = (end.tv_sec - start.tv_sec) +((end.tv_usec -
start.tv_usec)/100);
printf(time = %ld\n, elapsed);
printf (%d replies received\n, count1);
mdp_client_destroy (session);
return 0;
}
I ran the broker, worker, and the clients within the same machine. Here is
the recorded time:
# of clients in parallel
(each client sends 100k ) Time elapsed (seconds)
1 4
2 9
312
416
521
10 43
So for every 100k requests, the broker is taking about 4 seconds. Is this
the expected behavior? Am not sure how to achieve million messages per
second. Please advise.
Sincerely,
Vishal
On Thu, Dec 11, 2014 at 10:22 PM, André Caron andre.l.ca...@gmail.com
wrote:
You are observing lost replies because the broker is sending replies
faster than the client is consuming them: each time the broker sends a
reply, it is stored in an internal queue for that client. When the broker
sends more replies than the client can consume, this internal queue fills up
to tolerate shorts bursts (the client's RCVHWM and the broker's SNDHWM limit
indirectly control this tolerance). However, if the broker's send rate is
constantly faster than the client's recv rate, you need infinite memory to
keep avoid losing replies.
In this scenario, the clients are operating in request-reply, so if this
happens, they are sending requests faster than they can consume the replies.
Basically, your client is sending requests until both the client's reply
queue and the broker's reply queue (for that client) are saturated and then,
instead of reading replies, it continues sending requests. This doesn't
make any sense in any real world scenario. The real problem here is that
the test your are performing is not realistic.
Note that it would be different if it was pub-sub instead of req-rep
because the client would not be in control of the broker's reply rate.
To fix this to match a real-world scenario, you should modify your client
program such that it has at most N outstanding requests, where N is
guaranteed to fit in the cilent's reply queue and broker's reply queue:
- initially send N requests;
- each time you get a reply, send a new request.
If you do this and the client's RCVHWM and broker's SNDHWM are
sufficiently high to hold a total of N replies, then you will go down to 0
lost replies -- unless you hit other problems like disconnections, of
course.
Instead of doing that, you could technically increase the server's SNDHWM
to hold 100,000 replies per client (assuming
Re: [zeromq-dev] Majordomo handling malformed messages
We do need to make a new RFC, as the code has gone beyond RFC 18. It's worth making the broker more robust, if you are using it. On Fri, Dec 12, 2014 at 10:05 AM, Lucas Russo ler...@gmail.com wrote: Hello Everyone, I'm using the majordomo protocol to do a service-based thread lookup, sharing the same context among all threads. I would like to make the reference majordomo implementation (https://github.com/zeromq/majordomo/) a little bit more robust against malformed messages. Right now, if any of the parts (client, broker or worker) receive a wrong message format, it will just crash with the asserts being used or not check the message whatsoever. So, if you think this is a valid putting a little effort on, I would like to ask for suggestions on how to do that. First of all, maybe the majordomo protocol (http://rfc.zeromq.org/spec:18) would need to be changed, as the worker and/or broker does not send an error message back to the client in response to a malformed message received. Second, with the modification in the protocol, the work would be just to remove the asserts and replace them with proper checking and error messages back to the client. Do you guys think this would make a good improvement for the Majordomo protocol and reference implementation? Also, I know about the Malamute intent to be a general messaging protocol, but does it replace Majordomo? Regards, Lucas ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] event based messaging system
You don't need a timer per request. A typical way to do this sort of scheduling is to keep a queue ordered by event time, by inserting each new event into the queue by comparing event times. You set your single timer's expiration for the event closest to now (eg the head of the queue). When that event timer expires, you publish your notification, pop the head of the queue and set the timer to the next nearest event time. On Friday, December 12, 2014, Vishal Ahuja vahu...@gmail.com wrote: I am trying to design an event based messaging system which works as follows: Let us say that there are multiple products in a store, and their prices vary every day. Clients can login to the application, and request to know (via SMS) the price of a particular product on a particular date (say 1 month from the date of login). The application will not have access to a database. It will have to retrieve the information from a web page. So let us say that the user wants to know the price of a product on the 1st of Jan, then the application will have to access the website on that particular day itself. Also, another detail is that the price of the product can be changed any time during the day. I am not sure how to design this using zeromq, particularly I am not sure how to receive a client request and then act on it after a month or so. Will I have to start a timer for every request? Please help me with the design. Sincerely, Vishal ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zmq_poll performance question
Hey, I did not check for performance improvement, I simply tested that my program continued to work. The thing is, my poller only has 1 zmq socket and 1 file descriptor (that relies on POLLPRI -- its a GPIO pin). So in my case I could'nt really check for perf improvement. Also I am running this on a Raspberry Pi. I used it for about 1hour before sending the previous mail. Gonna work with this patch applied next week, making sure it keeps working. On Thu, Dec 11, 2014 at 2:39 PM, Francis Le Bourse zno-reply-francis.lebou...@sfr-sh.fr wrote: Hello, On 12/10/2014 6:54 PM, Arnaud Kapp wrote: Hello, Sorry it took me a while, but I finally go to test your patch. My setup that use POLLPRI seems to work properly with your patch applied :). Good. Do you see a performance improvement ? How long have you been using it ? Did you submit a PR to get it merged into libzmq master yet? No, not yet. I was waiting for feedback before. And I had another issue with a memory hog in libzmq. Cheers, Francis On Fri, Nov 28, 2014 at 11:35 AM, Francis Le Bourse zno-reply-francis.lebou...@sfr-sh.fr wrote: On 11/24/2014 8:08 PM, Arnaud Kapp wrote: Currently, the patch is written for 3.2.4. I'll wait to put it in libzmq master The first patch for 3.2.4 had an issue in zmq_poll(), I had tried a little too aggressive optimization by bypassing the first_pass processing. It is fixed in the current one. The patch for the current head is clean. Cheers, Francis Oh okay. This is the commit that added the flag: https://github.com/zeromq/libzmq/commit/779c37abc433cb6595ddeedaf86b280317656bdd libzmq was 4.1 at the time I believe. I'll probably look at it this week-end then :) On Mon, Nov 24, 2014 at 12:10 PM, Francis Le Bourse zno-reply-francis.lebou...@sfr-sh.fr wrote: Hi, On 11/24/2014 11:35 AM, Arnaud Kapp wrote: Hello, I recently added support for POLLPRI flag. It looks like it's not handled in your patch No, it isn't handled. In which version do you have added this flag ? Currently, the patch is written for 3.2.4. I'll wait to put it in libzmq master. and that it needs custom support. Since there is no test related to this flags you wouldn't notice. I can give it a look if you want. That would be nice. Cheers, Francis On Sat, Nov 22, 2014 at 2:16 PM, Pieter Hintjens p...@imatix.com wrote: I suggest you send the patch to libzmq master, and ensure all test cases pass. Then we can get this into the next version. On Fri, Nov 21, 2014 at 2:50 PM, Francis Le Bourse zno-reply-francis.lebou...@sfr-sh.fr wrote: Hi, On 11/6/2014 3:18 PM, Pieter Hintjens wrote: Oh, ok. Sounds like you have a good candidate for some before/after measurement and optimization. Are you going to try to make a patch for this? I have a patch candidate for this optimization, the performance improvement is very good and it doesn't seem to introduce any new instability. What is modified: - zmq_poll(), there is only one poll() now, - and epoll() from epoll.cpp Other calls to poll() and select() are left unmodified. I woulld be happy to have any feedback. Cheers, Francis On Thu, Nov 6, 2014 at 2:09 PM, Francis Le Bourse zno-reply-francis.lebou...@sfr-sh.fr wrote: On 11/6/2014 11:47 AM, Pieter Hintjens wrote: A simple optimization is, when you are polling sockets for input, to continue reading from an active socket using a non-blocking read. So you process all waiting messages on a socket and then only switch back to poll when needed. Thank you for you quick reply. Yes, but the question was more about the zmq_poll() internals. For 600+ file descriptors, zmq_poll() calls poll() a huge number of times for only a few that will trigger a POLLIN and the relevant information is already known / present in the pollfds array. The performance hit is there. Cheers, Francis On Thu, Nov 6, 2014 at 11:28 AM, Francis Le Bourse zno-reply-francis.lebou...@sfr-sh.fr wrote: Hi, I am looking at a performance issue in zmq, when the number of zsockets / file descriptors becomes large. The relevant calls are: poll+0x57 zmq_poll+0x2e3 zloop_start+0x1e8 main+0xb40 __libc_start_main+0xfd immediately followed by a loop of poll+0x57 zmq::signaler_t::wait(int)+0x33 zmq::mailbox_t::recv(zmq::command_t*, int)+0x78 zmq::socket_base_t::process_commands(int, bool)+0xbe zmq::socket_base_t::getsockopt(int, void*, unsigned long*)+0x135 zmq_getsockopt+0x75 zmq_poll+0x3da zloop_start+0x1e8 main+0xb40 __libc_start_main+0xfd The code in the loop is executed once per file descriptor in the initial pollarray, redoing a poll() system call each time. Is there a reason to proceed that way ? Would be possible to reuse the results of the first poll() in order to bypass the second set of system calls
[zeromq-dev] Implementing zeromq security
I'm trying to implement zeromq security for the first time and WITHOUT using CZMQ. Reading through http://hintjens.com/blog:49 ZAP protocol info http://rfc.zeromq.org/spec:27 code at zeromq-4.0.4\tests\tests_security_curve.cpp . I have a question and hope someone might be able to enlighten me At http://hintjens.com/blog:49 ; 'Internally, the authenticator talks to libzmq across http://rfc.zeromq.org/spec:27 a protocol called ZAP, aka RFC 27. Every single time a client tries to connect to a server, libzmq asks the ZAP handler (the authenticator), if there is one installed, to allow or deny the connection.' In zeromq-4.0.4\tests\tests_security_curve.cpp ; // Spawn ZAP handler // We create and bind ZAP socket in main thread to avoid case // where child thread does not start up fast enough. void *handler = zmq_socket (ctx, ZMQ_REP); assert (handler); rc = zmq_bind (handler, inproc://zeromq.zap.01); assert (rc == 0); void *zap_thread = zmq_threadstart (zap_handler, handler); Question is; I see that the zap_handler is provided with the pointer to the REP socket bound to inproc://zeromq.zap.01 however I don't understand how libzmq is made aware to direct authentication requests to the particular handler socket. I don't see any further references in the code to inproc://zeromq.zap.01? How does libzmq know where to send the authentication requests? One other thing; is there any api documentation for zmq_threadstart zmq_threadclose? Thanks, Riskybiz. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Implementing zeromq security
The inproc://zeromq.zap.01 url is hardcoded in libzmq https://github.com/zeromq/zeromq4-x/blob/v4.0.5/src/session_base.cpp#L285. The session internally creates a client connection to that endpoint. -MinRK On Fri, Dec 12, 2014 at 12:56 PM, Riskybiz riskybizl...@live.com wrote: I’m trying to implement zeromq security for the first time and WITHOUT using CZMQ. Reading through http://hintjens.com/blog:49 ZAP protocol info http://rfc.zeromq.org/spec:27 code at zeromq-4.0.4\tests\tests_security_curve.cpp . I have a question and hope someone might be able to enlighten me At http://hintjens.com/blog:49 ; ‘Internally, the authenticator talks to libzmq across a protocol called ZAP http://rfc.zeromq.org/spec:27, aka RFC 27. Every single time a client tries to connect to a server, libzmq asks the ZAP handler (the authenticator), if there is one installed, to allow or deny the connection.’ In zeromq-4.0.4\tests\tests_security_curve.cpp ; // Spawn ZAP handler // We create and bind ZAP socket in main thread to avoid case // where child thread does not start up fast enough. void *handler = zmq_socket (ctx, ZMQ_REP); assert (handler); rc = zmq_bind (handler, inproc://zeromq.zap.01); assert (rc == 0); void *zap_thread = zmq_threadstart (zap_handler, handler); Question is; I see that the zap_handler is provided with the pointer to the REP socket bound to inproc://zeromq.zap.01 however I don’t understand how libzmq is made aware to direct authentication requests to the particular handler socket. I don’t see any further references in the code to inproc://zeromq.zap.01? How does libzmq know where to send the authentication requests? One other thing; is there any api documentation for zmq_threadstart zmq_threadclose? Thanks, Riskybiz. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Using Strawhouse security pattern with ZeroMQ
I am trying to use Strawhouse security pattern in my zero-mq development. I
was following this wiki http://hintjens.com/blog:49 and when I try to run
below simple program to make sure I have everything installed, I got an
error -
#include czmq.h
int main (void) {
zctx_t *ctx = zctx_new ();
void *publisher = zsocket_new (ctx, ZMQ_PUB);
zsocket_set_curve_server (publisher, true);
puts (Hello, Curve!);
zctx_destroy (ctx);
*return* 0;
}
I tried to compile it like this -
gcc -o hello hello.c -lczmq -lzmq -lsodium
And the error I got -
/usr/bin/ld: warning: libzmq.so.4, needed by
/usr/local/lib/libczmq.so, may conflict with libzmq.so.3
Does anyone know what does this mean and what wrong I am doing?
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
