Throughput is bound by multiple factors. In your case, my guess is that your bottleneck is the throughput of a single TCP socket on your system. I doubt that you can reach millions of messages per second with only 3 clients. If you add more clients, you should see the server's throughput increase well over 30k.
Cheers, André On Thu, Dec 11, 2014 at 4:40 AM, Vishal Ahuja <[email protected]> wrote: > I removed the printfs, and increased the TCP send and receive buffers to > the max possible. When I have two clients (running in parallel) sending > 100k messages, there is no packet loss. But when I increase the number of > clients to three, then each client receives around 93k packets. Also, the > majordomo specification (http://rfc.zeromq.org/spec:7) says that the > broker should be able to switch millions of messages per second, whereas I > seeing a throughput of around 30k messages per second. Can someone tell how > to improve the throughput? > > Sincerely, > > Vishal > > On Thu, Dec 11, 2014 at 10:37 AM, Andre Caron <[email protected]> > wrote: > >> The broker uses a router socket, which silently drops outbound messages >> when the sndhwm is reached. If your client is reading more slowly than the >> broker is writing, you'll lose messages. >> >> In your case, the slow part is writing to the terminal. Try removing the >> printf in the recv loop and you should lose fewer messages. >> >> André >> >> > On Dec 10, 2014, at 9:40 PM, Vishal Ahuja <[email protected]> wrote: >> > >> > While reading the zeromq guide, I came across client code (pertaining >> to the majordomo pattern) which sends 100k requests in a loop, and then >> receives the reply in a second loop. >> > >> > #include "../include/mdp.h" >> > #include <time.h> >> > >> > >> > int main (int argc, char *argv []) >> > { >> > int verbose = (argc > 1 && streq (argv [1], "-v")); >> > mdp_client_t *session = mdp_client_new ("tcp://localhost:5555", >> verbose); >> > int count; >> > for (count = 0; count < 100000; count++) { >> > zmsg_t *request = zmsg_new (); >> > zmsg_pushstr (request, "Hello world"); >> > mdp_client_send (session, "echo", &request); >> > } >> > printf("sent all\n"); >> > >> > for (count = 0; count < 100000; count++) { >> > zmsg_t *reply = mdp_client_recv (session,NULL,NULL); >> > if (reply) >> > zmsg_destroy (&reply); >> > else >> > break; // Interrupted by Ctrl-C >> > printf("reply received:%d\n", count); >> > } >> > printf ("%d replies received\n", count); >> > mdp_client_destroy (&session); >> > return 0; >> > } >> > >> > I have added a counter to count the number of replies that the worker >> (test_worker.c) sends to the broker, and another counter in mdp_broker.c to >> count the number of replies the broker sends to a client. Both of these >> count up to 100k, but the client is receiving only around 37k replies. >> > >> > If the number of client requests is set to around 40k, then it receives >> all the replies. Can someone please tell me why packets are lost when the >> client sends more than 40k asynchronous requests? >> > >> > I tried setting the HWM to 100k for the broker socket, but the problem >> persists: >> > >> > static broker_t * >> > s_broker_new (int verbose) >> > { >> > broker_t *self = (broker_t *) zmalloc (sizeof (broker_t)); >> > int64_t hwm = 100000; >> > // Initialize broker state >> > self->ctx = zctx_new (); >> > self->socket = zsocket_new (self->ctx, ZMQ_ROUTER); >> > zmq_setsockopt(self->socket, ZMQ_SNDHWM, &hwm, sizeof(hwm)); >> > >> > zmq_setsockopt(self->socket, ZMQ_RCVHWM, &hwm, sizeof(hwm)); >> > self->verbose = verbose; >> > self->services = zhash_new (); >> > self->workers = zhash_new (); >> > self->waiting = zlist_new (); >> > self->heartbeat_at = zclock_time () + HEARTBEAT_INTERVAL; >> > return self; >> > } >> > >> > Vishal >> > _______________________________________________ >> > zeromq-dev mailing list >> > [email protected] >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
