Re: [zeromq-dev] PUB ZMQ_SNDHWM doesn't work
Hi Brian, From the later thread PUB/SUB unreliabiliity, I noticed that 4.0.x has resolved the issue PUB SNDHWM doesn't work. I verified it's true! Thanks, Zhichang 2014-06-12 18:22 GMT+08:00 Brian Knox bk...@digitalocean.com: Setting the HWM on a PUB socket to 0 would have the opposite effect of what you want. PUB sockets, when exceeding the HWM, drop messages. For a description of what different socket types do in a mute state, please see the documentation: http://api.zeromq.org/4-0:zmq-socket You should probably spend some time reading the zeromq guide as well. Cheers, Brian On Wed, Jun 11, 2014 at 9:24 PM, 余志昌 yuzhich...@gmail.com wrote: Hi, I tried to set hwm to zero at both sides of pubsub in order to avoid dropping messages. (I'm aware with possible memory overflow at this time.) However the PUB side ZMQ_SNDHWM customization doesn't work. Here's my PUB side code piece: // Socket to send messages to void *publisher = zmq_socket(context, ZMQ_PUB); // Raise the high water mark for outbound messages to unlimited. int sndhwm = 0; rc = zmq_setsockopt(publisher, ZMQ_SNDHWM, sndhwm, sizeof(int)); if(rc!=0) printf(%s:%d zmq_setsockopt() errno %d!\n, __FILE__, __LINE__, errno); assert(rc==0); sndhwm = -9; size_t s = sizeof(int); rc = zmq_getsockopt(publisher, ZMQ_SNDHWM, sndhwm, s); if(rc!=0) printf(%s:%d zmq_getsockopt() errno %d!\n, __FILE__, __LINE__, errno); printf(sndhwm: %d\n, sndhwm); assert(rc==0); rc = zmq_bind(publisher, SWITCH_OUT); if(rc!=0) cerr __FILE__ : __LINE__ zmq_bind() errno errno endl; assert(rc == 0); Here's gdb output(hwm is 1000. This doesn't match my expectation): (gdb) break pipe.cpp:157 Breakpoint 2 at 0x7fdbad9a7f30: file ../../src/pipe.cpp, line 157. (gdb) c Continuing. Breakpoint 2, zmq::pipe_t::check_write (this=0x134c8c0) at ../../src/pipe.cpp:157 157 in ../../src/pipe.cpp (gdb) bt #0 zmq::pipe_t::check_write (this=0x134c8c0) at ../../src/pipe.cpp:157 #1 0x7fdbad9a95a3 in zmq::pipe_t::write (this=0x134c8c0, msg_=0x7fff1280abf0) at ../../src/pipe.cpp:166 #2 0x7fdbad99be01 in zmq::dist_t::write (this=0x1302cb8, pipe_=0x134c8c0, msg_=0x3e8) at ../../src/dist.cpp:181 #3 0x7fdbad99bf93 in zmq::dist_t::distribute (this=0x1302cb8, msg_=0x7fff1280abf0, flags_=value optimized out) at ../../src/dist.cpp:145 #4 0x7fdbad99c17e in zmq::dist_t::send_to_matching (this=0x1302cb8, msg_=0x7fff1280abf0, flags_=2) at ../../src/dist.cpp:118 #5 0x7fdbad9bc2be in zmq::xpub_t::xsend (this=0x1302900, msg_=0x7fff1280abf0, flags_=2) at ../../src/xpub.cpp:129 #6 0x7fdbad9b02d0 in zmq::socket_base_t::send (this=0x1302900, msg_=0x7fff1280abf0, flags_=2) at ../../src/socket_base.cpp:656 #7 0x7fdbad9c365a in s_sendmsg (s_=0x1302900, msg_=0x7fff1280abf0, flags_=2) at ../../src/zmq.cpp:337 #8 0x7fdbad9c3c7e in zmq_send (s_=0x1302900, buf_=0x12f4110, len_=2, flags_=2) at ../../src/zmq.cpp:362 #9 0x00404810 in main (argc=value optimized out, argv=value optimized out) at ../src/switch.cpp:395 (gdb) p hwm $1 = 1000 (gdb) p lwm $2 = 500 ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] PUB ZMQ_SNDHWM doesn't work
Setting the HWM on a PUB socket to 0 would have the opposite effect of what you want. PUB sockets, when exceeding the HWM, drop messages. For a description of what different socket types do in a mute state, please see the documentation: http://api.zeromq.org/4-0:zmq-socket You should probably spend some time reading the zeromq guide as well. Cheers, Brian On Wed, Jun 11, 2014 at 9:24 PM, 余志昌 yuzhich...@gmail.com wrote: Hi, I tried to set hwm to zero at both sides of pubsub in order to avoid dropping messages. (I'm aware with possible memory overflow at this time.) However the PUB side ZMQ_SNDHWM customization doesn't work. Here's my PUB side code piece: // Socket to send messages to void *publisher = zmq_socket(context, ZMQ_PUB); // Raise the high water mark for outbound messages to unlimited. int sndhwm = 0; rc = zmq_setsockopt(publisher, ZMQ_SNDHWM, sndhwm, sizeof(int)); if(rc!=0) printf(%s:%d zmq_setsockopt() errno %d!\n, __FILE__, __LINE__, errno); assert(rc==0); sndhwm = -9; size_t s = sizeof(int); rc = zmq_getsockopt(publisher, ZMQ_SNDHWM, sndhwm, s); if(rc!=0) printf(%s:%d zmq_getsockopt() errno %d!\n, __FILE__, __LINE__, errno); printf(sndhwm: %d\n, sndhwm); assert(rc==0); rc = zmq_bind(publisher, SWITCH_OUT); if(rc!=0) cerr __FILE__ : __LINE__ zmq_bind() errno errno endl; assert(rc == 0); Here's gdb output(hwm is 1000. This doesn't match my expectation): (gdb) break pipe.cpp:157 Breakpoint 2 at 0x7fdbad9a7f30: file ../../src/pipe.cpp, line 157. (gdb) c Continuing. Breakpoint 2, zmq::pipe_t::check_write (this=0x134c8c0) at ../../src/pipe.cpp:157 157 in ../../src/pipe.cpp (gdb) bt #0 zmq::pipe_t::check_write (this=0x134c8c0) at ../../src/pipe.cpp:157 #1 0x7fdbad9a95a3 in zmq::pipe_t::write (this=0x134c8c0, msg_=0x7fff1280abf0) at ../../src/pipe.cpp:166 #2 0x7fdbad99be01 in zmq::dist_t::write (this=0x1302cb8, pipe_=0x134c8c0, msg_=0x3e8) at ../../src/dist.cpp:181 #3 0x7fdbad99bf93 in zmq::dist_t::distribute (this=0x1302cb8, msg_=0x7fff1280abf0, flags_=value optimized out) at ../../src/dist.cpp:145 #4 0x7fdbad99c17e in zmq::dist_t::send_to_matching (this=0x1302cb8, msg_=0x7fff1280abf0, flags_=2) at ../../src/dist.cpp:118 #5 0x7fdbad9bc2be in zmq::xpub_t::xsend (this=0x1302900, msg_=0x7fff1280abf0, flags_=2) at ../../src/xpub.cpp:129 #6 0x7fdbad9b02d0 in zmq::socket_base_t::send (this=0x1302900, msg_=0x7fff1280abf0, flags_=2) at ../../src/socket_base.cpp:656 #7 0x7fdbad9c365a in s_sendmsg (s_=0x1302900, msg_=0x7fff1280abf0, flags_=2) at ../../src/zmq.cpp:337 #8 0x7fdbad9c3c7e in zmq_send (s_=0x1302900, buf_=0x12f4110, len_=2, flags_=2) at ../../src/zmq.cpp:362 #9 0x00404810 in main (argc=value optimized out, argv=value optimized out) at ../src/switch.cpp:395 (gdb) p hwm $1 = 1000 (gdb) p lwm $2 = 500 ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] PUB ZMQ_SNDHWM doesn't work
Hi, I tried to set hwm to zero at both sides of pubsub in order to avoid dropping messages. (I'm file with possible memory overflow at this time.) However the PUB side ZMQ_SNDHWM customization doesn't work. Here's my PUB side code piece: // Socket to send messages to void *publisher = zmq_socket(context, ZMQ_PUB); // Raise the high water mark for outbound messages to unlimited. int sndhwm = 0; rc = zmq_setsockopt(publisher, ZMQ_SNDHWM, sndhwm, sizeof(int)); if(rc!=0) printf(%s:%d zmq_setsockopt() errno %d!\n, __FILE__, __LINE__, errno); assert(rc==0); sndhwm = -9; size_t s = sizeof(int); rc = zmq_getsockopt(publisher, ZMQ_SNDHWM, sndhwm, s); if(rc!=0) printf(%s:%d zmq_getsockopt() errno %d!\n, __FILE__, __LINE__, errno); printf(sndhwm: %d\n, sndhwm); assert(rc==0); rc = zmq_bind(publisher, SWITCH_OUT); if(rc!=0) cerr __FILE__ : __LINE__ zmq_bind() errno errno endl; assert(rc == 0); Here's gdb output(hwm is 1000. However I expect it to be zero according to ZMQ_SNDHWM option): (gdb) break pipe.cpp:157 Breakpoint 2 at 0x7fdbad9a7f30: file ../../src/pipe.cpp, line 157. (gdb) c Continuing. Breakpoint 2, zmq::pipe_t::check_write (this=0x134c8c0) at ../../src/pipe.cpp:157 157 in ../../src/pipe.cpp (gdb) bt #0 zmq::pipe_t::check_write (this=0x134c8c0) at ../../src/pipe.cpp:157 #1 0x7fdbad9a95a3 in zmq::pipe_t::write (this=0x134c8c0, msg_=0x7fff1280abf0) at ../../src/pipe.cpp:166 #2 0x7fdbad99be01 in zmq::dist_t::write (this=0x1302cb8, pipe_=0x134c8c0, msg_=0x3e8) at ../../src/dist.cpp:181 #3 0x7fdbad99bf93 in zmq::dist_t::distribute (this=0x1302cb8, msg_=0x7fff1280abf0, flags_=value optimized out) at ../../src/dist.cpp:145 #4 0x7fdbad99c17e in zmq::dist_t::send_to_matching (this=0x1302cb8, msg_=0x7fff1280abf0, flags_=2) at ../../src/dist.cpp:118 #5 0x7fdbad9bc2be in zmq::xpub_t::xsend (this=0x1302900, msg_=0x7fff1280abf0, flags_=2) at ../../src/xpub.cpp:129 #6 0x7fdbad9b02d0 in zmq::socket_base_t::send (this=0x1302900, msg_=0x7fff1280abf0, flags_=2) at ../../src/socket_base.cpp:656 #7 0x7fdbad9c365a in s_sendmsg (s_=0x1302900, msg_=0x7fff1280abf0, flags_=2) at ../../src/zmq.cpp:337 #8 0x7fdbad9c3c7e in zmq_send (s_=0x1302900, buf_=0x12f4110, len_=2, flags_=2) at ../../src/zmq.cpp:362 #9 0x00404810 in main (argc=value optimized out, argv=value optimized out) at ../src/switch.cpp:395 (gdb) p hwm $1 = 1000 (gdb) p lwm $2 = 500 ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] PUB ZMQ_SNDHWM doesn't work
I'm using ZeroMQ 3.2.3 2014-06-11 15:29 GMT+08:00 余志昌 yuzhich...@gmail.com: Hi, I tried to set hwm to zero at both sides of pubsub in order to avoid dropping messages. (I'm file with possible memory overflow at this time.) However the PUB side ZMQ_SNDHWM customization doesn't work. Here's my PUB side code piece: // Socket to send messages to void *publisher = zmq_socket(context, ZMQ_PUB); // Raise the high water mark for outbound messages to unlimited. int sndhwm = 0; rc = zmq_setsockopt(publisher, ZMQ_SNDHWM, sndhwm, sizeof(int)); if(rc!=0) printf(%s:%d zmq_setsockopt() errno %d!\n, __FILE__, __LINE__, errno); assert(rc==0); sndhwm = -9; size_t s = sizeof(int); rc = zmq_getsockopt(publisher, ZMQ_SNDHWM, sndhwm, s); if(rc!=0) printf(%s:%d zmq_getsockopt() errno %d!\n, __FILE__, __LINE__, errno); printf(sndhwm: %d\n, sndhwm); assert(rc==0); rc = zmq_bind(publisher, SWITCH_OUT); if(rc!=0) cerr __FILE__ : __LINE__ zmq_bind() errno errno endl; assert(rc == 0); Here's gdb output(hwm is 1000. However I expect it to be zero according to ZMQ_SNDHWM option): (gdb) break pipe.cpp:157 Breakpoint 2 at 0x7fdbad9a7f30: file ../../src/pipe.cpp, line 157. (gdb) c Continuing. Breakpoint 2, zmq::pipe_t::check_write (this=0x134c8c0) at ../../src/pipe.cpp:157 157 in ../../src/pipe.cpp (gdb) bt #0 zmq::pipe_t::check_write (this=0x134c8c0) at ../../src/pipe.cpp:157 #1 0x7fdbad9a95a3 in zmq::pipe_t::write (this=0x134c8c0, msg_=0x7fff1280abf0) at ../../src/pipe.cpp:166 #2 0x7fdbad99be01 in zmq::dist_t::write (this=0x1302cb8, pipe_=0x134c8c0, msg_=0x3e8) at ../../src/dist.cpp:181 #3 0x7fdbad99bf93 in zmq::dist_t::distribute (this=0x1302cb8, msg_=0x7fff1280abf0, flags_=value optimized out) at ../../src/dist.cpp:145 #4 0x7fdbad99c17e in zmq::dist_t::send_to_matching (this=0x1302cb8, msg_=0x7fff1280abf0, flags_=2) at ../../src/dist.cpp:118 #5 0x7fdbad9bc2be in zmq::xpub_t::xsend (this=0x1302900, msg_=0x7fff1280abf0, flags_=2) at ../../src/xpub.cpp:129 #6 0x7fdbad9b02d0 in zmq::socket_base_t::send (this=0x1302900, msg_=0x7fff1280abf0, flags_=2) at ../../src/socket_base.cpp:656 #7 0x7fdbad9c365a in s_sendmsg (s_=0x1302900, msg_=0x7fff1280abf0, flags_=2) at ../../src/zmq.cpp:337 #8 0x7fdbad9c3c7e in zmq_send (s_=0x1302900, buf_=0x12f4110, len_=2, flags_=2) at ../../src/zmq.cpp:362 #9 0x00404810 in main (argc=value optimized out, argv=value optimized out) at ../src/switch.cpp:395 (gdb) p hwm $1 = 1000 (gdb) p lwm $2 = 500 ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] PUB ZMQ_SNDHWM doesn't work
Hi, I tried to set hwm to zero at both sides of pubsub in order to avoid dropping messages. (I'm aware with possible memory overflow at this time.) However the PUB side ZMQ_SNDHWM customization doesn't work. Here's my PUB side code piece: // Socket to send messages to void *publisher = zmq_socket(context, ZMQ_PUB); // Raise the high water mark for outbound messages to unlimited. int sndhwm = 0; rc = zmq_setsockopt(publisher, ZMQ_SNDHWM, sndhwm, sizeof(int)); if(rc!=0) printf(%s:%d zmq_setsockopt() errno %d!\n, __FILE__, __LINE__, errno); assert(rc==0); sndhwm = -9; size_t s = sizeof(int); rc = zmq_getsockopt(publisher, ZMQ_SNDHWM, sndhwm, s); if(rc!=0) printf(%s:%d zmq_getsockopt() errno %d!\n, __FILE__, __LINE__, errno); printf(sndhwm: %d\n, sndhwm); assert(rc==0); rc = zmq_bind(publisher, SWITCH_OUT); if(rc!=0) cerr __FILE__ : __LINE__ zmq_bind() errno errno endl; assert(rc == 0); Here's gdb output(hwm is 1000. This doesn't match my expectation): (gdb) break pipe.cpp:157 Breakpoint 2 at 0x7fdbad9a7f30: file ../../src/pipe.cpp, line 157. (gdb) c Continuing. Breakpoint 2, zmq::pipe_t::check_write (this=0x134c8c0) at ../../src/pipe.cpp:157 157 in ../../src/pipe.cpp (gdb) bt #0 zmq::pipe_t::check_write (this=0x134c8c0) at ../../src/pipe.cpp:157 #1 0x7fdbad9a95a3 in zmq::pipe_t::write (this=0x134c8c0, msg_=0x7fff1280abf0) at ../../src/pipe.cpp:166 #2 0x7fdbad99be01 in zmq::dist_t::write (this=0x1302cb8, pipe_=0x134c8c0, msg_=0x3e8) at ../../src/dist.cpp:181 #3 0x7fdbad99bf93 in zmq::dist_t::distribute (this=0x1302cb8, msg_=0x7fff1280abf0, flags_=value optimized out) at ../../src/dist.cpp:145 #4 0x7fdbad99c17e in zmq::dist_t::send_to_matching (this=0x1302cb8, msg_=0x7fff1280abf0, flags_=2) at ../../src/dist.cpp:118 #5 0x7fdbad9bc2be in zmq::xpub_t::xsend (this=0x1302900, msg_=0x7fff1280abf0, flags_=2) at ../../src/xpub.cpp:129 #6 0x7fdbad9b02d0 in zmq::socket_base_t::send (this=0x1302900, msg_=0x7fff1280abf0, flags_=2) at ../../src/socket_base.cpp:656 #7 0x7fdbad9c365a in s_sendmsg (s_=0x1302900, msg_=0x7fff1280abf0, flags_=2) at ../../src/zmq.cpp:337 #8 0x7fdbad9c3c7e in zmq_send (s_=0x1302900, buf_=0x12f4110, len_=2, flags_=2) at ../../src/zmq.cpp:362 #9 0x00404810 in main (argc=value optimized out, argv=value optimized out) at ../src/switch.cpp:395 (gdb) p hwm $1 = 1000 (gdb) p lwm $2 = 500 ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] PUB ZMQ_SNDHWM doesn't work
Hi, I tried to set hwm to zero at both sides of pubsub in order to avoid dropping messages. (I'm aware with possible memory overflow at this time.) However the PUB side ZMQ_SNDHWM customization doesn't work. Here's my PUB side code piece: // Socket to send messages to void *publisher = zmq_socket(context, ZMQ_PUB); // Raise the high water mark for outbound messages to unlimited. int sndhwm = 0; rc = zmq_setsockopt(publisher, ZMQ_SNDHWM, sndhwm, sizeof(int)); if(rc!=0) printf(%s:%d zmq_setsockopt() errno %d!\n, __FILE__, __LINE__, errno); assert(rc==0); sndhwm = -9; size_t s = sizeof(int); rc = zmq_getsockopt(publisher, ZMQ_SNDHWM, sndhwm, s); if(rc!=0) printf(%s:%d zmq_getsockopt() errno %d!\n, __FILE__, __LINE__, errno); printf(sndhwm: %d\n, sndhwm); assert(rc==0); rc = zmq_bind(publisher, SWITCH_OUT); if(rc!=0) cerr __FILE__ : __LINE__ zmq_bind() errno errno endl; assert(rc == 0); Here's gdb output(hwm is 1000. This doesn't match my expectation): (gdb) break pipe.cpp:157 Breakpoint 2 at 0x7fdbad9a7f30: file ../../src/pipe.cpp, line 157. (gdb) c Continuing. Breakpoint 2, zmq::pipe_t::check_write (this=0x134c8c0) at ../../src/pipe.cpp:157 157 in ../../src/pipe.cpp (gdb) bt #0 zmq::pipe_t::check_write (this=0x134c8c0) at ../../src/pipe.cpp:157 #1 0x7fdbad9a95a3 in zmq::pipe_t::write (this=0x134c8c0, msg_=0x7fff1280abf0) at ../../src/pipe.cpp:166 #2 0x7fdbad99be01 in zmq::dist_t::write (this=0x1302cb8, pipe_=0x134c8c0, msg_=0x3e8) at ../../src/dist.cpp:181 #3 0x7fdbad99bf93 in zmq::dist_t::distribute (this=0x1302cb8, msg_=0x7fff1280abf0, flags_=value optimized out) at ../../src/dist.cpp:145 #4 0x7fdbad99c17e in zmq::dist_t::send_to_matching (this=0x1302cb8, msg_=0x7fff1280abf0, flags_=2) at ../../src/dist.cpp:118 #5 0x7fdbad9bc2be in zmq::xpub_t::xsend (this=0x1302900, msg_=0x7fff1280abf0, flags_=2) at ../../src/xpub.cpp:129 #6 0x7fdbad9b02d0 in zmq::socket_base_t::send (this=0x1302900, msg_=0x7fff1280abf0, flags_=2) at ../../src/socket_base.cpp:656 #7 0x7fdbad9c365a in s_sendmsg (s_=0x1302900, msg_=0x7fff1280abf0, flags_=2) at ../../src/zmq.cpp:337 #8 0x7fdbad9c3c7e in zmq_send (s_=0x1302900, buf_=0x12f4110, len_=2, flags_=2) at ../../src/zmq.cpp:362 #9 0x00404810 in main (argc=value optimized out, argv=value optimized out) at ../src/switch.cpp:395 (gdb) p hwm $1 = 1000 (gdb) p lwm $2 = 500 ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
