Re: [zeromq-dev] zmq::ctx_t::terminate hang
Thank you very much, it works.
muziwind
- Original Message -
From: Michel Pelletier
To: muziw...@sina.com, ZeroMQ development list
Subject: Re: [zeromq-dev] zmq::ctx_t::terminate hang
Date: 2014-08-08 21:12
Check out the ZMQ_LINGER option:
http://api.zeromq.org/4-0:zmq-setsockopt
More information is also in the guide.
-Michel
On Fri, Aug 8, 2014 at 12:56 AM, wrote:
> hi all
>
> zmq I used is zeromq-4.0.4.tar.gz
>
> I met a problem. I have a server and client.
>
> After close server I still use client send message, then I close client UI,
> the client will hang. I call this ConditionA
>
> gdb show the
>
> #0 0x003e9c2d7e33 in poll () from /lib64/libc.so.6
> #1 0x76720066 in zmq::signaler_t::wait (this=0x17dd928,
> timeout_=-1) at signaler.cpp:173
> #2 0x7671117e in zmq::mailbox_t::recv (this=0x17dd8c8,
> cmd_=0x7fffb830, timeout_=-1) at mailbox.cpp:72
> #3 0x7670886e in zmq::ctx_t::terminate (this=0x17dd830) at
> ctx.cpp:142
> #4 0x767359a0 in zmq_ctx_term (ctx_=0x17dd830) at zmq.cpp:171
> #5 0x77bb6f4d in zmq::context_t::close (this=0x1c7ab80) at
> /home/dev/TigerTeam/PAISetup/PAI/include/zmq/zmq.hpp:304
> #6 0x77bb6f24 in zmq::context_t::~context_t (this=0x1c7ab80,
> __in_chrg=) at
> /home/dev/TigerTeam/PAISetup/PAI/include/zmq/zmq.hpp:297
> #7 0x77bb5785 in tiger::MessageCenter::~MessageCenter
> (this=0x77de67a0, __in_chrg=) at
> MessageCenter.cpp:319
> #8 0x003e9c235fd2 in exit () from /lib64/libc.so.6
> #9 0x003e9c21ec64 in __libc_start_main () from /lib64/libc.so.6
> #10 0x00405f49 in _start ()
>
>
>
> but if I not use client to send message after close server, then i close
> client UI, the client can quit normally. I call this ConditionB
>
>
>
> And on the client, I use a thread to do send, like this
>
>
>
> while(quit ==false)
>
> {
>
> send();
>
> while(quit == false)
>
> {
>
> poll(items, 1, 1000);
>
> if (items[0] & ZMQ_POLLIN)
>
> {
>
> recv();
>
> }
>
> }
>
> }
>
>
>
>
> and I found something during debug
>
> ConditionB on exit, these breakpoints happen in order.
>
> 1.
> #0 zmq::object_t::send_reap (this=0x2478d50, socket_=0x2478d50) at
> object.cpp:303
> #1 0x7ffb78eb6a46 in zmq::socket_base_t::close (this= out>) at socket_base.cpp:837
> #2 0x7ffb78ecb7f0 in zmq_close (s_=0x2478d50) at zmq.cpp:264
> #3 0x7ffb7a34d085 in zmq::socket_t::close (this=0x24817c0) at
> /home/dev/TigerTeam/PAISetup/PAI/include/zmq/zmq.hpp:365
> #4 0x7ffb7a3499e3 in tiger::BroadcastThread::~BroadcastThread
> (this=0x23e47c0, __in_chrg=) at MessageCenter.cpp:108
>
>
> 2.
> #0 zmq::object_t::send_stop (this=0x2478d50) at object.cpp:176
> #1 0x7ffb78e9e821 in zmq::ctx_t::terminate (this=0x23a3020) at
> ctx.cpp:134
> #2 0x7ffb78ecb9a0 in zmq_ctx_term (ctx_=0x23a3020) at zmq.cpp:171
> #3 0x7ffb7a34cf4d in zmq::context_t::close (this=0x211df40) at
> /home/dev/TigerTeam/PAISetup/PAI/include/zmq/zmq.hpp:304
> #4 0x7ffb7a34cf24 in zmq::context_t::~context_t (this=0x211df40,
> __in_chrg=) at
> /home/dev/TigerTeam/PAISetup/PAI/include/zmq/zmq.hpp:297
> #5 0x7ffb7a34b785 in tiger::MessageCenter::~MessageCenter
> (this=0x7ffb7a57c7a0, __in_chrg=) at
> MessageCenter.cpp:319
>
>
> 3.
> #0 zmq::object_t::send_stop (this=0x24817e0) at object.cpp:176
> #1 0x7ffb78e9cee1 in zmq::ctx_t::destroy_socket (this=0x23a3020,
> socket_=0x2478d50) at ctx.cpp:314
> #2 0x7ffb78eb67ee in zmq::socket_base_t::check_destroy (this=0x2478d50)
> at socket_base.cpp:1028
> #3 0x7ffb78ea3482 in zmq::epoll_t::loop (this=0x244da00) at
> epoll.cpp:165
> #4 0x7ffb78ec2e3b in thread_routine (arg_=0x244da70) at thread.cpp:81
> #5 0x003e9ce077e1 in start_thread () from /lib64/libpthread.so.0
> #6 0x003e9c2e153d in clone () from /lib64/libc.so.6
>
>
>
> but ConditionA on exit, only 1 and 2 happen.
>
>
>
> does anyone have clues or experiences about this?
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Samples of ZMTP?
https://github.com/zeromq/libzmtp is a portable c implementation. https://github.com/zeromq/zmtp-java is a rough java implementation that _only_ handles negotiation. On Mon, Aug 11, 2014 at 5:04 PM, Stéphane Wirtel wrote: > Here is the reference: http://rfc.zeromq.org/spec:37 > > On 11 Aug 2014, at 22:01, Stéphane Wirtel wrote: > >> Hi all, >> >> Based on the ABNF grammar, I am looking for samples of ZMTP. >> >> Do you have some examples because I would like to test my implementation >> of the parser. >> >> Regards, >> > -- > Stéphane Wirtel - http://wirtel.be - @matrixise > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zeromq, abort(), and high reliability environments
On Mon, Aug 11, 2014 at 5:15 PM, Dylan Cali wrote: >> Must not throw an assertion there. An exception in the highlevel >> bindings (if the language has any) could be ok. That's a matter of >> taste and language style. > > The problem with assertions/aborts from a user's perspective is that > they cannot be caught like a 'real' Exception. So, even with the jzmq > bindings there are some situations (e.g. out of file descriptors) > where libzmq will abort and take an entire Java application with it, > giving it no chance for cleanup/recovery. The situations where 0mq asserts cannot be caught anyway, because they occur in the background thread. There is no "caller" to raise an exception into. As Thomas pointed out, it is unlikely that the situation is recoverable anyway. The only simple thing 0mq can do is log and abort. > My users say they don't feel comfortable using zeromq (or a client > library that uses zeromq), in a high reliability application because > of the possibility it will abort. That possibility always exists and their sense of comfort is an illusion whether you use 0mq or not. There are many internal and external reasons for systems to fail. 0mq aborting may actually be one of the least probable terminal events to occur in your application. -Michel ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zeromq, abort(), and high reliability environments
What exactly would an application do in the face of exhausting file descriptors to recover? I have encountered numerous cases with the JVM where it's notion of finalization is decidedly too lazy to be of practical use (memory mapped files/regions for instance). I've also encountered several instances where bugs in Hotspot have taken out a process. Linux has it's OOM killer feature which gives you no chance to recover. Many (perhaps most) of these situations are simply unrecoverable in any practical sense except for terminating the process and letting some supervisor restart the process. In my mind, this is a completely distinct issue from whether or not the library uses asserts vs EINVAL to report incorrect usage. I tend to prefer the latter in libraries I don't directly control, but I think it is incumbent upon a given language binding to enforce correct usage and report incorrect usage in the way that is most idiomatic to the language in question. In other-words, if the wrapper wants recoverable argument validation, then it is the wrapper's responsibility to implement that; assert() (and it's variants) are completely reasonable idioms for C and C++ pre-condition enforcement. On Mon, Aug 11, 2014 at 7:15 PM, Dylan Cali wrote: > > Must not throw an assertion there. An exception in the highlevel > > bindings (if the language has any) could be ok. That's a matter of > > taste and language style. > > The problem with assertions/aborts from a user's perspective is that > they cannot be caught like a 'real' Exception. So, even with the jzmq > bindings there are some situations (e.g. out of file descriptors) > where libzmq will abort and take an entire Java application with it, > giving it no chance for cleanup/recovery. > > My users say they don't feel comfortable using zeromq (or a client > library that uses zeromq), in a high reliability application because > of the possibility it will abort. > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zeromq, abort(), and high reliability environments
> Must not throw an assertion there. An exception in the highlevel > bindings (if the language has any) could be ok. That's a matter of > taste and language style. The problem with assertions/aborts from a user's perspective is that they cannot be caught like a 'real' Exception. So, even with the jzmq bindings there are some situations (e.g. out of file descriptors) where libzmq will abort and take an entire Java application with it, giving it no chance for cleanup/recovery. My users say they don't feel comfortable using zeromq (or a client library that uses zeromq), in a high reliability application because of the possibility it will abort. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Samples of ZMTP?
Here is the reference: http://rfc.zeromq.org/spec:37 On 11 Aug 2014, at 22:01, Stéphane Wirtel wrote: > Hi all, > > Based on the ABNF grammar, I am looking for samples of ZMTP. > > Do you have some examples because I would like to test my implementation > of the parser. > > Regards, > -- Stéphane Wirtel - http://wirtel.be - @matrixise ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Samples of ZMTP?
Hi all, Based on the ABNF grammar, I am looking for samples of ZMTP. Do you have some examples because I would like to test my implementation of the parser. Regards, Stephane -- Stéphane Wirtel - http://wirtel.be - @matrixise ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Custom authenticator
Awesome. My handler seems to be getting messages. Now to get the rest of the design build around it. Thanks for the help! On Sat, Aug 9, 2014 at 5:49 PM, Pieter Hintjens wrote: > You can look at the test cases in libzmq to see examples of custom ZAP > handlers (CZMQ is only one option, you can indeed write your own > handlers). > > As for signing keys... there's been a long thread on certificate > formats on this list, the upshot wasn't clear. > > On Fri, Aug 8, 2014 at 10:44 PM, Charles West wrote: > > Also, does anyone know of a good way to sign using CurveZMQ keys? I > could > > bind them to a second key (used for signing) using a permission signed > by a > > certificate authority, but that seems clunky. > > > > > > > > On Fri, Aug 8, 2014 at 4:42 PM, Charles West wrote: > >> > >> Hello, > >> > >> I've been digging into the spec for CurveZMQ as part of my efforts to > >> build a secure alternative to ROS. I believe I have figured out what I > need > >> to do for the next part, but I thought I should ask to see if I am on > the > >> right track and see if there might be better ways that more experienced > >> people know of. > >> > >> I need to maintain an in-memory list of accepted keys for each socket > and > >> have connections for each of those sockets accepted/rejected based on > the > >> associated key stores. > >> > >> It looks like once security domains are implemented I will be able to > make > >> something of this nature by creating a security domain for each socket > and a > >> folder to maintain the allowed certificates for each domain. In the > mean > >> time, I could have a context for each socket and its own associated > folder > >> (clunky, but works). However, as this is suppose to be a background > >> library, it would be much better if it didn't need to have a folder with > >> write access to do its own book keeping. > >> > >> 27/ZAP - ZeroMQ Authentication Protocol and looking at the source for > CZMQ > >> seems to indicate a better way. If I am reading it correctly, ZeroMQ > will > >> send any connection requests over to an inproc server with endpoint > >> "inproc://zeromq.zap.01". This server is normally made automatically by > >> CZMQ calls, but it is not necessary that the library creates it. > Instead, > >> my code could bind the endpoint and implement its part of the 27/ZAP > >> protocol (the curve part, at least). It can maintain its own list of > keys > >> and implement the security domains to allow a unique in-memory store to > be > >> kept for each object. > >> > >> If I may ask, does this last solution sound right? Is there any better > >> way to do it? > >> > >> Thank you for your time, > >> Charlie West > > > > > > > > ___ > > zeromq-dev mailing list > > zeromq-dev@lists.zeromq.org > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] forwarding a message with metadata
> > 1) I want to use CURVE and zmq_msg_gets() > > When I recv() a message on the ROUTER socket it has metadata attached > to it. Can I (and how) forward this message across the PAIR socket savely > keeping the metadata pointer intact so the app can still call > zmq_msg_gets() on it? caveat; I haven't tried it, but I did not see anything obvious in the implementation that would preclude metadata attached to a message from being forwarded over a PAIR socket by simply sending the received message. The underlying pipe just copies the msg struct by-value to a queue then calls init(), resetting the supplied message to an empty state, without messing with the refcount on the attached metadata. The receiving side will take ownership of the associated metadata reference when it dequeues the message. On Mon, Aug 11, 2014 at 10:14 AM, Goswin von Brederlow wrote: > Hi, > > I hope you don't mind but I want to brainstorm a bit. > > I want a generic solution to handle heartbeats that I can reuse in > different applications. The most transparent way for this seems to be > to create a seperate thread for it. So the design looks something like > this: > > [client] PAIR <--inproc--> [heartbeat thread] PAIR-DEALER <--tcp--> > > and > > <--tcp--> ROUTER-PAIR [heartbeat thread] <--inproc--> PAIR [server] > > > The heartbeat thread just forwards messages between the app and the > outside world and back monitoring the traffic. And when there is no > traffic it inserts heartbeats at regular intervals. It also filters > out incoming heartbeats from the outside world. > > So far this is pretty simple. But now for the interesting parts: > > 1) I want to use CURVE and zmq_msg_gets() > > When I recv() a message on the ROUTER socket it has metadata attached > to it. Can I (and how) forward this message across the PAIR socket savely > keeping the metadata pointer intact so the app can still call > zmq_msg_gets() on it? > > 2) I want to report disapeared peers to the app > > Basically I have normal messages and control/monitoring messages. I've > been thinking that on a ROUTER socket each message starts with an > identity frame. So I could send a 0 frame to indicate the message is a > control message. > > But what do I do with the client? I can't use the same with a DEALER > socket. I would have to prefix every message with 0 (control) or 1 > (data) and that would need change the application. I couldn't just > plug in or remove the heartbeat without rewriting the message parsing. > > A short time ago we talked about using some bit in the message itself > to say wether it is a control message or normal and having in-band > control messages for connect/dicsonnect and so on. Maybe it would be a > good time to design and implement something like that now and use it > here? > > 2b) Combining 2 and 1. Can I set my own metadata for control messages? > > I think the lack of a User-Id on a message could be used to identify > control messages. Right? > > 3) I want to connect/disconnect/bind/unbind or set HWM and so on > > The app and the DEALER/ROUTER sockets are in different threads. This > makes modifying them directly impossible (never share a socket between > threads, right?). > > So this would also need some form of control message but going the > other way this time. > > Should this be just a custom control message that is specific to my > use case or would it make sense to define a set of control messages > for all the socket operations in zmq so other use cases use the same > syntax? I'm thinking this could be reusable for proxies. > > MfG > Goswin > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zeromq, abort(), and high reliability environments
On Mon, Aug 11, 2014 at 04:37:18PM +0200, Goswin von Brederlow wrote: > On Mon, Aug 11, 2014 at 04:14:19PM +0200, Pieter Hintjens wrote: > > On Mon, Aug 11, 2014 at 11:33 AM, Goswin von Brederlow > > wrote: > > > > > My suggestion is that if you find an assertion that gets triggered > > > then patch it out and handle the error properly and send a pull > > > request for the fix. > > > > Respectfully disagree. Exceptions indicate unrecoverable failure of > > one kind or another. The fix depends on the case. > > > > CZMQ fwiw uses exceptions to check arguments, e.g. asserts if caller > > passes NULL when not allowed. This is extremely effective. If the > > application is misusing the API then it's incapable of handling error > > codes. > > If the caller passes NULL when not allowed that is a bug. So you can > assert there. That is not what I ment. > > What was ment is like when a lowlevel recv() call returns EAGAIN > because some signal occured. zmq must not throw an exception there. Must not throw an assertion there. An exception in the highlevel bindings (if the language has any) could be ok. That's a matter of taste and language style. > Signals do happen from time to time and zmq must deal with syscalls > getting interrupted (which it does). > > > I've started adding more aggressive CZMQ-style exceptions to libzmq as > > well, for the options API, enabled with the --with-militant configure > > switch. > > > > -Pieter > > MfG > Goswin > > PS: Personally I prever an error with EINVAL to an assertion failure > on bad arguments. Anything recoverable should not abort(). Easier for > bindings to deal with in a meaningfull way. > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] forwarding a message with metadata
Hi, I hope you don't mind but I want to brainstorm a bit. I want a generic solution to handle heartbeats that I can reuse in different applications. The most transparent way for this seems to be to create a seperate thread for it. So the design looks something like this: [client] PAIR <--inproc--> [heartbeat thread] PAIR-DEALER <--tcp--> and <--tcp--> ROUTER-PAIR [heartbeat thread] <--inproc--> PAIR [server] The heartbeat thread just forwards messages between the app and the outside world and back monitoring the traffic. And when there is no traffic it inserts heartbeats at regular intervals. It also filters out incoming heartbeats from the outside world. So far this is pretty simple. But now for the interesting parts: 1) I want to use CURVE and zmq_msg_gets() When I recv() a message on the ROUTER socket it has metadata attached to it. Can I (and how) forward this message across the PAIR socket savely keeping the metadata pointer intact so the app can still call zmq_msg_gets() on it? 2) I want to report disapeared peers to the app Basically I have normal messages and control/monitoring messages. I've been thinking that on a ROUTER socket each message starts with an identity frame. So I could send a 0 frame to indicate the message is a control message. But what do I do with the client? I can't use the same with a DEALER socket. I would have to prefix every message with 0 (control) or 1 (data) and that would need change the application. I couldn't just plug in or remove the heartbeat without rewriting the message parsing. A short time ago we talked about using some bit in the message itself to say wether it is a control message or normal and having in-band control messages for connect/dicsonnect and so on. Maybe it would be a good time to design and implement something like that now and use it here? 2b) Combining 2 and 1. Can I set my own metadata for control messages? I think the lack of a User-Id on a message could be used to identify control messages. Right? 3) I want to connect/disconnect/bind/unbind or set HWM and so on The app and the DEALER/ROUTER sockets are in different threads. This makes modifying them directly impossible (never share a socket between threads, right?). So this would also need some form of control message but going the other way this time. Should this be just a custom control message that is specific to my use case or would it make sense to define a set of control messages for all the socket operations in zmq so other use cases use the same syntax? I'm thinking this could be reusable for proxies. MfG Goswin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zeromq, abort(), and high reliability environments
On Mon, Aug 11, 2014 at 04:14:19PM +0200, Pieter Hintjens wrote: > On Mon, Aug 11, 2014 at 11:33 AM, Goswin von Brederlow > wrote: > > > My suggestion is that if you find an assertion that gets triggered > > then patch it out and handle the error properly and send a pull > > request for the fix. > > Respectfully disagree. Exceptions indicate unrecoverable failure of > one kind or another. The fix depends on the case. > > CZMQ fwiw uses exceptions to check arguments, e.g. asserts if caller > passes NULL when not allowed. This is extremely effective. If the > application is misusing the API then it's incapable of handling error > codes. If the caller passes NULL when not allowed that is a bug. So you can assert there. That is not what I ment. What was ment is like when a lowlevel recv() call returns EAGAIN because some signal occured. zmq must not throw an exception there. Signals do happen from time to time and zmq must deal with syscalls getting interrupted (which it does). > I've started adding more aggressive CZMQ-style exceptions to libzmq as > well, for the options API, enabled with the --with-militant configure > switch. > > -Pieter MfG Goswin PS: Personally I prever an error with EINVAL to an assertion failure on bad arguments. Anything recoverable should not abort(). Easier for bindings to deal with in a meaningfull way. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] get socket condition
On Mon, Aug 11, 2014 at 05:47:14PM +0800, bino oetomo wrote: > Dear All, > c/q Goswin von Brederlow .. Really appreciate your response > On Mon, August 11, 2014 5:36 pm, Goswin von Brederlow wrote: > > There are lots of ways the underlying tcp socket can die. But zeromq > > will reconnect the socket again and again and again. > > > > Only problem is that you can loose messages when it dies with messages > > in flight. Zeromq has reliable delivery (if a message arrives then it is > > all of a message), not garanteed delivery. > > > > So . .For now .. all I need to concern is just the HWM ? > > Sincerely > -bino- A PUSH socket will block when the HWM is reached or return EAGAIN with ZMQ_DONTWAIT. Easy enough to handle. MfG Goswin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] inproc connect before bind req rep
Hi, in libzmq/tests/test_inproc_connect.cpp there are connect-before-bind-tests for PAIR-PAIR and PUB-SUB but not for REQ-REP. It seems that this is still not working. I tested it and the REQ-socket connected but silently dropped the message. Are there any plans to fix that? - Werner ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zeromq, abort(), and high reliability environments
On Mon, Aug 11, 2014 at 11:33 AM, Goswin von Brederlow wrote: > My suggestion is that if you find an assertion that gets triggered > then patch it out and handle the error properly and send a pull > request for the fix. Respectfully disagree. Exceptions indicate unrecoverable failure of one kind or another. The fix depends on the case. CZMQ fwiw uses exceptions to check arguments, e.g. asserts if caller passes NULL when not allowed. This is extremely effective. If the application is misusing the API then it's incapable of handling error codes. I've started adding more aggressive CZMQ-style exceptions to libzmq as well, for the options API, enabled with the --with-militant configure switch. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] get socket condition
Dear All, c/q Goswin von Brederlow .. Really appreciate your response On Mon, August 11, 2014 5:36 pm, Goswin von Brederlow wrote: > There are lots of ways the underlying tcp socket can die. But zeromq > will reconnect the socket again and again and again. > > Only problem is that you can loose messages when it dies with messages > in flight. Zeromq has reliable delivery (if a message arrives then it is > all of a message), not garanteed delivery. > So . .For now .. all I need to concern is just the HWM ? Sincerely -bino- ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] get socket condition
On Mon, Aug 11, 2014 at 05:10:11PM +0800, bino oetomo wrote:
>
>
> Dear all ...
>
> Let's say I (using python) have simple :
>
> ctx = zmq.Context()
> socket=ctx.socket(zmq.PUSH)
> socket.setsockopt(zmq.SNDHWM, 10)
> socket.connect('tcp://127.0.0.1:9001')
>
> I knew that when HWM reach, I'll get EAGAIN exception.
>
> But, is there any posibilities that a socket is accidently/silenty
> crash/dead ?
>
> If so ... what is the error code ?
>
> Is there any docs that explain zmq.ZMQError ? I mean :
> - What error code (int)
> - What error name
> - Meaning of the error, or what caused the error ?
>
>
> Sincerely
> -bino-
There are lots of ways the underlying tcp socket can die. But zeromq
will reconnect the socket again and again and again.
Only problem is that you can loose messages when it dies with messages
in flight. Zeromq has reliable delivery (if a message arrives then it
is all of a message), not garanteed delivery.
MfG
Goswin
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zeromq, abort(), and high reliability environments
On Mon, Aug 11, 2014 at 07:52:02AM +0100, Gerry Steele wrote: > How about not sending an ack to your users until the unit of work they > input has cleared the pipeline? That way the input application can decide > what to do. Obviously depends on your application... What if the input application gets the SIGABRT? Zeromq should imho never fail an assertion. That should be reserved for bugs, not exceptional circumstances. Although with out of memory the application simply gets killed by the OOM killer or gets a segfault due to memory overcommit. There isn't much you can do there. My suggestion is that if you find an assertion that gets triggered then patch it out and handle the error properly and send a pull request for the fix. MfG Goswin > On 9 Aug 2014 03:12, "Dylan Cali" wrote: > > > Hey guys, > > > > What is the right way to use zeromq in high reliability environments? In > > certain insane/impossible situations (e.g. out of memory, out of file > > descriptors, etc) libzmq assertions will fail and it will abort. > > > > I came across a thread by Martin where he addresses a similar situation > > [1]. If > > I'm reading his argument correctly, the gist in general is: If it's > > impossible > > to connect due to some error, than you're dead in the water anyways. Crash > > loudly and immediately with the error (the Fail-Fast paradigm), fix the > > error, > > and then restart the process. > > > > I actually agree with this philosophy, but a user would say "You > > terminated my > > entire application stack and didn't give me a chance to cleanup! I had > > very important data > > in memory and it's gone!" This is especially the case with Java > > programmers who > > Always Expect an Exception. > > > > For example, in the case of being out of file descriptors, the jzmq > > bindings will abort, > > but a Java programmer would expect to get an Exception with the "Too Many > > Open > > Files" error. > > > > I guess one possible retort is: if the data in memory was so important, why > > didn't you have redundancy/failover/some kind of playback log? Why did you > > put > > all your eggs in one basket assuming your process would never crash? > > > > Is that the right answer here (basically blame the user for not having > > disaster > > recovery), or is there a different/better way to address the high > > reliability > > scenario? > > > > I came across another thread where Martin gets this very > > complaint (zeromq aborted my application!), and basically says well, if > > you really, really want to, > > you can install a signal handler for SIGABRT, but caveat emptor [2]. > > > > To me, this is playing with fire, dangerous, and just a Bad Idea. But > > maybe it's > > worth the risk in high reliability environments? > > > > > > Thanks in advance for any advice or thoughts. > > > > [1] http://lists.zeromq.org/pipermail/zeromq-dev/2009-May/000784.html > > [2] http://lists.zeromq.org/pipermail/zeromq-dev/2011-October/013608.html MfG Goswin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] get socket condition
Dear all ...
Let's say I (using python) have simple :
ctx = zmq.Context()
socket=ctx.socket(zmq.PUSH)
socket.setsockopt(zmq.SNDHWM, 10)
socket.connect('tcp://127.0.0.1:9001')
I knew that when HWM reach, I'll get EAGAIN exception.
But, is there any posibilities that a socket is accidently/silenty
crash/dead ?
If so ... what is the error code ?
Is there any docs that explain zmq.ZMQError ? I mean :
- What error code (int)
- What error name
- Meaning of the error, or what caused the error ?
Sincerely
-bino-
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
