Re: qpid-0.20 Execution exception: resource-locked
On 05/12/16 22:38, Bee wrote: Thank you Steve and Gordon for your suggestion. Is this meaning the client tries to reconnect the first time, but gets the explicit error, then it stops trying to reconnect? Yes, because it gets an explicit error returned from the broker. Can you explain how increasing reconnect_interval_min would help if the first reconnection attempt is supposed to happen immediately? The error happens because of a race between the client reconnecting and the broker handling the loss of the earlier client connection. By delaying the first reconnect attempt the idea is to give the broker a little more time so that hopefully it has processed the loss of the earlier connection before the reconnect attempt. It's not foolproof of course, for that you would have to handle the exception and explicitly reconnect yourself. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: qpid-0.20 Execution exception: resource-locked
Thank you Steve and Gordon for your suggestion. Is this meaning the client tries to reconnect the first time, but gets the explicit error, then it stops trying to reconnect? Can you explain how increasing reconnect_interval_min would help if the first reconnection attempt is supposed to happen immediately? From: Gordon Sim <g...@redhat.com> Sent: Monday, December 5, 2016 12:26 PM To: users@qpid.apache.org Subject: Re: qpid-0.20 Execution exception: resource-locked On 05/12/16 19:41, Bee wrote: > Hello, I'm quite new here. Hopefully, I can get some help for qpid-0.20 I'm > using at work. > > A few times now, I get this error: > [Protocol] error Execution exception: resource-locked: Cannot grant exclusive > access to queue (qpid/broker/SessionAdapter.cpp:324) > > Please correct me, but my understanding is this is a race condition where > connection unexpectedly close, broker-client tries to re-initiate the > session, but the exclusive queue has not been freed yet. > When this happens, client or broker (I'm not sure) stops trying to establish > the session again. It will happen when the broker thinks there is already a session using that queue. One possibility is that the client timed out due not not receiving a heartbeat before the broker did. Because the client gets an explicit error, it does not try to reconnect (it only does that if the connection is lost). > My question is, how can I ensure the session can re-establish again by > itself. And, how can I reproduce this issue. You could try sending the SIGSTOP signal to the broker. That prevents the broker sending out heartbeats, then send SIGCONT after 2*heartbeat interval so that the broker wakes up in time to process the reconnect, but may not have processed the disconnect fully. You'll need to run it several times and perhaps vary the time between stop and cont a little to try and reproduce. > I've tried iptables, it seems to reconnect as it's supposed to. I've tried > qpid-config del queue, I get msg that queue is deleted, but not the same > error msg. > > I'm using these options for broker > reconnect: true > heartbeat: 1 > tcp_nodelay: true > reconnect_timeout: 2.0 Just in passing, the reconnect_timeout is the maximum total time that the client will spend trying to reconnect. 2 seconds seems like a small value for that. > reconnect_interval_min: 0.1 > reconnect_interval_max: 2.0 > > > Any insights will be very appreciated. Steve's suggestion is probably the simplest, i.e. wait a bit longer before the first reconnect attempt to try and avoid the race. In recent versions of the API, there is an explicit reconnect() method (assuming you are using c++ qpid::messaging of course) on Connection that you could use when catching this particular error. Not available on 0.20 though, so would need a newer client. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: qpid-0.20 Execution exception: resource-locked
On 05/12/16 19:41, Bee wrote: Hello, I'm quite new here. Hopefully, I can get some help for qpid-0.20 I'm using at work. A few times now, I get this error: [Protocol] error Execution exception: resource-locked: Cannot grant exclusive access to queue (qpid/broker/SessionAdapter.cpp:324) Please correct me, but my understanding is this is a race condition where connection unexpectedly close, broker-client tries to re-initiate the session, but the exclusive queue has not been freed yet. When this happens, client or broker (I'm not sure) stops trying to establish the session again. It will happen when the broker thinks there is already a session using that queue. One possibility is that the client timed out due not not receiving a heartbeat before the broker did. Because the client gets an explicit error, it does not try to reconnect (it only does that if the connection is lost). My question is, how can I ensure the session can re-establish again by itself. And, how can I reproduce this issue. You could try sending the SIGSTOP signal to the broker. That prevents the broker sending out heartbeats, then send SIGCONT after 2*heartbeat interval so that the broker wakes up in time to process the reconnect, but may not have processed the disconnect fully. You'll need to run it several times and perhaps vary the time between stop and cont a little to try and reproduce. I've tried iptables, it seems to reconnect as it's supposed to. I've tried qpid-config del queue, I get msg that queue is deleted, but not the same error msg. I'm using these options for broker reconnect: true heartbeat: 1 tcp_nodelay: true reconnect_timeout: 2.0 Just in passing, the reconnect_timeout is the maximum total time that the client will spend trying to reconnect. 2 seconds seems like a small value for that. reconnect_interval_min: 0.1 reconnect_interval_max: 2.0 Any insights will be very appreciated. Steve's suggestion is probably the simplest, i.e. wait a bit longer before the first reconnect attempt to try and avoid the race. In recent versions of the API, there is an explicit reconnect() method (assuming you are using c++ qpid::messaging of course) on Connection that you could use when catching this particular error. Not available on 0.20 though, so would need a newer client. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
RE: qpid-0.20 Execution exception: resource-locked
Try increasing your reconnect_interval_min to 1.0 > -Original Message- > From: Bee [mailto:nbe...@hotmail.com] > Sent: Monday, December 05, 2016 2:42 PM > To: users@qpid.apache.org > Subject: qpid-0.20 Execution exception: resource-locked > > Hello, I'm quite new here. Hopefully, I can get some help for qpid-0.20 I'm > using at work. > > A few times now, I get this error: > [Protocol] error Execution exception: resource-locked: Cannot grant > exclusive access to queue (qpid/broker/SessionAdapter.cpp:324) > > Please correct me, but my understanding is this is a race condition where > connection unexpectedly close, broker-client tries to re-initiate the session, > but the exclusive queue has not been freed yet. > When this happens, client or broker (I'm not sure) stops trying to establish > the session again. > > My question is, how can I ensure the session can re-establish again by itself. > And, how can I reproduce this issue. I've tried iptables, it seems to > reconnect > as it's supposed to. I've tried qpid-config del queue, I get msg that queue > is > deleted, but not the same error msg. > > I'm using these options for broker > reconnect: true > heartbeat: 1 > tcp_nodelay: true > reconnect_timeout: 2.0 > reconnect_interval_min: 0.1 > reconnect_interval_max: 2.0 > > > Any insights will be very appreciated. - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
