[jira] [Commented] (PROTON-1890) [c++] implement idle_timeout and heartbeats
[ https://issues.apache.org/jira/browse/PROTON-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482020#comment-17482020 ] Praveen Bodke commented on PROTON-1890: --- FYI, we are able to reproduce the issue with 0.36 version too. > [c++] implement idle_timeout and heartbeats > --- > > Key: PROTON-1890 > URL: https://issues.apache.org/jira/browse/PROTON-1890 > Project: Qpid Proton > Issue Type: Bug > Components: cpp-binding >Affects Versions: proton-0.16.0 >Reporter: Praveen Bodke >Assignee: Clifford Jansen >Priority: Major > Attachments: PROTON-1890.zip, examples.diff > > > This is similar issue reported in PROTON-1782 for ruby. > We are facing this issue in cpp binding and i am able to reproduce the issue > with scheduled_send_03.cpp example. The test scenario is to drop all the > packets from both the interfaces after the successful connection. The only > change i made to this example is to send messages continuously inside the > send() method. The other end is detecting the error as it is sending the > empty frames and no response is heard. > The proton is not sending the heartbeat messages (empty frames) as the sender > is busy in sending the data frames. Is it not necessary to send empty frames > even if the data frames are sent? > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-1900) Enabling the proton logs from a daemon process
[ https://issues.apache.org/jira/browse/PROTON-1900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17390114#comment-17390114 ] Praveen Bodke commented on PROTON-1900: --- Hello Alan, Did you get a chance to implement an API for C++ logging? C++ API would really help us in understanding the protocol and proton library in a better way and also help us in debugging the issues. Please let us know. Thank you. Regards Praveen. > Enabling the proton logs from a daemon process > -- > > Key: PROTON-1900 > URL: https://issues.apache.org/jira/browse/PROTON-1900 > Project: Qpid Proton > Issue Type: Improvement > Components: cpp-binding >Reporter: Praveen Bodke >Priority: Minor > Attachments: log_hack.cpp > > > Is there a way to enable the proton logs from a daemon process. > As of now, we are able to get the proton logs from PN_TRACE_FRM and other > environment variables for non-daemon process. > We looked into the proton library specially the CPP binding and could not > find any specific API's. We are using the proton version 0.16. > It would be very helpful if we can get the logs from a daemon process. > Thanks. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-1900) Enabling the proton logs from a daemon process
[ https://issues.apache.org/jira/browse/PROTON-1900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16566308#comment-16566308 ] Praveen Bodke commented on PROTON-1900: --- Thank you Allan. We really appreciate it. I will test it my daemon application and will update the ticket. Regards Praveen > Enabling the proton logs from a daemon process > -- > > Key: PROTON-1900 > URL: https://issues.apache.org/jira/browse/PROTON-1900 > Project: Qpid Proton > Issue Type: Improvement > Components: cpp-binding >Reporter: Praveen Bodke >Priority: Minor > Attachments: log_hack.cpp > > > Is there a way to enable the proton logs from a daemon process. > As of now, we are able to get the proton logs from PN_TRACE_FRM and other > environment variables for non-daemon process. > We looked into the proton library specially the CPP binding and could not > find any specific API's. We are using the proton version 0.16. > It would be very helpful if we can get the logs from a daemon process. > Thanks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Created] (PROTON-1900) Enabling the proton logs from a daemon process
Praveen Bodke created PROTON-1900: - Summary: Enabling the proton logs from a daemon process Key: PROTON-1900 URL: https://issues.apache.org/jira/browse/PROTON-1900 Project: Qpid Proton Issue Type: Improvement Components: cpp-binding Reporter: Praveen Bodke Is there a way to enable the proton logs from a daemon process. As of now, we are able to get the proton logs from PN_TRACE_FRM and other environment variables for non-daemon process. We looked into the proton library specially the CPP binding and could not find any specific API's. We are using the proton version 0.16. It would be very helpful if we can get the logs from a daemon process. Thanks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-1890) [c++] implement idle_timeout and heartbeats
[ https://issues.apache.org/jira/browse/PROTON-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16545633#comment-16545633 ] Praveen Bodke commented on PROTON-1890: --- Thanks for the update Alan. I have tested the same reproducer in 0.24 version and i see that the issue is fixed. >From the Frame logs its visible that after the close frame is sent, the proton >immediately sending the EOS frame to close the socket. Following are the frame log snippets. [0x98780f0]:0 -> (EMPTY FRAME) push-while-in push-while-in [0x98780f0]:0 -> (EMPTY FRAME) push-while-in push-while-in push-while-in [0x98780f0]:0 -> (EMPTY FRAME) push-while-in push-while-in [0x98780f0]:0 -> (EMPTY FRAME) push-while-in push-while-in push-while-in [0x98780f0]:0 -> @close(24) [error=@error(29) [condition=:"amqp:resource-limit-exceeded", description="local-idle-timeout expired"]] [0x98780f0]: -> EOS amqp:resource-limit-exceeded: local-idle-timeout expired Is there a way to enable proton logs from the CPP code instead of using the PN_TRACE_FRM environment variable. My application is running under daemon and will not have the console to display the logs. > [c++] implement idle_timeout and heartbeats > --- > > Key: PROTON-1890 > URL: https://issues.apache.org/jira/browse/PROTON-1890 > Project: Qpid Proton > Issue Type: Bug > Components: cpp-binding >Affects Versions: 0.16.0 >Reporter: Praveen Bodke >Assignee: Alan Conway >Priority: Major > Attachments: PROTON-1890.zip, examples.diff > > > This is similar issue reported in PROTON-1782 for ruby. > We are facing this issue in cpp binding and i am able to reproduce the issue > with scheduled_send_03.cpp example. The test scenario is to drop all the > packets from both the interfaces after the successful connection. The only > change i made to this example is to send messages continuously inside the > send() method. The other end is detecting the error as it is sending the > empty frames and no response is heard. > The proton is not sending the heartbeat messages (empty frames) as the sender > is busy in sending the data frames. Is it not necessary to send empty frames > even if the data frames are sent? > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-1890) [c++] implement idle_timeout and heartbeats
[ https://issues.apache.org/jira/browse/PROTON-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542326#comment-16542326 ] Praveen Bodke commented on PROTON-1890: --- Thanks for the update Alan. We really appreciate your quick verification. I > [c++] implement idle_timeout and heartbeats > --- > > Key: PROTON-1890 > URL: https://issues.apache.org/jira/browse/PROTON-1890 > Project: Qpid Proton > Issue Type: Bug > Components: cpp-binding >Affects Versions: 0.16.0 >Reporter: Praveen Bodke >Assignee: Alan Conway >Priority: Major > Attachments: PROTON-1890.zip > > > This is similar issue reported in PROTON-1782 for ruby. > We are facing this issue in cpp binding and i am able to reproduce the issue > with scheduled_send_03.cpp example. The test scenario is to drop all the > packets from both the interfaces after the successful connection. The only > change i made to this example is to send messages continuously inside the > send() method. The other end is detecting the error as it is sending the > empty frames and no response is heard. > The proton is not sending the heartbeat messages (empty frames) as the sender > is busy in sending the data frames. Is it not necessary to send empty frames > even if the data frames are sent? > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-1890) [c++] implement idle_timeout and heartbeats
[ https://issues.apache.org/jira/browse/PROTON-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541375#comment-16541375 ] Praveen Bodke commented on PROTON-1890: --- Update: After analyzing the proton frame logs at sender application P2 side, we observed that the proton is able to detect the idle-time-out expiry and sending a CLOSE frame to P1. But the CLOSE frame is not reached to P1 (due to the fact that this scenario is dropping all the packets between P1 and P2 interface to simulate the bad network) and eventually it is not letting the application layer know about this error until it receives EOS frame happens after the TCP re-transmission timeout (after ~16 minutes). The time gap between the following two FRAME logs is about ~16 minutes which is in line with TCP re-transmission timeout in our environment. [0x8c4ea60]:0 -> @close(24) [error=@error(29) [condition=:"amqp:resource-limit-exceeded", description="local-idle-timeout expired"]] [0x8c4ea60]: <- EOS amqp:resource-limit-exceeded: local-idle-timeout expired (connection aborted) I believe the proton library should handle such scenario in which if the CLOSE frame is sent due to idle-time-out error, it should trigger the on_transport_error callback to the handler. Please let me know your thoughts on this. Thanks. Praveen Bodke > [c++] implement idle_timeout and heartbeats > --- > > Key: PROTON-1890 > URL: https://issues.apache.org/jira/browse/PROTON-1890 > Project: Qpid Proton > Issue Type: Bug > Components: cpp-binding >Affects Versions: 0.16.0 >Reporter: Praveen Bodke >Priority: Major > Attachments: PROTON-1890.zip > > > This is similar issue reported in PROTON-1782 for ruby. > We are facing this issue in cpp binding and i am able to reproduce the issue > with scheduled_send_03.cpp example. The test scenario is to drop all the > packets from both the interfaces after the successful connection. The only > change i made to this example is to send messages continuously inside the > send() method. The other end is detecting the error as it is sending the > empty frames and no response is heard. > The proton is not sending the heartbeat messages (empty frames) as the sender > is busy in sending the data frames. Is it not necessary to send empty frames > even if the data frames are sent? > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-1890) [c++] implement idle_timeout and heartbeats
[ https://issues.apache.org/jira/browse/PROTON-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541345#comment-16541345 ] Praveen Bodke commented on PROTON-1890: --- Hello Robbie Gemmell, With some modifications to credit window, now i am able to reproduce the issue. Attaching the modified scheduled_send_03.cpp and server_direct.cpp programs. Please let me know if you need any more information. I would be happy to provide you. Let me go through the scenario here. STEP 1: Two peers P1 and P2 have established the connection and following are the frame information. server listening on :1239/examples [0x225f360]: <- AMQP [0x225f360]:0 <- @open(16) [container-id="b3d1fa29-84eb-4c1e-bd1e-2a11123b2a64", hostname="10.1.14.102", channel-max=32767, idle-time-out=15000] [0x225f360]:0 <- @begin(17) [next-outgoing-id=0, incoming-window=2147483647, outgoing-window=2147483647] [0x225f360]:0 <- @attach(18) [name="2b590c4a-e237-4522-81f1-8442f2f40844", handle=0, role=false, snd-settle-mode=2, rcv-settle-mode=0, source=@source(40) [durable=0, timeout=0, dynamic=false], target=@target(41) [address="example", durable=0, timeout=0, dynamic=false], initial-delivery-count=0, max-message-size=0] [0x225f360]: -> AMQP [0x225f360]:0 -> @open(16) [container-id="816b5f5c-4b6d-4e50-8a3e-096963ec5e26", channel-max=32767, idle-time-out=1] [0x225f360]:0 -> @begin(17) [remote-channel=0, next-outgoing-id=0, incoming-window=2147483647, outgoing-window=2147483647] [0x225f360]:0 -> @attach(18) [name="2b590c4a-e237-4522-81f1-8442f2f40844", handle=0, role=true, snd-settle-mode=2, rcv-settle-mode=0, source=@source(40) [durable=0, timeout=0, dynamic=false], target=@target(41) [durable=0, timeout=0, dynamic=false], initial-delivery-count=0, max-message-size=0] [0x225f360]:0 -> @flow(19) [next-incoming-id=0, incoming-window=2147483647, next-outgoing-id=0, outgoing-window=2147483647, handle=0, delivery-count=0, link-credit=100, drain=false] [0x225f360]:0 <- @attach(18) [name="2430291c-75ac-4a22-9e6d-dc5e179df431", handle=1, role=true, snd-settle-mode=2, rcv-settle-mode=0, source=@source(40) [address="X.X.X.X:1239/example", durable=0, timeout=0, dynamic=false], target=@target(41) [durable=0, timeout=0, dynamic=false], initial-delivery-count=0, max-message-size=0] [0x225f360]:0 <- @flow(19) [next-incoming-id=0, incoming-window=2147483647, next-outgoing-id=0, outgoing-window=2147483647, handle=1, delivery-count=0, link-credit=200, drain=false] P1 receiver credit window is set to 200, this means the P2 sender credit limit is 200. [0x225f360]:0 <- @flow(19) [next-incoming-id=0, incoming-window=2147483647, next-outgoing-id=0, outgoing-window=2147483647, handle=1, delivery-count=0, link-credit=200, drain=false] P2 receiver credit window is set to 100 [0x225f360]:0 -> @flow(19) [next-incoming-id=0, incoming-window=2147483647, next-outgoing-id=0, outgoing-window=2147483647, handle=0, delivery-count=0, link-credit=100, drain=false] STEP 2: P2 sends the messages continuously.. during this time frame, we drop all the packets between two peers. STEP 3: P1 detects the idle-time-out expiry after sending the some empty frames. [0x1a36360]:0 -> @flow(19) [next-incoming-id=2000, incoming-window=2147483647, next-outgoing-id=0, outgoing-window=2147483647, handle=0, delivery-count=2000, link-credit=100, drain=false] [0x1a36360]:0 -> @disposition(21) [role=true, first=1958, last=1999, settled=true, state=@accepted(36) []] [0x1a36360]:0 -> (EMPTY FRAME) [0x1a36360]:0 -> (EMPTY FRAME) [0x1a36360]:0 -> @close(24) [error=@error(29) [condition=:"amqp:resource-limit-exceeded", description="local-idle-timeout expired"]] [0x1a36360]: -> EOS amqp:resource-limit-exceeded: local-idle-timeout expired STEP 4: P2 still in ESTABLISHED state and never disconnects. [^PROTON-1890.zip] > [c++] implement idle_timeout and heartbeats > --- > > Key: PROTON-1890 > URL: https://issues.apache.org/jira/browse/PROTON-1890 > Project: Qpid Proton > Issue Type: Bug > Components: cpp-binding >Affects Versions: 0.16.0 >Reporter: Praveen Bodke >Priority: Major > Attachments: PROTON-1890.zip > > > This is similar issue reported in PROTON-1782 for ruby. > We are facing this issue in cpp binding and i am able to reproduce the issue > with scheduled_send_03.cpp example. The test scenario is to drop all the > packets from both the interfaces after the successful connection. The only > change i made to this example is to send messages continuously inside the > send() method. The other end is detecting the error as it is sending the > empty frames and no response is heard. > The proton is not sending the heartbeat messages (empty frames) as the sender > is busy in sending the data frames. Is it not necessary to se
[jira] [Updated] (PROTON-1890) [c++] implement idle_timeout and heartbeats
[ https://issues.apache.org/jira/browse/PROTON-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Praveen Bodke updated PROTON-1890: -- Attachment: PROTON-1890.zip > [c++] implement idle_timeout and heartbeats > --- > > Key: PROTON-1890 > URL: https://issues.apache.org/jira/browse/PROTON-1890 > Project: Qpid Proton > Issue Type: Bug > Components: cpp-binding >Affects Versions: 0.16.0 >Reporter: Praveen Bodke >Priority: Major > Attachments: PROTON-1890.zip > > > This is similar issue reported in PROTON-1782 for ruby. > We are facing this issue in cpp binding and i am able to reproduce the issue > with scheduled_send_03.cpp example. The test scenario is to drop all the > packets from both the interfaces after the successful connection. The only > change i made to this example is to send messages continuously inside the > send() method. The other end is detecting the error as it is sending the > empty frames and no response is heard. > The proton is not sending the heartbeat messages (empty frames) as the sender > is busy in sending the data frames. Is it not necessary to send empty frames > even if the data frames are sent? > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (PROTON-1890) [c++] implement idle_timeout and heartbeats
[ https://issues.apache.org/jira/browse/PROTON-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540489#comment-16540489 ] Praveen Bodke commented on PROTON-1890: --- Thank you Robbie Gemmell for clarifying the heartbeat information. We really appreciate that. As you pointed out "improper usage if it ties up the container thread", this was the real issue in sample program. Thanks for that. I have modified the program to have a different thread to send messages to a queue and the proton container thread to read from the queue only if it has a sufficient credit available. With this change, the proton is working great and detecting the idle-time-out expiry. Can you also clarify if there are any plans to improve the proton container to have a separate thread to listen for all the underlying proton errors and inform the application to close if necessary? - I am detailing the information about my test in case if some one is interested. Steps to reproduce: The test scenario is to check if the proton application is timesout if it did not hear from the peer 1. Two peers P1 (scheduled_send_03) and P2 (scheduled_send_03) have established the connection properly Following is the Frame information. (P1 idle time out is 30 seconds and P2 idle time out is 30 seconds) The advertised idle_time_out in open frame is generally half of the configured idle time out. [0x89873d0]: -> AMQP [0x89873d0]:0 -> @open(16) [container-id="a578196f-c4c1-415a-a055-032dc91b1635", hostname="X.X.X.X", channel-max=32767, idle-time-out=15000] [0x89873d0]:0 -> @begin(17) [next-outgoing-id=0, incoming-window=2147483647, outgoing-window=2147483647] [0x89873d0]:0 -> @attach(18) [name="8dc9567d-6840-4e03-931e-cd2df90b6378", handle=0, role=false, snd-settle-mode=2, rcv-settle-mode=0, source=@source(40) [durable=0, timeout=0, dynamic=false], target=@target(41) [address="example", durable=0, timeout=0, dynamic=false], initial-delivery-count=0, max-message-size=0] [0x89873d0]: <- AMQP [0x89873d0]:0 <- @open(16) [container-id="2a746915-26a9-4d33-ba7d-a2586d8a5452", channel-max=32767, idle-time-out=15000] [0x89873d0]:0 <- @begin(17) [remote-channel=0, next-outgoing-id=0, incoming-window=2147483647, outgoing-window=2147483647] [0x89873d0]:0 <- @attach(18) [name="8dc9567d-6840-4e03-931e-cd2df90b6378", handle=0, role=true, snd-settle-mode=2, rcv-settle-mode=0, source=@source(40) [durable=0, timeout=0, dynamic=false], target=@target(41) [durable=0, timeout=0, dynamic=false], initial-delivery-count=0, max-message-size=0] [0x89873d0]:0 <- @flow(19) [next-incoming-id=0, incoming-window=2147483647, next-outgoing-id=0, outgoing-window=2147483647, handle=0, delivery-count=0, link-credit=10, drain=false] P1 Netstat -- netstat -apn | grep 1239 tcp 116 0 X.X.X.X:48132 X.X.X.X:1239 ESTABLISHED 32682/./scheduled_s 2. enabled the traffic rule to drop the messages from both P1 and P2 interfaces 3. P1 is sending data messages continuously in a while loop 4. No packets arrived at P2, so P2 started sending empty frames. P2 detected the idle-time-out error and closed the connection. Note that messages from P2 is not reaching P1. server listening on X.X.X.X:1239/examples [0x130d360]: <- AMQP [0x130d360]:0 <- @open(16) [container-id="a578196f-c4c1-415a-a055-032dc91b1635", hostname="X.X.X.X", channel-max=32767, idle-time-out=15000] [0x130d360]:0 <- @begin(17) [next-outgoing-id=0, incoming-window=2147483647, outgoing-window=2147483647] [0x130d360]:0 <- @attach(18) [name="8dc9567d-6840-4e03-931e-cd2df90b6378", handle=0, role=false, snd-settle-mode=2, rcv-settle-mode=0, source=@source(40) [durable=0, timeout=0, dynamic=false], target=@target(41) [address="example", durable=0, timeout=0, dynamic=false], initial-delivery-count=0, max-message-size=0] [0x130d360]: -> AMQP [0x130d360]:0 -> @open(16) [container-id="2a746915-26a9-4d33-ba7d-a2586d8a5452", channel-max=32767, idle-time-out=15000] [0x130d360]:0 -> @begin(17) [remote-channel=0, next-outgoing-id=0, incoming-window=2147483647, outgoing-window=2147483647] [0x130d360]:0 -> @attach(18) [name="8dc9567d-6840-4e03-931e-cd2df90b6378", handle=0, role=true, snd-settle-mode=2, rcv-settle-mode=0, source=@source(40) [durable=0, timeout=0, dynamic=false], target=@target(41) [durable=0, timeout=0, dynamic=false], initial-delivery-count=0, max-message-size=0] [0x130d360]:0 -> @flow(19) [next-incoming-id=0, incoming-window=2147483647, next-outgoing-id=0, outgoing-window=2147483647, handle=0, delivery-count=0, link-credit=10, drain=false] [0x130d360]:0 -> (EMPTY FRAME) [0x130d360]:0 -> (EMPTY FRAME) [0x130d360]:0 -> (EMPTY FRAME) [0x130d360]:0 -> @close(24) [error=@error(29) [condition=:"amqp:resource-limit-exceeded", description="local-idle-timeout expired"]] [0x130d360]: -> EOS amqp:re
[jira] [Created] (PROTON-1890) [c++] implement idle_timeout and heartbeats
Praveen Bodke created PROTON-1890: - Summary: [c++] implement idle_timeout and heartbeats Key: PROTON-1890 URL: https://issues.apache.org/jira/browse/PROTON-1890 Project: Qpid Proton Issue Type: Bug Components: cpp-binding Affects Versions: 0.16.0 Reporter: Praveen Bodke This is similar issue reported in PROTON-1782 for ruby. We are facing this issue in cpp binding and i am able to reproduce the issue with scheduled_send_03.cpp example. The test scenario is to drop all the packets from both the interfaces after the successful connection. The only change i made to this example is to send messages continuously inside the send() method. The other end is detecting the error as it is sending the empty frames and no response is heard. The proton is not sending the heartbeat messages (empty frames) as the sender is busy in sending the data frames. Is it not necessary to send empty frames even if the data frames are sent? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
