[jira] [Commented] (DISPATCH-1614) Edge router crash when interior closes edge uplink connection
[ https://issues.apache.org/jira/browse/DISPATCH-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17076661#comment-17076661 ] ASF GitHub Bot commented on DISPATCH-1614: -- asfgit commented on pull request #714: DISPATCH-1614: Nullify address inlink and outlink when connection lost URL: https://github.com/apache/qpid-dispatch/pull/714 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Edge router crash when interior closes edge uplink connection > - > > Key: DISPATCH-1614 > URL: https://issues.apache.org/jira/browse/DISPATCH-1614 > Project: Qpid Dispatch > Issue Type: Bug > Components: Container >Affects Versions: 1.11.0 >Reporter: Charles E. Rolke >Assignee: Charles E. Rolke >Priority: Major > Attachments: DISPATCH-1614-edge-router-log.txt > > > When the connection to the interior router goes down then the > addr.edge_outlink link proper is freed but the pointer to the link is still > in the addr struct. > Later when the addr is unbound the link is dereferenced causing a segfault. > An attached log trace shows the events. Search for link _0x1c31210_ using > mobile address _M0e61_. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1614) Edge router crash when interior closes edge uplink connection
[ https://issues.apache.org/jira/browse/DISPATCH-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17076660#comment-17076660 ] ASF subversion and git services commented on DISPATCH-1614: --- Commit fbba3554db0a20e993865ba25a17ae0295cb3911 in qpid-dispatch's branch refs/heads/master from Charles E. Rolke [ https://gitbox.apache.org/repos/asf?p=qpid-dispatch.git;h=fbba355 ] DISPATCH-1614: Nullify address inlink and outlink when connection lost Prevent segfault referencing deleted links when connection is lost. Assert that the pointers are null when a connection is created. This closes #714 > Edge router crash when interior closes edge uplink connection > - > > Key: DISPATCH-1614 > URL: https://issues.apache.org/jira/browse/DISPATCH-1614 > Project: Qpid Dispatch > Issue Type: Bug > Components: Container >Affects Versions: 1.11.0 >Reporter: Charles E. Rolke >Assignee: Charles E. Rolke >Priority: Major > Attachments: DISPATCH-1614-edge-router-log.txt > > > When the connection to the interior router goes down then the > addr.edge_outlink link proper is freed but the pointer to the link is still > in the addr struct. > Later when the addr is unbound the link is dereferenced causing a segfault. > An attached log trace shows the events. Search for link _0x1c31210_ using > mobile address _M0e61_. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] [qpid-dispatch] asfgit closed pull request #714: DISPATCH-1614: Nullify address inlink and outlink when connection lost
asfgit closed pull request #714: DISPATCH-1614: Nullify address inlink and outlink when connection lost URL: https://github.com/apache/qpid-dispatch/pull/714 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1616) Scraper could export facts for creating sequence diagrams
[ https://issues.apache.org/jira/browse/DISPATCH-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17076640#comment-17076640 ] ASF subversion and git services commented on DISPATCH-1616: --- Commit fad96d5ccb73fe81c0ddc7f168583a41a3d52410 in qpid-dispatch's branch refs/heads/master from Charles E. Rolke [ https://gitbox.apache.org/repos/asf?p=qpid-dispatch.git;h=fad96d5 ] DISPATCH-1616: Add Scraper tooling to export source for sequence diagrams > Scraper could export facts for creating sequence diagrams > - > > Key: DISPATCH-1616 > URL: https://issues.apache.org/jira/browse/DISPATCH-1616 > Project: Qpid Dispatch > Issue Type: Improvement > Components: Tools >Affects Versions: 1.11.0 >Reporter: Charles E. Rolke >Assignee: Charles E. Rolke >Priority: Major > > Scraper has the AMQP to-from information needed to generate sequences > diagrams. With this info and a tool that matched AMQP message-sent to > message-received then users could easily generate sequence diagrams. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1615) Backtrace of leaked allocations does not show object address
[
https://issues.apache.org/jira/browse/DISPATCH-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17076601#comment-17076601
]
ASF GitHub Bot commented on DISPATCH-1615:
--
ChugR commented on pull request #715: DISPATCH-1615: Improve recent backtrace
debug output
URL: https://github.com/apache/qpid-dispatch/pull/715
Add searchable summary line that includes router identifier,
timestamp, object type, and object address.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Backtrace of leaked allocations does not show object address
>
>
> Key: DISPATCH-1615
> URL: https://issues.apache.org/jira/browse/DISPATCH-1615
> Project: Qpid Dispatch
> Issue Type: Improvement
> Components: Container
>Reporter: Charles E. Rolke
>Assignee: Charles E. Rolke
>Priority: Minor
>
> Also, in the event of hundreds of leaked objects the debug log requires a
> program to reconstruct the details into legible form.
> This patch would replace the timestamp line with a CSV line that would
> identify the object. The fields are:
> * "Leak" - literal word for grepping
> * debug_dump file name - identifies which router
> * timestamp
> * address of leaked object
> Then when a log file reveals the address of an object it may be more
> precisely linked to the leaks.
> For instance:
> {{Leak,/home/qpid-dispatch/build/tests/system_test.dir/system_tests_policy/someTest/setUpClass/INT.B-qddebug.txt,2020-04-06
> 09:55:56.968527 -0400,qd_link_ref_t,0x15149d0}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] [qpid-dispatch] ChugR opened a new pull request #715: DISPATCH-1615: Improve recent backtrace debug output
ChugR opened a new pull request #715: DISPATCH-1615: Improve recent backtrace debug output URL: https://github.com/apache/qpid-dispatch/pull/715 Add searchable summary line that includes router identifier, timestamp, object type, and object address. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (DISPATCH-1614) Edge router crash when interior closes edge uplink connection
[ https://issues.apache.org/jira/browse/DISPATCH-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17076591#comment-17076591 ] ASF GitHub Bot commented on DISPATCH-1614: -- ChugR commented on pull request #714: DISPATCH-1614: Nullify address inlink and outlink when connection lost URL: https://github.com/apache/qpid-dispatch/pull/714 Prevent segfault referencing deleted links when connection is lost. Assert that the pointers are null when a connection is created. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Edge router crash when interior closes edge uplink connection > - > > Key: DISPATCH-1614 > URL: https://issues.apache.org/jira/browse/DISPATCH-1614 > Project: Qpid Dispatch > Issue Type: Bug > Components: Container >Affects Versions: 1.11.0 >Reporter: Charles E. Rolke >Assignee: Charles E. Rolke >Priority: Major > Attachments: DISPATCH-1614-edge-router-log.txt > > > When the connection to the interior router goes down then the > addr.edge_outlink link proper is freed but the pointer to the link is still > in the addr struct. > Later when the addr is unbound the link is dereferenced causing a segfault. > An attached log trace shows the events. Search for link _0x1c31210_ using > mobile address _M0e61_. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[GitHub] [qpid-dispatch] ChugR opened a new pull request #714: DISPATCH-1614: Nullify address inlink and outlink when connection lost
ChugR opened a new pull request #714: DISPATCH-1614: Nullify address inlink and outlink when connection lost URL: https://github.com/apache/qpid-dispatch/pull/714 Prevent segfault referencing deleted links when connection is lost. Assert that the pointers are null when a connection is created. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Created] (DISPATCH-1616) Scraper could export facts for creating sequence diagrams
Charles E. Rolke created DISPATCH-1616: -- Summary: Scraper could export facts for creating sequence diagrams Key: DISPATCH-1616 URL: https://issues.apache.org/jira/browse/DISPATCH-1616 Project: Qpid Dispatch Issue Type: Improvement Components: Tools Affects Versions: 1.11.0 Reporter: Charles E. Rolke Assignee: Charles E. Rolke Scraper has the AMQP to-from information needed to generate sequences diagrams. With this info and a tool that matched AMQP message-sent to message-received then users could easily generate sequence diagrams. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Assigned] (DISPATCH-1614) Edge router crash when interior closes edge uplink connection
[ https://issues.apache.org/jira/browse/DISPATCH-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles E. Rolke reassigned DISPATCH-1614: -- Assignee: Charles E. Rolke > Edge router crash when interior closes edge uplink connection > - > > Key: DISPATCH-1614 > URL: https://issues.apache.org/jira/browse/DISPATCH-1614 > Project: Qpid Dispatch > Issue Type: Bug > Components: Container >Affects Versions: 1.11.0 >Reporter: Charles E. Rolke >Assignee: Charles E. Rolke >Priority: Major > Attachments: DISPATCH-1614-edge-router-log.txt > > > When the connection to the interior router goes down then the > addr.edge_outlink link proper is freed but the pointer to the link is still > in the addr struct. > Later when the addr is unbound the link is dereferenced causing a segfault. > An attached log trace shows the events. Search for link _0x1c31210_ using > mobile address _M0e61_. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (DISPATCH-1531) Connection fails if client creates a rx link with address len > ~8,133 bytes
[ https://issues.apache.org/jira/browse/DISPATCH-1531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ganesh Murthy updated DISPATCH-1531: Fix Version/s: (was: 1.12.0) > Connection fails if client creates a rx link with address len > ~8,133 bytes > > > Key: DISPATCH-1531 > URL: https://issues.apache.org/jira/browse/DISPATCH-1531 > Project: Qpid Dispatch > Issue Type: Bug > Components: Router Node >Affects Versions: 1.10.0 >Reporter: Ken Giusti >Assignee: Ken Giusti >Priority: Major > Attachments: address-loader > > > Using the attached client, attempt to create a single rx link with an address > 8,134 bytes long: > ./address-loader --verbose --conns 1 --links 1 --length 8134 > Connection fails with: > AMQP:FRAME:0 <- @close(24) [error=@error(29) > [condition=:"amqp:connection:framing-error", description="malformed frame"]] > Try again with --length 8133 - succeeds. > Not sure if this is intentional, or if this is a proton issue, but no > warning+ logs are issued by the router so I can't really tell what is going > on here. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (DISPATCH-1530) system_tests_multi_tenancy fails occasionally
[ https://issues.apache.org/jira/browse/DISPATCH-1530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ganesh Murthy updated DISPATCH-1530: Fix Version/s: (was: 1.12.0) > system_tests_multi_tenancy fails occasionally > -- > > Key: DISPATCH-1530 > URL: https://issues.apache.org/jira/browse/DISPATCH-1530 > Project: Qpid Dispatch > Issue Type: Improvement > Components: Tests >Affects Versions: 1.9.0 >Reporter: Ganesh Murthy >Assignee: Charles E. Rolke >Priority: Major > Attachments: DISPATCH-1530-autolink-log-lines.txt, > DISPATCH-1530-test2-log.txt > > > FAIL: test_36_two_router_waypoint_external_addr > (system_tests_multi_tenancy.RouterTest) > -- > Traceback (most recent call last): > File "/foo/qpid-dispatch/tests/system_tests_multi_tenancy.py", line 459, in > test_36_two_router_waypoint_external_addr > self.assertEqual(None, test.error) > AssertionError: None != u'Timeout Expired: n_sent=10 n_rcvd=8 n_thru=8' > -- > Ran 36 tests in 73.577s -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Created] (DISPATCH-1615) Backtrace of leaked allocations does not show object address
Charles E. Rolke created DISPATCH-1615:
--
Summary: Backtrace of leaked allocations does not show object
address
Key: DISPATCH-1615
URL: https://issues.apache.org/jira/browse/DISPATCH-1615
Project: Qpid Dispatch
Issue Type: Improvement
Components: Container
Reporter: Charles E. Rolke
Assignee: Charles E. Rolke
Also, in the event of hundreds of leaked objects the debug log requires a
program to reconstruct the details into legible form.
This patch would replace the timestamp line with a CSV line that would identify
the object. The fields are:
* "Leak" - literal word for grepping
* debug_dump file name - identifies which router
* timestamp
* address of leaked object
Then when a log file reveals the address of an object it may be more precisely
linked to the leaks.
For instance:
{{Leak,/home/qpid-dispatch/build/tests/system_test.dir/system_tests_policy/someTest/setUpClass/INT.B-qddebug.txt,2020-04-06
09:55:56.968527 -0400,qd_link_ref_t,0x15149d0}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (DISPATCH-1533) Connection drop if > 255 links per conn opened with address == 8133 bytes
[ https://issues.apache.org/jira/browse/DISPATCH-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ganesh Murthy updated DISPATCH-1533: Fix Version/s: (was: 1.12.0) > Connection drop if > 255 links per conn opened with address == 8133 bytes > - > > Key: DISPATCH-1533 > URL: https://issues.apache.org/jira/browse/DISPATCH-1533 > Project: Qpid Dispatch > Issue Type: Bug > Components: Router Node >Affects Versions: 1.9.0, 1.10.0 >Reporter: Ken Giusti >Assignee: Ken Giusti >Priority: Major > Attachments: address-loader > > > Using the attached reproducer attempting to create > 255 links on a single > connection fails with > AMQP:FRAME:0 <- @close(24) [error=@error(29) > [condition=:"amqp:connection:framing-error", description="malformed frame"]] > Run the reproducer: > ./address-loader --verbose --conns 1 --links 1000 --length 8133 > Note this appears to be connection related rather than a route table > limitation. It is possible to open many connections with 255 links: > ./address-loader --verbose --conns 20 --links 255 --length 8133 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Re: [jira] [Created] (PROTON-2189) proactor C client has abnormally long pauses during message send
> > > *Hey Mick, * > *That did the trick - color me impressed.* > OK. Ken, impressed: [image: image.png] On Mon, Apr 6, 2020 at 9:18 AM Ken Giusti wrote: > Hey Mick, > > That did the trick - color me impressed. > > If that's the intended behavior for sending streaming messages it wasn't > clear to me. I'll update the JIRA with your recommendations, thanks! > > > On Sat, Apr 4, 2020 at 12:32 PM Michael Goulish > wrote: > >> Ken -- >> >> I have a proactor client in Mercury and it handles timeouts differently >> than your code. >> I don't remember now exactly why this was necessary, but I do vaguely >> recall weird >> behavior before I fixed it, at aconway's suggestion. >> >> I think that, in the PN_PROACTOR_TIMEOUT case in the event handler, in >> my code >> it was not correct to send at that time and then reset the timer. (As you >> are doing.) >> >> Instead, in that case all I did was to wake the cnx: >> pn_connection_wake ( context->connection ); >> >> You can then send, and reset the timer only when you get an event of type: >> PN_CONNECTION_WAKE: >> send_message ( context ); >> pn_proactor_set_timeout ( context->proactor, context->throttle ); >> >> In my case, I am sending entire messages -- but this is how I get it to >> 'throttle' the send-rate, i.e. send 1 message per N msec. >> >> >> I don't remember exactly what Bad Thing was happening to me before I >> started doing it this way, but I have a Bad Feeling that it may have been >> similar to what you describe. >> >> >> >> >> >> On Sat, Apr 4, 2020 at 12:00 PM Ken Giusti (Jira) >> wrote: >> >>> Ken Giusti created PROTON-2189: >>> -- >>> >>> Summary: proactor C client has abnormally long pauses >>> during message send >>> Key: PROTON-2189 >>> URL: https://issues.apache.org/jira/browse/PROTON-2189 >>> Project: Qpid Proton >>> Issue Type: Bug >>> Components: proton-c >>> Affects Versions: proton-c-0.30.0 >>> Environment: To compile the clients install qpid-proton-c-devel >>> and simply compile: >>> >>> gcc -O2 -g -Wall -lqpid-proton -lm -o clogger clogger.c >>> >>> To reproduce my test, build qdrouterd and run it in the background. >>> You need to have a consumer attached. There is a test receiver client >>> in the qdrouterd build in /tests/test-receiver. This receiver >>> is designed to handle streaming messages (by default sent to 'test-address') >>> >>> Run the consumer in the background then run each clogger (default params >>> are fine). >>> >>> You should observe that clogger-reactor runs smoothly (use >>> PN_TRACE_FRM=1 on qdrouterd as well). >>> >>> You'll see clogger-reactor send the message header, then nothing for >>> awhile, then send the entire message. >>> >>> Use "-D" for debug output to see how many bytes have been written to >>> pn_link_send() >>> >>> >>> Reporter: Ken Giusti >>> Attachments: clogger-proactor.c, clogger-reactor.c >>> >>> I have a proactor-based C test client that has the ability to slowly >>> send extremely large messages slowly. This is done by sending 'chunks' of >>> body data with pauses in between. >>> >>> This client was designed to test large streaming messages against >>> qdrouterd. >>> >>> The behavior of this client is unexpected - I would expect the message >>> data to appear "on the wire" in bursts relatively quickly. In reality the >>> data is buffered - in some cases over 1 GB is buffered - before it is >>> written (as indicated by the lack @transfer frames dumped by the client AND >>> the qdrouterd). In some cases it takes up to 30 seconds before the >>> client's data starts being written to the client. >>> >>> I've refactored the client to use reactor instead and the data flows as >>> expected. There is minimal buffering and no abnormally long pauses. >>> >>> The clients are attached. >>> >>> It is quite likely the proactor client is incorrectly implemented, but I >>> used the qdrouterd I/O loop as the model and cannot see what may be wrong. >>> >>> >>> >>> >>> >>> -- >>> This message was sent by Atlassian Jira >>> (v8.3.4#803005) >>> >>> - >>> To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org >>> For additional commands, e-mail: dev-h...@qpid.apache.org >>> >>> > > -- > -K >
[jira] [Commented] (PROTON-2189) proactor C client has abnormally long pauses during message send
[ https://issues.apache.org/jira/browse/PROTON-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17076338#comment-17076338 ] Ken Giusti commented on PROTON-2189: Mick has designed a proactor client that can properly stream messages. See his email to the mailing list: [http://qpid.2158936.n2.nabble.com/Re-jira-Created-PROTON-2189-proactor-C-client-has-abnormally-long-pauses-during-message-send-td7691016.html] I've updated my proactor client as described and can confirm it now streams as I expected. Attached is the patch. I'll leave it to the devs to decide if this is Not A Bug or a documentation/example gap. [^0001-tweak-clogger-to-fix-streaming.patch] > proactor C client has abnormally long pauses during message send > > > Key: PROTON-2189 > URL: https://issues.apache.org/jira/browse/PROTON-2189 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: proton-c-0.30.0 > Environment: To compile the clients install qpid-proton-c-devel and > simply compile: > gcc -O2 -g -Wall -lqpid-proton -lm -o clogger clogger.c > To reproduce my test, build qdrouterd and run it in the background. > You need to have a consumer attached. There is a test receiver client in the > qdrouterd build in /tests/test-receiver. This receiver is > designed to handle streaming messages (by default sent to 'test-address') > Run the consumer in the background then run each clogger (default params are > fine). > You should observe that clogger-reactor runs smoothly (use PN_TRACE_FRM=1 on > qdrouterd as well). > You'll see clogger-reactor send the message header, then nothing for awhile, > then send the entire message. > Use "-D" for debug output to see how many bytes have been written to > pn_link_send() >Reporter: Ken Giusti >Priority: Major > Attachments: 0001-tweak-clogger-to-fix-streaming.patch, > clogger-proactor.c, clogger-reactor.c > > > I have a proactor-based C test client that has the ability to slowly send > extremely large messages slowly. This is done by sending 'chunks' of body > data with pauses in between. > This client was designed to test large streaming messages against qdrouterd. > The behavior of this client is unexpected - I would expect the message data > to appear "on the wire" in bursts relatively quickly. In reality the data is > buffered - in some cases over 1 GB is buffered - before it is written (as > indicated by the lack @transfer frames dumped by the client AND the > qdrouterd). In some cases it takes up to 30 seconds before the client's data > starts being written to the client. > I've refactored the client to use reactor instead and the data flows as > expected. There is minimal buffering and no abnormally long pauses. > The clients are attached. > It is quite likely the proactor client is incorrectly implemented, but I used > the qdrouterd I/O loop as the model and cannot see what may be wrong. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (PROTON-2189) proactor C client has abnormally long pauses during message send
[ https://issues.apache.org/jira/browse/PROTON-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ken Giusti updated PROTON-2189: --- Attachment: 0001-tweak-clogger-to-fix-streaming.patch > proactor C client has abnormally long pauses during message send > > > Key: PROTON-2189 > URL: https://issues.apache.org/jira/browse/PROTON-2189 > Project: Qpid Proton > Issue Type: Bug > Components: proton-c >Affects Versions: proton-c-0.30.0 > Environment: To compile the clients install qpid-proton-c-devel and > simply compile: > gcc -O2 -g -Wall -lqpid-proton -lm -o clogger clogger.c > To reproduce my test, build qdrouterd and run it in the background. > You need to have a consumer attached. There is a test receiver client in the > qdrouterd build in /tests/test-receiver. This receiver is > designed to handle streaming messages (by default sent to 'test-address') > Run the consumer in the background then run each clogger (default params are > fine). > You should observe that clogger-reactor runs smoothly (use PN_TRACE_FRM=1 on > qdrouterd as well). > You'll see clogger-reactor send the message header, then nothing for awhile, > then send the entire message. > Use "-D" for debug output to see how many bytes have been written to > pn_link_send() >Reporter: Ken Giusti >Priority: Major > Attachments: 0001-tweak-clogger-to-fix-streaming.patch, > clogger-proactor.c, clogger-reactor.c > > > I have a proactor-based C test client that has the ability to slowly send > extremely large messages slowly. This is done by sending 'chunks' of body > data with pauses in between. > This client was designed to test large streaming messages against qdrouterd. > The behavior of this client is unexpected - I would expect the message data > to appear "on the wire" in bursts relatively quickly. In reality the data is > buffered - in some cases over 1 GB is buffered - before it is written (as > indicated by the lack @transfer frames dumped by the client AND the > qdrouterd). In some cases it takes up to 30 seconds before the client's data > starts being written to the client. > I've refactored the client to use reactor instead and the data flows as > expected. There is minimal buffering and no abnormally long pauses. > The clients are attached. > It is quite likely the proactor client is incorrectly implemented, but I used > the qdrouterd I/O loop as the model and cannot see what may be wrong. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Re: [jira] [Created] (PROTON-2189) proactor C client has abnormally long pauses during message send
Hey Mick, That did the trick - color me impressed. If that's the intended behavior for sending streaming messages it wasn't clear to me. I'll update the JIRA with your recommendations, thanks! On Sat, Apr 4, 2020 at 12:32 PM Michael Goulish wrote: > Ken -- > > I have a proactor client in Mercury and it handles timeouts differently > than your code. > I don't remember now exactly why this was necessary, but I do vaguely > recall weird > behavior before I fixed it, at aconway's suggestion. > > I think that, in the PN_PROACTOR_TIMEOUT case in the event handler, in my > code > it was not correct to send at that time and then reset the timer. (As you > are doing.) > > Instead, in that case all I did was to wake the cnx: > pn_connection_wake ( context->connection ); > > You can then send, and reset the timer only when you get an event of type: > PN_CONNECTION_WAKE: > send_message ( context ); > pn_proactor_set_timeout ( context->proactor, context->throttle ); > > In my case, I am sending entire messages -- but this is how I get it to > 'throttle' the send-rate, i.e. send 1 message per N msec. > > > I don't remember exactly what Bad Thing was happening to me before I > started doing it this way, but I have a Bad Feeling that it may have been > similar to what you describe. > > > > > > On Sat, Apr 4, 2020 at 12:00 PM Ken Giusti (Jira) wrote: > >> Ken Giusti created PROTON-2189: >> -- >> >> Summary: proactor C client has abnormally long pauses during >> message send >> Key: PROTON-2189 >> URL: https://issues.apache.org/jira/browse/PROTON-2189 >> Project: Qpid Proton >> Issue Type: Bug >> Components: proton-c >> Affects Versions: proton-c-0.30.0 >> Environment: To compile the clients install qpid-proton-c-devel >> and simply compile: >> >> gcc -O2 -g -Wall -lqpid-proton -lm -o clogger clogger.c >> >> To reproduce my test, build qdrouterd and run it in the background. >> You need to have a consumer attached. There is a test receiver client in >> the qdrouterd build in /tests/test-receiver. This receiver is >> designed to handle streaming messages (by default sent to 'test-address') >> >> Run the consumer in the background then run each clogger (default params >> are fine). >> >> You should observe that clogger-reactor runs smoothly (use PN_TRACE_FRM=1 >> on qdrouterd as well). >> >> You'll see clogger-reactor send the message header, then nothing for >> awhile, then send the entire message. >> >> Use "-D" for debug output to see how many bytes have been written to >> pn_link_send() >> >> >> Reporter: Ken Giusti >> Attachments: clogger-proactor.c, clogger-reactor.c >> >> I have a proactor-based C test client that has the ability to slowly send >> extremely large messages slowly. This is done by sending 'chunks' of body >> data with pauses in between. >> >> This client was designed to test large streaming messages against >> qdrouterd. >> >> The behavior of this client is unexpected - I would expect the message >> data to appear "on the wire" in bursts relatively quickly. In reality the >> data is buffered - in some cases over 1 GB is buffered - before it is >> written (as indicated by the lack @transfer frames dumped by the client AND >> the qdrouterd). In some cases it takes up to 30 seconds before the >> client's data starts being written to the client. >> >> I've refactored the client to use reactor instead and the data flows as >> expected. There is minimal buffering and no abnormally long pauses. >> >> The clients are attached. >> >> It is quite likely the proactor client is incorrectly implemented, but I >> used the qdrouterd I/O loop as the model and cannot see what may be wrong. >> >> >> >> >> >> -- >> This message was sent by Atlassian Jira >> (v8.3.4#803005) >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org >> For additional commands, e-mail: dev-h...@qpid.apache.org >> >> -- -K
