[
https://issues.apache.org/jira/browse/DISPATCH-767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078271#comment-16078271
]
ASF GitHub Bot commented on DISPATCH-767:
-----------------------------------------
Github user alanconway commented on a diff in the pull request:
https://github.com/apache/qpid-dispatch/pull/172#discussion_r126163415
--- Diff: src/message.c ---
@@ -996,21 +1094,47 @@ qd_message_t *qd_message_receive(pn_delivery_t
*delivery)
}
//
+ // The discard flag indicates if we should continue receiving the
message.
+ // This is pertinent in the case of large messages. When large
messages are being received, we try to send out part of the
+ // message that has been received so far. If we not able to send it
anywhere, there is no need to keep creating buffers
+ //
+ bool discard = qd_message_is_discard((qd_message_t*)msg);
+
+ //
// Get a reference to the tail buffer on the message. This is the
buffer into which
- // we will store incoming message data. If there is no buffer in the
message, allocate
- // an empty one and add it to the message.
+ // we will store incoming message data. If there is no buffer in the
message, this is the
+ // first time we are here and we need to allocate an empty one and add
it to the message.
//
- buf = DEQ_TAIL(msg->content->buffers);
- if (!buf) {
- buf = qd_buffer();
- DEQ_INSERT_TAIL(msg->content->buffers, buf);
+ if (!discard) {
+ buf = DEQ_TAIL(msg->content->buffers);
+ if (!buf) {
+ buf = qd_buffer();
+ DEQ_INSERT_TAIL(msg->content->buffers, buf);
+ }
}
while (1) {
- //
- // Try to receive enough data to fill the remaining space in the
tail buffer.
- //
- rc = pn_link_recv(link, (char*) qd_buffer_cursor(buf),
qd_buffer_capacity(buf));
+ if (discard) {
+ char dummy[BUFFER_SIZE];
+ rc = pn_link_recv(link, dummy, BUFFER_SIZE);
+ }
+ else {
+
+ //
+ // Make sure our buffer chain length is always less than
MAX_BUFFER_LENGTH. We don't want to add any more buffers beyond
MAX_BUFFER_LENGTH.
+ //
+ //
+ //sys_mutex_lock(msg->content->lock);
+ //if (DEQ_SIZE(msg->content->buffers) >
MAX_BUFFER_CHAIN_LENGTH) {
+ // return (qd_message_t*) msg;
+ //}
+ //sys_mutex_unlock(msg->content->lock);
+
+ //
+ // Try to receive enough data to fill the remaining space in
the tail buffer.
+ //
+ rc = pn_link_recv(link, (char*) qd_buffer_cursor(buf),
qd_buffer_capacity(buf));
+ }
//
// If we receive PN_EOS, we have come to the end of the message.
--- End diff --
FYI: instead of watching for PN_EOS, you can also check for
(pn_delivery_pending(d) == 0 && !pn_delivery_partial(d)). They are equivalent,
PN_EOS is correct - sometimes the pending/partial test is more convenient
because it stays true on the delivery so you can check it any time, not just at
return from pn_link_recv(). Only FYI, no need to change this code.
> Message Cut-Through/Streaming for efficient handling of large messages
> ----------------------------------------------------------------------
>
> Key: DISPATCH-767
> URL: https://issues.apache.org/jira/browse/DISPATCH-767
> Project: Qpid Dispatch
> Issue Type: Improvement
> Components: Router Node
> Reporter: Ted Ross
> Assignee: Ganesh Murthy
> Fix For: 1.0.0
>
>
> When large, multi-frame messages are sent through the router, there is no
> need to wait for the entire message to arrive before starting to send it
> onward.
> This feature causes the router to route the first frame and allow subsequent
> frames in a delivery to be streamed out in pipeline fashion. Ideally, the
> memory usage in the router should only involve pending frames. This would
> allow the router to handle arbitrary numbers of concurrent arbitrarily large
> messages.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
