[ https://issues.apache.org/jira/browse/DISPATCH-767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078271#comment-16078271 ]
ASF GitHub Bot commented on DISPATCH-767: ----------------------------------------- Github user alanconway commented on a diff in the pull request: https://github.com/apache/qpid-dispatch/pull/172#discussion_r126163415 --- Diff: src/message.c --- @@ -996,21 +1094,47 @@ qd_message_t *qd_message_receive(pn_delivery_t *delivery) } // + // The discard flag indicates if we should continue receiving the message. + // This is pertinent in the case of large messages. When large messages are being received, we try to send out part of the + // message that has been received so far. If we not able to send it anywhere, there is no need to keep creating buffers + // + bool discard = qd_message_is_discard((qd_message_t*)msg); + + // // Get a reference to the tail buffer on the message. This is the buffer into which - // we will store incoming message data. If there is no buffer in the message, allocate - // an empty one and add it to the message. + // we will store incoming message data. If there is no buffer in the message, this is the + // first time we are here and we need to allocate an empty one and add it to the message. // - buf = DEQ_TAIL(msg->content->buffers); - if (!buf) { - buf = qd_buffer(); - DEQ_INSERT_TAIL(msg->content->buffers, buf); + if (!discard) { + buf = DEQ_TAIL(msg->content->buffers); + if (!buf) { + buf = qd_buffer(); + DEQ_INSERT_TAIL(msg->content->buffers, buf); + } } while (1) { - // - // Try to receive enough data to fill the remaining space in the tail buffer. - // - rc = pn_link_recv(link, (char*) qd_buffer_cursor(buf), qd_buffer_capacity(buf)); + if (discard) { + char dummy[BUFFER_SIZE]; + rc = pn_link_recv(link, dummy, BUFFER_SIZE); + } + else { + + // + // Make sure our buffer chain length is always less than MAX_BUFFER_LENGTH. We don't want to add any more buffers beyond MAX_BUFFER_LENGTH. + // + // + //sys_mutex_lock(msg->content->lock); + //if (DEQ_SIZE(msg->content->buffers) > MAX_BUFFER_CHAIN_LENGTH) { + // return (qd_message_t*) msg; + //} + //sys_mutex_unlock(msg->content->lock); + + // + // Try to receive enough data to fill the remaining space in the tail buffer. + // + rc = pn_link_recv(link, (char*) qd_buffer_cursor(buf), qd_buffer_capacity(buf)); + } // // If we receive PN_EOS, we have come to the end of the message. --- End diff -- FYI: instead of watching for PN_EOS, you can also check for (pn_delivery_pending(d) == 0 && !pn_delivery_partial(d)). They are equivalent, PN_EOS is correct - sometimes the pending/partial test is more convenient because it stays true on the delivery so you can check it any time, not just at return from pn_link_recv(). Only FYI, no need to change this code. > Message Cut-Through/Streaming for efficient handling of large messages > ---------------------------------------------------------------------- > > Key: DISPATCH-767 > URL: https://issues.apache.org/jira/browse/DISPATCH-767 > Project: Qpid Dispatch > Issue Type: Improvement > Components: Router Node > Reporter: Ted Ross > Assignee: Ganesh Murthy > Fix For: 1.0.0 > > > When large, multi-frame messages are sent through the router, there is no > need to wait for the entire message to arrive before starting to send it > onward. > This feature causes the router to route the first frame and allow subsequent > frames in a delivery to be streamed out in pipeline fashion. Ideally, the > memory usage in the router should only involve pending frames. This would > allow the router to handle arbitrary numbers of concurrent arbitrarily large > messages. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org