Hi Josh, On Wed, Sep 15, 2010 at 10:52 PM, Josh Berkus <[email protected]> wrote: > On 9/14/10 12:15 PM, David Shrewsbury wrote: >> #2 does not suffer from the concurrency issue of #1, but it does >> mean marginally more work for TransactionApplier implementers >> to buffer GBP messages until the end Transaction message is >> read. > > How does the receiver know how many messages are coming in that > transaction? That is, how will it know when it's done receiving the > bulk load and can proceed? And is there any possibility of the bulk > load messages arriving out of order?
Great questions. The embedded Statement message within the Transaction message contains an "end_segment" attribute as well as a "segment_id" attribute. When a Transaction message has reached its size limit and needs to be broken up, we set "end_segment" to false, and increment the value of "segment_id" for the corresponding Statement in the next Transaction message. When every Statement message within the Transaction message has end_segment = true, then the receiver will know that it has received the last message in the Transaction. You can see an example of this in my blog post here: http://dshrewsbury.blogspot.com/2010/08/drizzle-transaction-message-limit.html Note that in that example in the blog post, segment_id did not increment properly (it should have been 2 in the second message). That is a bug that has recently been fixed. And there is no possibility that the bulk load messages could arrive out of order. The messages are sent sequentially within the single thread that is handling the bulk load. Thanks for the feedback! -Dave _______________________________________________ Mailing list: https://launchpad.net/~drizzle-discuss Post to : [email protected] Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
