I'm not one that would be working on TransactionAppliers but it seems like #2 sounds like a generally more robust and superior solution, particularly with the known issues with option #1 in MySQL. I see #1 come up from time to time and it would be nice to see that pain alleviated.
$0.02 Tim S. On Sep 14, 2010, at 2:15 PM, David Shrewsbury wrote: > Hi all, > > Some recent changes to the Drizzle kernel have been made so > that we can now support bulk loads in the replication stream > (e.g., the transaction log). As a result, we now have an issue > where we'd like to hear some feedback from the community. > > Before the change, a single Google Protobuf Transaction message > was guaranteed to contain all statements executed within the > transaction. In order to support bulk loads, there is now a limit > on the size we allow a Transaction message to grow to, so a > large transaction could likely create more than one GPB Transaction > message, but each would be linked together using the same > transaction ID. > > Now the issue: There is currently no guarantee that readers of > the replication stream will receive all Transaction messages from > a single transaction in an atomic fashion. In other words, if you > have a large transaction (like a bulk load) happening on one > connection, and other transactions occurring in different connections, > the replication stream will contain intermingled GPB Transaction > messages. > > The options for handling this are: > > 1) Place locks on the replication stream and buffer Transaction > messages, likely to disk, until the transaction is complete and > send them all down the stream together, locking the stream to > do so. This is how MySQL handles writing RBR events to the > binlog. > > 2) Let the replication stream readers do the buffering of the > Transaction messages themselves. > > #1 causes concurrency issues on the master. MySQL suffers > from this same problem, and they note it in the documentation > as a disadvantage of using the RBR format: > > http://dev.mysql.com/doc/refman/5.1/en/replication-sbr-rbr.html > > #2 does not suffer from the concurrency issue of #1, but it does > mean marginally more work for TransactionApplier implementers > to buffer GBP messages until the end Transaction message is > read. > > Thoughts? > > -Dave > > _______________________________________________ > Mailing list: https://launchpad.net/~drizzle-discuss > Post to : [email protected] > Unsubscribe : https://launchpad.net/~drizzle-discuss > More help : https://help.launchpad.net/ListHelp > _______________________________________________ Mailing list: https://launchpad.net/~drizzle-discuss Post to : [email protected] Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
