Hello Sir, > we just need to parse MySQL binlog and convert into drizzle transaction.proto, we won't have to change it at all.
Yeah I was trying to convey the same thing but in a different manner. As I mentioned > 1. As MySQL binlogs are row based, we need to change or introduce new > 'Blueprints' for data in proto file (this file will be a modified copy of > current transaction.proto). If in future if we find some DDL inconsistency or any other issue that *may* require changing transaction.proto then we should modify it and keep that under different version. As this will obviously help in entirely independent working of our module and we don't affect Drizzle slave or applier at the same time. > I wouldn't worry too much about it at this stage, we could attempt the SQL and just error out if it doesn't apply. Yeah great! I introduced Checksum and DDL heuristics just as a part of proposal. We have to create basic prototype first then we will keep on improving it. Apart from this if you have any other thing in mind that I should write in my proposal then please share. On Sun, Apr 21, 2013 at 9:48 PM, Stewart Smith <[email protected]>wrote: > kuldeep porwal <[email protected]> writes: > > *2. To Replicate from MySQL to Drizzle (Gsoc Idea): * > > > > This process is quiet simple. We read the MySQL row based binlog from the > > publisher (master), then we *form* the new Transaction Message using > > a slightly modified version of 'transaction.proto' file. This Event of > > forming Transaction Message will be CALLED in the * binlog_event of > > libdrizzle-redux binlog api *. Once we have the Transaction Message, > reader > > plugin will push it to publisher plugin on master and then the salve > > machine (subscriber) will read and replicate it via Transaction > > Applier. > > I think we should take a slightly different approach. Instead of > changing the Drizzle replication log format, we read the MySQL binlog > and immediately convert into Drizzle transaction messages... this means > that the process of connecting to mysqld and converting to drizzle > transaction message is entirely independent of the drizzle slave apply > code. > > > The Trickiest part is that we are using all the Replication modules of > > Drizzle but to take MySQL binlog and covert them into Drizzle's > Transaction > > message we are building a NEW module. > > yes, doing the conversion will be slightly tricky, we can start off easy > and not care about DDL or data types we don't support. i.e. replicate a > workload that would work against Drizzle anyway. > > > 1. As MySQL binlogs are row based, we need to change or introduce new > > 'Blueprints' for data in proto file (this file will be a modified copy of > > current transaction.proto). > > I don't think we need this, we just need to parse MySQL binlog and > convert into drizzle transaction.proto, we won't have to change it at all. > > > 2. Also the log can be big so to provide the error checking ans safe > > replication we can add CHECKSUM. > > 3. We can use some kind of Heuristics to remove DDL based inconsistencies > > between MySQL and Drizzle. > > I wouldn't worry too much about it at this stage, we could attempt the > SQL and just error out if it doesn't apply. > > > -- > Stewart Smith > -- Regards, Kuldeep Porwal IIIT Hyderabad 09550605256 http://web.iiit.ac.in/~kuldeep.porwal
_______________________________________________ Mailing list: https://launchpad.net/~drizzle-discuss Post to : [email protected] Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
