Yeah, fair enough. My concern is that as people play around with the plugins and invent new ones, there will never emerge a standard way to manage replication in drizzle. If people start messing around with yet more ways to store replication logs (of which, I believe there are already two in drizzle), and yet more ways to handle storing and applying relay logs, there's nothing allowing them to re-use things like your DATA_DICTIONARY tables without simply copying the code.
Completely independent replication plugins make sense if you are running replication down two different channels to two different destinations (like replicating to drizzle slaves and a memcache farm); you don't want the metadata for each channel to conflict But it doesn't make sense if you want to use one replication channel as a replacement for some other one (e.g., maybe I want to use a memory-based way of storing replication log data on a master instead of an innodb table for performance purposes). In a case like that, I don't want to re-learn how to do the drizzle equivalent of a CHANGE MASTER and deal with nuances that are plugin-specific unless they are actually necessary. Should there be some kind of global transaction ID that is unique across replicas? I'd say yes. But how do we implement that? Seems like it should be independent from the plugins concerned with the transport of the transactions (to me, anyway), so that can be handled the same way across transport mechanisms (if we so choose). Once you have that global transaction ID, then in theory changing masters becomes relatively easy. But how do we do that? Currently, it's completely up to the plugin. Without a standard way to do that, operations for drizzle becomes that much more expensive. I certainly understand that this is all new, etc. I also see the benefit of the complete and total freedom of how it is now. I'm totally uncertain if there will ever need to emerge alternative master applier / slave plugins or not within the standard master -> slave paradigm. However, I *think* I see a subset of possible replication use cases that could benefit from some extra common apis that are probably somewhat orthogonal to replication transport. I guess I just want to know that the door isn't shut for those kinds of enhancements in the drizzle kernel if they become more obvious down the road. Jay Janssen On Tuesday, April 19, 2011 at 3:13 PM, David Shrewsbury wrote: > Hi Jay, > > This really goes against the design that we currently have. There is no > requirement for any applier plugin to maintain a store of the messages > for later retrieval. And COMMIT_ID really only applies to the InnoDB > replication plugin. Each applier is supposed to do something different > with the messages. Forcing them each to store in a manner similar to > the InnoDB replication plugin seems counterproductive (if I'm understanding > your request correctly). If each is doing the same thing, why even bother > having more than one? They'd each just be storing the same message. > > So the idea behind the applier plugin API is that you don't have to worry > about what's going on in the kernel. If you want to store the message locally, > great. If not, do whatever you want with it (send it off to another machine, > save it in a file, etc). > > Does that make sense? > > -Dave > > > On Mon, Apr 18, 2011 at 8:47 PM, [email protected] > <[email protected]> wrote: > > Hi all, > > I've been looking through the replication code and it seems to me that > > Drizzle could use a few extra interfaces for plugins to use. I definitely > > don't know all the discussion that has already occurred around this, or > > exactly how early or not it is in the development of the replication > > plugins, so please pardon any ignorance on my part of any master plan in > > all this. > > > > On the master side, it seems like there should be some common api call to > > collect data from the replication logs, regardless of how it is stored. The > > current 'slave' plugin simply connects to the master and selects off of the > > SYS_REPLICATION table, which (I believe) is only updated by the innodb > > replicator. Wouldn't it be better to either a) define a common table to > > select from and require all master replicator plugins to use that table, or > > b) use some SQL construct that says something to the effect of 'GET > > COMMIT_ID=x' or equivalent, and let the master plugins implement the hook > > for that however it wants. I'm writing this without knowledge whatever SQL > > standards that drizzle uses, so excuse me there. This would allow the > > plugin to simply focus how to efficiently store (or not) and serve the data > > to the slaves as rapidly as possible, and would allow the two existing > > master replication plugins to work regardless of the plugin used on the > > slave side. > > > > Similarly on the slave side, some simple api to manage the connection to > > the master and get replication data in the standard way above would be > > handy, as well as some standard api to 'apply this master commit id that I > > have' call. Again, here, I see advantage in plugins focusing on how to > > efficiently buffer replication from the master (or not) and apply it to the > > local drizzle instance. > > > > In both cases (master and slave), with the common apis any metadata and > > status stored about replication could be done identically (or at least with > > separate plugins) so people experimenting with various ways to manage the > > replication stream wouldn't have to muck with replication metadata and > > vice-versa. This would allow conventions about changing masters, seeing > > slave status, etc. to not have to change between plugins unless the > > replication model was dramatically different from the standard enough that > > those standards wouldn't make sense anyway. > > > > > > Jay Janssen > > > > > > > > _______________________________________________ > > Mailing list: https://launchpad.net/~drizzle-discuss > > Post to : [email protected] > > Unsubscribe : https://launchpad.net/~drizzle-discuss > > More help : https://help.launchpad.net/ListHelp > > > >
_______________________________________________ Mailing list: https://launchpad.net/~drizzle-discuss Post to : [email protected] Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
