Brian Aker wrote: > Hi! > > On Nov 7, 2008, at 5:42 PM, Mats Kindahl wrote: > >> BLOBs should not be sent in-band. Instead, just a BLOB locator should >> be sent >> (e.g., a URL) and then the storage engine, server main, or whatever, >> can open a >> connection and feed the blob into the correct place directly instead >> of copying >> in into the receiver buffer and then feeding that to the storage >> engine (usually >> by copying it). > > The problem with this is that the server is not built to accommodate > this (and it would be a massive rewrite to change it). We have no > specific was to version blobs internally and they are generated at row > request for the request. Any sort of "grab later" would not guarantee > that it was the same blob (or that the blob even continued to exist). If > someone wants this sort of behavior they need to need to normalize their > schema in a manner to support it. > > Is this the best solution? No. It would be a massive overhaul to change > the way row retrieval works though to support it. Even then, most > engines probably couldn't do it. I think it would be nice to find a way > to optionally support it if an engine could do it, but that is a > different issue (and can be solved without really messing with the > protocol by much). >
I don't think a massive rewrite is necessary: here is a quick sketch on how it could be done. On the master side: - We introduce a new event called "Raw blob" or something like that. - The dump thread is change to simply skip this event when it is seen and proceed after it. - Whenever a blob is processed, it is written as a BLOB event after then event (or before, it might be more practical) holding the statement. - The blob locater is encoded to hold the binlog position of the blob event for the blob. On the slave side: - On reception of a statement holding a blob locater, the storage engine that is supposed to accept it is queried if it support BLOB Streaming. - If it does, the blob locater is passed to the engine, who then proceeds to fetch it according to the procedure mentioned below. - If it does not support it, the slave thread fetches the BLOB according to the same procedure (below), re-builds the statement (easily done by assigning the blob to a user variable and ensuring that the blob locater looks like a user variable. Fetching the blob: - When a request for a blob arrives, the master spawns a new thread (or fetches one from a pool). - The locater hold the binlog position, so it proceeds by reading that position, where it expects to find a BLOB event. - It proceeds to send the blob the the requester. That way, the versioning is handled by the binary log. The size of the binary log is approximately the same, since it would have been written to the binary log anyway, but the slave has the option of handling the blob in such a manner that it does not stall processing of following events. No rewrite of engines is necessary, but for those that want, they can use the blob streaming. Just my few cents, Mats Kindahl > Cheers, > -Brian > > > -- > _______________________________________________________ > Brian "Krow" Aker, brian at tangent.org > Seattle, Washington > http://krow.net/ <-- Me > http://tangent.org/ <-- Software > _______________________________________________________ > You can't grep a dead tree. > > > -- Mats Kindahl Lead Software Developer Replication Team MySQL AB, www.mysql.com
begin:vcard fn:Mats Kindahl n:Kindahl;Mats org:Sun Microsystems adr;quoted-printable:;;Tegv=C3=A4gen 3;Storvreta;SE;74334;Sweden email;internet:[EMAIL PROTECTED] title:Lead Replication Software Developer x-mozilla-html:FALSE version:2.1 end:vcard
_______________________________________________ Mailing list: https://launchpad.net/~drizzle-discuss Post to : [email protected] Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
