On Fri, 2009-01-23 at 16:49 -0500, Tom Lane wrote: > Simon Riggs <si...@2ndquadrant.com> writes: > > On Fri, 2009-01-23 at 10:33 -0500, Tom Lane wrote: > >> Right, the WAL-record-processing API is not really at issue, since it's > >> been proven internally to the core code. My concern is with the other > >> part, namely exactly how are we going to identify and install additional > >> rmgrs. > > > The patch is just > > * a hook in StartupXLOG to allow loading arbitrary code into Startup > > * some slight redefinition of RmgrTable to allow arbitrary code to add > > or modify the contents of that table of functions. (Being able to modify > > the table is an not necessary for index extensions, but is for other > > uses). > > * some safeguards people requested > > Well, that really seems to just prove my point. You've defined a hook > and not thought carefully about how people will use it.
This was originally proposed on 19 August and a patch submitted to the September commit fest. http://archives.postgresql.org/pgsql-hackers/2008-08/msg00794.php After about 30 emails of technical rebuttal we have a list of possible uses that can't be done sensibly any other way. * WAL filtering * Recovery when we have buggy index AMs, yet without losing data * Pluggable indexes * Extracting user data from WAL records (very challenging though) Those uses require the ability to both add to *and* modify all of the RmgrTable entries. If this was just for pluggable indexes then the API probably would look a little different, but it's not. The simplicity of the hook proposal says nothing about the careful thought behind it, it just relates to the wide variety of beneficial uses. At any point there we might have hit serious problems with the patch, but we didn't. I've done my best to cover the objections raised with code or suggested control mechanisms, so I'm not expecting anyone to agree with my first musings. > The main thing > that I can see right now that we'd need is some way to determine who > gets which rmgr index. (Maybe community assignment of numbers --- > similar to what we've defined for pg_statistic kind codes --- is fine, http://archives.postgresql.org/pgsql-hackers/2008-08/msg00916.php > or maybe it isn't; in any case we need an answer for that before this > hook can be considered usable.) Furthermore, maybe that's not the only > problem. I'd feel a lot better about this if the hook patch were done > in parallel with development of actual WAL support in an actual external ... I agree we need an external module and I learned that lesson from the earier API proposal you mentioned. The supplied WAL filter plugin was/is a valid use for this and, as discussed, is the only practical way of doing WAL filtering. As I said, am happy to make a few mods to make that more acceptable. I've deferred on this patch sometimes because of my other work, but also because I sensed there might be some feeling that people thought this was a threat to the project from some commercial usurpation (e.g. like InnoDB). I asked to be contacted off-list if that was the case but nobody has, so I have assumed this to be a decision based on technical merit alone. After considering all that has been said I feel this idea has merit. Yes, we need more and better plugins and this patch is the seed for those. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
