Hi, On 2014-03-11 15:57:39 -0400, Peter Eisentraut wrote: > Where, if anywhere, is the current documentation for writing or using a > logical decoding output plugin consumer thingy?
There's a pending patch for it. The corresponding commit is http://git.postgresql.org/gitweb/?p=users/andresfreund/postgres.git;a=commit;h=5eeedd55b2d7e53b5fdcdab6a8e74bb666d75bcc I welcome feedback about it. I've spent a fair bit of time immersed in this stuff, and I am not really sure anymore what's understandable and whatnot ;) It's referencing pg_recvlogical which isn't committed yet (that's the commit just before), that's why the docs weren't committed with the feature itself. > src/backend/replication/logical/logical.c, which textually contains most > of the functions that appear to interact with the test_decoding module, > contains this in the header comment: > > """ > The idea is that a consumer provides three callbacks, one to read WAL, > one to prepare a data write, and a final one for actually writing since > their implementation depends on the type of consumer. Check > logicalfunc.c for an example implementations of a fairly simple consumer > and a implementation of a WAL reading callback that's suitable for > simpler consumers. > """ > > There is no file logicalfunc.c. Hrmpf: There's a missing 's', it's logicalfuncs.c. > And test_decoding actually uses five callbacks, not three. The callbacks logicalfuncs.c header comment is talking about adding a new method to output data. Currently you can stream out changes via walsender and via the SQL SRFs. But it might be interesting to e.g. consume the changes in a bgworker, without going through either SQL or walsender. To do that you need the three callbacks referenced above. > Is a consumer the same as a decoder? A consumer is just the recipient of the changestream. I.e the walsender that streams out the changestream, or the SRF that spills the data into a tuplestore. I don't think the term "decoder" is used anywhere, but if it is, it'd be the output plugin. > test_decoding.c contains this: > > /* These must be available to pg_dlsym() */ > static void pg_decode_startup(LogicalDecodingContext *ctx, > OutputPluginOptions *opt, bool is_init); > ... > > which is surely wrong. Hm, these days the comment should be above _PG_init() and _PG_output_plugin_init(). That changed around several times... Could you perhaps commit the attached patch fixing the issues you mentioned? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
diff --git a/contrib/test_decoding/test_decoding.c b/contrib/test_decoding/test_decoding.c index e356c7c..31aa012 100644 --- a/contrib/test_decoding/test_decoding.c +++ b/contrib/test_decoding/test_decoding.c @@ -33,6 +33,7 @@ PG_MODULE_MAGIC; +/* These must be available to pg_dlsym() */ extern void _PG_init(void); extern void _PG_output_plugin_init(OutputPluginCallbacks *cb); @@ -43,7 +44,6 @@ typedef struct bool include_timestamp; } TestDecodingData; -/* These must be available to pg_dlsym() */ static void pg_decode_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt, bool is_init); static void pg_decode_shutdown(LogicalDecodingContext *ctx); diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index 13a22d4..04e407a 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -12,14 +12,17 @@ * together provide logical decoding, primarily by providing so * called LogicalDecodingContexts. The goal is to encapsulate most of the * internal complexity for consumers of logical decoding, so they can - * create and consume a changestream with a low amount of code. + * create and consume a changestream with a low amount of code. Builtin + * consumers are the walsender and SQL SRF interface, but it's possible to + * add further ones without changing core code, e.g. to consume changes in + * a bgworker. * * The idea is that a consumer provides three callbacks, one to read WAL, * one to prepare a data write, and a final one for actually writing since * their implementation depends on the type of consumer. Check - * logicalfunc.c for an example implementations of a fairly simple consumer + * logicalfuncs.c for an example implementation of a fairly simple consumer * and a implementation of a WAL reading callback that's suitable for - * simpler consumers. + * simple consumers. *------------------------------------------------------------------------- */
-- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
