On Mon, Mar 9, 2015 at 11:18 PM, Kouhei Kaigai <kai...@ak.jp.nec.com> wrote:
> The attached patch integrates a suggestion from Ashutosh Bapat.
> It allows to track set of relations involved in a join, but
> replaced by foreign-/custom-scan. It enables to make correct
> EXPLAIN output, if FDW/CSP driver makes human readable symbols
> using deparse_expression() or others.
>
> Differences from v7 is identical with what I posted on the
> join push-down support thread.
I took a look at this patch today and noticed that it incorporates not
only documentation for the new functionality it adds, but also for the
custom-scan functionality whose documentation I previously excluded
from commit on the grounds that it needed more work, especially to
improve the English. That decision was not popular at the time, and I
think we need to remedy it before going further with this. I had
hoped that someone else would care about this work enough to help with
the documentation, but it seems not, so today I went through the
documentation in this patch, excluded all of the stuff specific to
custom joins, and heavily edited the rest. The result is attached.
If there are no objections, I'll commit this; then, someone can rebase
this patch over these changes and we can proceed from there.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/doc/src/sgml/custom-scan.sgml b/doc/src/sgml/custom-scan.sgml
new file mode 100644
index 0000000..fa6f457
--- /dev/null
+++ b/doc/src/sgml/custom-scan.sgml
@@ -0,0 +1,284 @@
+<!-- doc/src/sgml/custom-scan.sgml -->
+
+<chapter id="custom-scan">
+ <title>Writing A Custom Scan Provider</title>
+
+ <indexterm zone="custom-scan">
+ <primary>custom scan provider</primary>
+ <secondary>handler for</secondary>
+ </indexterm>
+
+ <para>
+ <productname>PostgreSQL</> supports a set of experimental facilities which
+ are intended to allow extension modules to add new scan types to the system.
+ Unlike a <link linkend="fdwhandler">foreign data wrapper</>, which is only
+ responsible for knowing how to scan its own foreign tables, a custom scan
+ provider can provide an alternative method of scanning any relation in the
+ system. Typically, the motivation for writing a custom scan provider will
+ be to allow the use of some optimization not supported by the core
+ system, such as caching or some form of hardware acceleration. This chapter
+ outlines how to write a new custom scan provider.
+ </para>
+
+ <para>
+ Implementing a new type of custom scan is a three-step process. First,
+ during planning, it is necessary to generate access paths representing a
+ scan using the proposed strategy. Second, if one of those access paths
+ is selected by the planner as the optimal strategy for scanning a
+ particular relation, the access path must be converted to a plan.
+ Finally, it must be possible to execute the plan and generate the same
+ results that would have been generated for any other access path targeting
+ the same relation.
+ </para>
+
+ <sect1 id="custom-scan-path">
+ <title>Implementing Custom Paths</title>
+
+ <para>
+ A custom scan provider will typically add paths by setting the following
+ hook, which is called after the core code has generated what it believes
+ to be the complete and correct set of access paths for the relation.
+<programlisting>
+typedef void (*set_rel_pathlist_hook_type) (PlannerInfo *root,
+ RelOptInfo *rel,
+ Index rti,
+ RangeTblEntry *rte);
+extern PGDLLIMPORT set_rel_pathlist_hook_type set_rel_pathlist_hook;
+</programlisting>
+ </para>
+
+ <para>
+ Although this hook function can be used to examine, modify, or remove
+ paths generated by the core system, a custom scan provider will typically
+ confine itself to generating <structname>CustomPath</> objects and adding
+ them to <literal>rel</> using <function>add_path</>. The custom scan
+ provider is responsible for initializing the <structname>CustomPath</>
+ object, which is declared like this:
+<programlisting>
+typedef struct CustomPath
+{
+ Path path;
+ uint32 flags;
+ List *custom_private;
+ const CustomPathMethods *methods;
+} CustomPath;
+</programlisting>
+ </para>
+
+ <para>
+ <structfield>path</> must be initialized as for any other path, including
+ the row-count estimate, start and total cost, and sort ordering provided
+ by this path. <structfield>flags</> is a bitmask, which should include
+ <literal>CUSTOMPATH_SUPPORT_BACKWARD_SCAN</> if the custom path can support
+ a backward scan and <literal>CUSTOMPATH_SUPPORT_MARK_RESTORE</> if it
+ can support mark and restore. Both capabilities are optional.
+ <structfield>custom_private</> can be used to store the custom path's
+ private data. Private data should be stored in a form that can be handled
+ by <literal>nodeToString</>, so that debugging routines which attempt to
+ print the custom path will work as designed. <structfield>methods</> must
+ point to a (usually statically allocated) object implementing the required
+ custom path methods, of which there are currently only two, as further
+ detailed below.
+ </para>
+
+ <sect2 id="custom-scan-path-callbacks">
+ <title>Custom Path Callbacks</title>
+
+ <para>
+<programlisting>
+Plan *(*PlanCustomPath) (PlannerInfo *root,
+ RelOptInfo *rel,
+ CustomPath *best_path,
+ List *tlist,
+ List *clauses);
+</programlisting>
+ Convert a custom path to a finished plan. The return value will generally
+ be a <literal>CustomScan</> object, which the callback must allocate and
+ initialize. See <xref linkend="custom-scan-plan"> for more details.
+ </para>
+
+ <para>
+<programlisting>
+void (*TextOutCustomPath) (StringInfo str,
+ const CustomPath *node);
+</programlisting>
+ Generate additional output when <function>nodeToString</> is invoked on
+ this custom path. This callback is optional. Since
+ <function>nodeToString</> will automatically dump all fields in the
+ structure that it can see, including <structfield>custom_private</>, this
+ is only useful if the <structname>CustomPath</> is actually embedded in a
+ larger struct containing additional fields.
+ </para>
+ </sect2>
+ </sect1>
+
+ <sect1 id="custom-scan-plan">
+ <title>Implementing Custom Plans</title>
+
+ <para>
+ A custom scan is represented in a finished plan tree using the following
+ structure:
+<programlisting>
+typedef struct CustomScan
+{
+ Scan scan;
+ uint32 flags;
+ List *custom_exprs;
+ List *custom_private;
+ const CustomScanMethods *methods;
+} CustomScan;
+</programlisting>
+ </para>
+
+ <para>
+ <structfield>scan</> must be initialized as for any other scan, including
+ estimated costs, target lists, qualifications, and so on.
+ <structfield>flags</> is a bitmask with the same meaning as in
+ <structname>CustomPath</>. <structfield>custom_exprs</> should be used to
+ store expression trees that will need to be fixed up by
+ <filename>setrefs.c</> and <filename>subselect.c</>, while
+ <literal>custom_private</> should be used to store other private data that
+ is only used by the custom scan provider itself. Plan trees must be able
+ to be duplicated using <function>copyObject</>, so all the data stored
+ within these two fields must consist of nodes that function can handle.
+ <structfield>methods</> must point to a (usually statically allocated)
+ object implementing the required custom scan methods, which are further
+ detailed below.
+ </para>
+
+ <sect2 id="custom-scan-plan-callbacks">
+ <title>Custom Scan Callbacks</title>
+ <para>
+<programlisting>
+Node *(*CreateCustomScanState) (CustomScan *cscan);
+</programlisting>
+ Allocate a <structname>CustomScanState</> for this
+ <structname>CustomScan</>. The actual allocation will often be larger than
+ required for an ordinary <structname>CustomScanState</>, because many
+ scan types will wish to embed that as the first field of a large structure.
+ The value returned must have the node tag and <structfield>methods</>
+ set appropriately, but the other fields need not be initialized at this
+ stage; after <function>ExecInitCustomScan</> performs basic initialization,
+ the <function>BeginCustomScan</> callback will be invoked to give the
+ custom scan state a chance to do whatever else is needed.
+ </para>
+
+ <para>
+<programlisting>
+void (*TextOutCustomScan) (StringInfo str,
+ const CustomScan *node);
+</programlisting>
+ Generate additional output when <function>nodeToString</> is invoked on
+ this custom plan. This callback is optional. Since a
+ <structname>CustomScan</> must be copyable by <function>copyObject</>,
+ custom scan providers cannot substitute a larger structure that embeds a
+ <structname>CustomScan</> for the structure itself, as would be possible
+ for a <structname>CustomPath</> or <structname>CustomScanState</>.
+ Therefore, providing this callback is unlikely to be useful.
+ </para>
+ </sect2>
+ </sect1>
+
+ <sect1 id="custom-scan-scan">
+ <title>Implementing Custom Scans</title>
+
+ <para>
+ When a <structfield>CustomScan</> is executed, its execution state is
+ represented by a <structfield>CustomScanState</>, which is declared as
+ follows.
+<programlisting>
+typedef struct CustomScanState
+{
+ ScanState ss;
+ uint32 flags;
+ const CustomExecMethods *methods;
+} CustomScanState;
+</programlisting>
+ </para>
+
+ <para>
+ <structfield>ss</> must be initialized as for any other scanstate;
+ <structfield>flags</> is a bitmask with the same meaning as in
+ <structname>CustomPath</> and <structname>CustomScan</>.
+ <structfield>methods</> must point to a (usually statically allocated)
+ object implementing the required custom scan state methods, which are
+ further detailed below. Typically, a <structname>CustomScanState</>, which
+ need not support <function>copyObject</>, will actually be a larger
+ structure embedding the above as its first member.
+ </para>
+
+ <sect2 id="custom-scan-scan-callbacks">
+ <title>Custom Execution-Time Callbacks</title>
+
+ <para>
+<programlisting>
+void (*BeginCustomScan) (CustomScanState *node,
+ EState *estate,
+ int eflags);
+</programlisting>
+ Complete initalization of the supplied <structname>CustomScanState</>.
+ Some initialization is performed by <function>ExecInitCustomScan</>, but
+ any private fields should be initialized here.
+ </para>
+
+ <para>
+<programlisting>
+TupleTableSlot *(*ExecCustomScan) (CustomScanState *node);
+</programlisting>
+ Fetch the next scan tuple. If any tuples remain, it should set
+ <literal>ps_ResultTupleSlot</> and then returns the tuple slot. If not,
+ <literal>NULL</> or an empty slot should be returned.
+ </para>
+
+ <para>
+<programlisting>
+void (*EndCustomScan) (CustomScanState *node);
+</programlisting>
+ Clean up any private data associated with the <literal>CustomScanState</>.
+ This method is required, but may not need to do anything if the associated
+ data does not exist or will be cleaned up automatically.
+ </para>
+
+ <para>
+<programlisting>
+void (*ReScanCustomScan) (CustomScanState *node);
+</programlisting>
+ Rewind the current scan to the beginning and prepare to rescan the
+ relation.
+ </para>
+
+ <para>
+<programlisting>
+void (*MarkPosCustomScan) (CustomScanState *node);
+</programlisting>
+ Save the current scan position so that it can subsequently be restored
+ by the <function>RestrPosCustomScan</> callback. This calback is optional,
+ and need only be supplied if
+ <literal>CUSTOMPATH_SUPPORT_MARK_RESTORE</> flag is set.
+ </para>
+
+ <para>
+<programlisting>
+void (*RestrPosCustomScan) (CustomScanState *node);
+</programlisting>
+ Restore the previous scan position as saved by the
+ <function>MarkPosCustomScan</> callback. This callback is optional,
+ and need only be supplied if
+ <literal>CUSTOMPATH_SUPPORT_MARK_RESTORE</> flag is set.
+ </para>
+
+ <para>
+<programlisting>
+void (*ExplainCustomScan) (CustomScanState *node,
+ List *ancestors,
+ ExplainState *es);
+</programlisting>
+ Output additional information on <command>EXPLAIN</> that involves
+ custom-scan node. This callback is optional. Common data stored in the
+ <structname>ScanState</>, such as the target list and scan relation, will
+ be shown even without this callback, but the callback allows the display
+ of additional, private state.
+ </para>
+ </sect2>
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index f03b72a..89fff77 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -93,6 +93,7 @@
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
+<!ENTITY custom-scan SYSTEM "custom-scan.sgml">
<!ENTITY logicaldecoding SYSTEM "logicaldecoding.sgml">
<!ENTITY protocol SYSTEM "protocol.sgml">
<!ENTITY sources SYSTEM "sources.sgml">
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index a648a4c..e378d69 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -242,6 +242,7 @@
&nls;
&plhandler;
&fdwhandler;
+ &custom-scan;
&geqo;
&indexam;
&gist;
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
