On Tue, Jan 26, 2016 at 1:30 AM, Kouhei Kaigai <kai...@ak.jp.nec.com> wrote: > What enhancement will be necessary to implement similar feature of > partial seq-scan using custom-scan interface? > > It seems to me callbacks on the three points below are needed. > * ExecParallelEstimate > * ExecParallelInitializeDSM > * ExecParallelInitializeWorker > > Anything else? > Does ForeignScan also need equivalent enhancement?
For postgres_fdw, running the query from a parallel worker would change the transaction semantics. Suppose you begin a transaction, UPDATE data on the foreign server, and then run a parallel query. If the leader performs the ForeignScan it will see the uncommitted UPDATE, but a worker would have to make its own connection which not be part of the same transaction and which would therefore not see the update. That's a problem. Also, for postgres_fdw, and many other FDWs I suspect, the assumption is that most of the work is being done on the remote side, so doing the work in a parallel worker doesn't seem super interesting. Instead of incurring transfer costs to move the data from remote to local, we incur two sets of transfer costs: first remote to local, then worker to leader. Ouch. I think a more promising line of inquiry is to try to provide asynchronous execution when we have something like: Append -> Foreign Scan -> Foreign Scan ...so that we can return a row from whichever Foreign Scan receives data back from the remote server first. So it's not impossible that an FDW author could want this, but mostly probably not. I think. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
