Re: [HACKERS] Custom Scans and private data

2015-08-27 Thread Andres Freund
On 2015-08-27 23:23:54 +0100, Robert Haas wrote: > I think it's great that you are experimenting with the feature, and I > think we ought to try to make it better, even if that requires > incompatible changes to the API. Yea, I think it's too early to consider this API stable at all. That's imo ju

Re: [HACKERS] Custom Scans and private data

2015-08-27 Thread Andres Freund
Hi, On 2015-08-27 18:59:13 -0400, Tom Lane wrote: > But I do think that letting custom-scan extensions fool about with > the semantics of plan-copying (and plan-serialization for that matter) > is a risky choice that is not justified by some marginal gains in code > readability To me the likeliho

Re: [HACKERS] Custom Scans and private data

2015-08-27 Thread Tom Lane
Robert Haas writes: > On Wed, Aug 26, 2015 at 11:23 PM, Andres Freund wrote: >> While that comment made me laugh, I'm not really sure why my examples >> are bait. One is the probably most used fdw, the other the only user of >> the custom scan interface I know of. > I suspect what Tom is getting

Re: [HACKERS] Custom Scans and private data

2015-08-27 Thread Robert Haas
On Wed, Aug 26, 2015 at 11:23 PM, Andres Freund wrote: >> > Looking at >> > postgres_fdw and the pg-strom example linked upthread imo pretty clearly >> > shows how ugly this is. There's also the rather noticeable difference >> > that we already have callbacks in the node for custom scans (used by

Re: [HACKERS] Custom Scans and private data

2015-08-27 Thread Andres Freund
On 2015-08-26 23:55:16 +, Kouhei Kaigai wrote: > - _copyCustomScan() needs to allow to allocate larger than sizeof(CustomScan), > according to the requirement by custom-scan provider. I think it's somewhat nice to allow larger objects, but I don't think it's absolutely necessary. > - We may

Re: [HACKERS] Custom Scans and private data

2015-08-26 Thread Kouhei Kaigai
> On 2015-08-26 00:55:48 +, Kouhei Kaigai wrote: > > As Tom pointed out, the primary reason why CustomScan required provider > > to save its private data on custom_exprs/custom_private were awareness > > of copyObject(). > > Well, a callback brings that with it as well. I do think it makes sen

Re: [HACKERS] Custom Scans and private data

2015-08-26 Thread Andres Freund
On 2015-08-26 00:55:48 +, Kouhei Kaigai wrote: > As Tom pointed out, the primary reason why CustomScan required provider > to save its private data on custom_exprs/custom_private were awareness > of copyObject(). Well, a callback brings that with it as well. I do think it makes sense to *allow

Re: [HACKERS] Custom Scans and private data

2015-08-26 Thread Andres Freund
On 2015-08-25 19:22:04 -0400, Tom Lane wrote: > Andres Freund writes: > > I think it was a noticeable mistake in the fdw case, but we already > > released with that. We shouldn't make the same mistake twice. > > I don't agree that it was a mistake, and I do think there is value in > consistency.

Re: [HACKERS] Custom Scans and private data

2015-08-25 Thread Kouhei Kaigai
> On 2015-08-25 14:42:32 -0400, Tom Lane wrote: > > Andres Freund writes: > > > Since we already have CustomScan->methods, it seems to be rather > > > reasonable to have a CopyCustomScan callback and let it do the copying > > > of the private data if present? Or possibly of the whole node, which'd

Re: [HACKERS] Custom Scans and private data

2015-08-25 Thread Tom Lane
Andres Freund writes: > On 2015-08-25 14:42:32 -0400, Tom Lane wrote: >> In any case, since this convention already exists for FDWs I'm not >> sure why we should make it different for CustomScan. > I think it was a noticeable mistake in the fdw case, but we already > released with that. We should

Re: [HACKERS] Custom Scans and private data

2015-08-25 Thread Andres Freund
On 2015-08-25 14:42:32 -0400, Tom Lane wrote: > Andres Freund writes: > > Since we already have CustomScan->methods, it seems to be rather > > reasonable to have a CopyCustomScan callback and let it do the copying > > of the private data if present? Or possibly of the whole node, which'd > > allow

Re: [HACKERS] Custom Scans and private data

2015-08-25 Thread Tom Lane
Andres Freund writes: > Since we already have CustomScan->methods, it seems to be rather > reasonable to have a CopyCustomScan callback and let it do the copying > of the private data if present? Or possibly of the whole node, which'd > allow to embed it into a bigger node? Weren't there rumbling

[HACKERS] Custom Scans and private data

2015-08-25 Thread Andres Freund
(sending again, forgot to cc hackers, sorry for the duplicate) Hi, I'm trying to use the custom scan API to replace code that currently does everything via hooks and isn't safe against copyObject() (Yes, that's not a grand plan). To me dealing with CustomScan->custom_private seems to be extraord