[HACKERS] Arguments to foreign tables?

2012-11-04 Thread Jeff Davis
Is there any fundamental or philosophical reason why a foreign table can't accept arguments? Should that be a TODO? Right now, to accept arguments to a from-clause item, you have to use an SRF, which is much more limited than a foreign table. If foreign tables could accept arguments, then SRFs cou

Re: [HACKERS] Arguments to foreign tables?

2012-11-04 Thread Tom Lane
Jeff Davis writes: > Is there any fundamental or philosophical reason why a foreign table > can't accept arguments? That isn't a table; it's some sort of function. Now that we have LATERAL, there is no good reason to contort SQL's syntax and semantics in the direction you suggest.

Re: [HACKERS] Arguments to foreign tables?

2012-11-04 Thread Willem Leenen
[HACKERS] Arguments to foreign tables? > From: pg...@j-davis.com > To: pgsql-hackers@postgresql.org > Date: Sun, 4 Nov 2012 11:59:20 -0800 > > Is there any fundamental or philosophical reason why a foreign table > can't accept arguments? Should that be a TODO? > > Rig

Re: [HACKERS] Arguments to foreign tables?

2012-11-04 Thread John R Pierce
On 11/04/12 11:59 AM, Jeff Davis wrote: Is there any fundamental or philosophical reason why a foreign table can't accept arguments? Should that be a TODO? what does that even mean?how would 'data' accept 'arguments' ??! -- john r pierceN 37, W 122 santa cruz

Re: [HACKERS] Arguments to foreign tables?

2012-11-04 Thread Willem Leenen
> > what does that even mean?how would 'data' accept 'arguments' ??! > > That's Sherlock Holmes! "Don't argue with your data"

Re: [HACKERS] Arguments to foreign tables?

2012-11-05 Thread Jeff Davis
On Sun, 2012-11-04 at 15:13 -0500, Tom Lane wrote: > Jeff Davis writes: > > Is there any fundamental or philosophical reason why a foreign table > > can't accept arguments? > > That isn't a table; it's some sort of function. Now that we have > LATERAL, there is no good reason to contort SQL's sy

Re: [HACKERS] Arguments to foreign tables?

2012-11-06 Thread Ronan Dunklau
Maybe you could set some options on the foreign table before selecting from it ? Another way you could achieve the same result would be to give some column a special meaning (like it is done in the twitter_fdw for example). If you don't mind, do you have a specific use-case for this ? -- Ronan Du

Re: [HACKERS] Arguments to foreign tables?

2012-11-06 Thread Dimitri Fontaine
Jeff Davis writes: > Take something as simple as generate_series: right now, it materializes > the entire thing if it's in the FROM clause, but it wouldn't need to if > it could use the foreign table mechanism. So, my understanding of your proposal is that a good way to implement streaming SRF wo

Re: [HACKERS] Arguments to foreign tables?

2012-11-06 Thread Simon Riggs
On 6 November 2012 06:09, Jeff Davis wrote: > On Sun, 2012-11-04 at 15:13 -0500, Tom Lane wrote: >> Jeff Davis writes: >> > Is there any fundamental or philosophical reason why a foreign table >> > can't accept arguments? >> >> That isn't a table; it's some sort of function. Now that we have >>

Re: [HACKERS] Arguments to foreign tables?

2012-11-06 Thread Tom Lane
Dimitri Fontaine writes: > Jeff Davis writes: >> Take something as simple as generate_series: right now, it materializes >> the entire thing if it's in the FROM clause, but it wouldn't need to if >> it could use the foreign table mechanism. > So, my understanding of your proposal is that a good

Re: [HACKERS] Arguments to foreign tables?

2012-11-06 Thread Tom Lane
Simon Riggs writes: > (On another thought: why do Foreign Scans get to avoid the > materialization we hate with SRFs?) [ shrug... ] That works both ways. If it would be convenient for an FDW to return a tuplestore, which is hardly unlikely, it's out of luck; it has to do the expansion itself.

Re: [HACKERS] Arguments to foreign tables?

2012-11-06 Thread Jeff Davis
On Tue, 2012-11-06 at 08:35 +, Simon Riggs wrote: > Tom is complaining about conflating the two features, which does seem > unnecessary. But we can still merge code. That's a good way to put it. > Providing run-time parameters or table-level options to Foreign Scans > seems sensible. I can se

Re: [HACKERS] Arguments to foreign tables?

2012-11-06 Thread Jeff Davis
On Tue, 2012-11-06 at 09:39 +0100, Dimitri Fontaine wrote: > Jeff Davis writes: > > Take something as simple as generate_series: right now, it materializes > > the entire thing if it's in the FROM clause, but it wouldn't need to if > > it could use the foreign table mechanism. > > So, my understa

Re: [HACKERS] Arguments to foreign tables?

2012-11-06 Thread Jeff Davis
On Tue, 2012-11-06 at 09:19 +0100, Ronan Dunklau wrote: > Maybe you could set some options on the foreign table before selecting > from it ? > Another way you could achieve the same result would be to give some > column a special meaning (like it is done in the twitter_fdw for > example). > > > I

Re: [HACKERS] Arguments to foreign tables?

2012-11-06 Thread Pavel Stehule
2012/11/6 Jeff Davis : > On Tue, 2012-11-06 at 09:19 +0100, Ronan Dunklau wrote: >> Maybe you could set some options on the foreign table before selecting >> from it ? >> Another way you could achieve the same result would be to give some >> column a special meaning (like it is done in the twitter_

Re: [HACKERS] Arguments to foreign tables?

2012-11-06 Thread Jeff Davis
On Tue, 2012-11-06 at 10:55 -0500, Tom Lane wrote: > I'd also opine that the FDW APIs are pretty darn heavyweight for an SRF. > There might be a small number of SRFs for which it's actually worth > dealing with the planner in full generality, but surely not very many. I was not suggesting that we

Re: [HACKERS] Arguments to foreign tables?

2012-11-06 Thread Tom Lane
Jeff Davis writes: > On Tue, 2012-11-06 at 10:55 -0500, Tom Lane wrote: >> I'd also opine that the FDW APIs are pretty darn heavyweight for an SRF. >> There might be a small number of SRFs for which it's actually worth >> dealing with the planner in full generality, but surely not very many. > I

Re: [HACKERS] Arguments to foreign tables?

2012-11-06 Thread Tom Lane
Jeff Davis writes: > On Tue, 2012-11-06 at 09:19 +0100, Ronan Dunklau wrote: >> If you don't mind, do you have a specific use-case for this ? > Let's say that your remote data source is a stream of data that is not > actually being stored anywhere, e.g. network events. The data you want > to retr

Re: [HACKERS] Arguments to foreign tables?

2012-11-06 Thread Jeff Davis
On Tue, 2012-11-06 at 12:57 -0500, Tom Lane wrote: > That particular example can be handled perfectly well today, with > > select * from stream_table where tscol < 'whatever'; > > The FDW could be coded to throw an error if the query doesn't provide a > WHERE clause that constrains the time