Re: [HACKERS] idea: global temp tables

2009-05-12 Thread Bruce Momjian
Robert Haas wrote: On Thu, Apr 30, 2009 at 8:36 AM, Peter Eisentraut pete...@gmx.net wrote: The archives for this thread http://archives.postgresql.org//pgsql-hackers/2009-04/threads.php#01329 show a bunch of missing messages. ?Were they being stored in a temporary table?

Re: [HACKERS] idea: global temp tables

2009-05-01 Thread Tom Lane
Greg Stark st...@enterprisedb.com writes: Well I claim it's not just a nice bonus but is the difference between implementing something which falls technically within the standard's rules but fails to actually be useful for the standard's intended purpose. I agree with Kevin's objection that

Re: [HACKERS] idea: global temp tables

2009-05-01 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: Any chance that some of these improvements could be applied to temp tables created with the PostgreSQL-specific syntax while we're at it? You mean the Postgres-specific behavior, no? Trying to support

Re: [HACKERS] idea: global temp tables

2009-05-01 Thread Greg Stark
On Wed, Apr 29, 2009 at 6:42 PM, Tom Lane t...@sss.pgh.pa.us wrote: Greg Stark st...@enterprisedb.com writes: Well I claim it's not just a nice bonus but is the difference between implementing something which falls technically within the standard's rules but fails to actually be useful for the

Re: [HACKERS] idea: global temp tables

2009-05-01 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: Any chance that some of these improvements could be applied to temp tables created with the PostgreSQL-specific syntax while we're at it? You mean the Postgres-specific behavior, no? Trying to support a table without *any* pre-existing

Re: [HACKERS] idea: global temp tables

2009-05-01 Thread Kevin Grittner
Peter Eisentraut pete...@gmx.net wrote: There are actually two orthogonal properties at work here: How the table is visible with respect to modules (LOCAL/GLOBAL) and whether the table disappears at the end of the session (currently yes, proposed new behavior optionally no). Pavel's

Re: [HACKERS] idea: global temp tables

2009-05-01 Thread Greg Stark
On Wed, Apr 29, 2009 at 4:24 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: I understand not everyone has a need for such a thing, but isn't that true of most features in PostgreSQL? Well I think implementing a feature which only works if it's used at low transaction rates as a

Re: [HACKERS] idea: global temp tables

2009-05-01 Thread Peter Eisentraut
On Tuesday 28 April 2009 19:38:25 Tom Lane wrote: One thing I just noticed is that the spec does not consider GLOBAL/LOCAL to be optional --- per spec you *must* write one or the other in front of TEMPORARY. So we could adopt the view that omitting this keyword implies our current non-spec

Re: [HACKERS] idea: global temp tables

2009-05-01 Thread Zdenek Kotala
Dne 28.04.09 16:59, Alvaro Herrera napsal(a): Pavel Stehule escribió: Maybe we could make this work by fiddling with a different smgr -- on it, smgr_sync would be a noop, as would smgr_immedsync, and we could kludge something up to truncate relations during recovery. Maybe set path like

Re: [HACKERS] idea: global temp tables

2009-05-01 Thread Kevin Grittner
Peter Eisentraut pete...@gmx.net wrote: On the matter of LOCAL/GLOBAL, I think the correct thing to do is to reject LOCAL and accept GLOBAL as equivalent to the default. Oops. Read that too quickly before my first reply. I think that we should, as Tom said, warn on *both* for 8.4, and

Re: [HACKERS] idea: global temp tables

2009-05-01 Thread Peter Eisentraut
The archives for this thread http://archives.postgresql.org//pgsql-hackers/2009-04/threads.php#01329 show a bunch of missing messages. Were they being stored in a temporary table? Anywhere, here is what I had meant to say but only got through to a few ... On Tuesday 28 April 2009 19:38:25

Re: [HACKERS] idea: global temp tables

2009-05-01 Thread Kevin Grittner
James Mansion ja...@mansionfamily.plus.com wrote: Wouldn't it be cleaner just to defer creation of real files to support the structures associated with a temp table until it i snecessary to spill the data from the backend's RAM? This data doesn't need to be in shared memory and the tables

Re: [HACKERS] idea: global temp tables

2009-05-01 Thread Robert Haas
On Thu, Apr 30, 2009 at 8:36 AM, Peter Eisentraut pete...@gmx.net wrote: The archives for this thread http://archives.postgresql.org//pgsql-hackers/2009-04/threads.php#01329 show a bunch of missing messages.  Were they being stored in a temporary table? Anywhere, here is what I had meant

Re: [HACKERS] idea: global temp tables

2009-04-30 Thread James Mansion
Kevin Grittner wrote: contexts. I don't think the updates to the system tables have the same magnitude of performance hit as creating these tables, especially if write barriers are on. Wouldn't it be cleaner just to defer creation of real files to support the structures associated with a

Re: [HACKERS] idea: global temp tables

2009-04-30 Thread Kevin Grittner
Greg Stark st...@enterprisedb.com wrote: creating/deleting a few dozen rows in the system catalogs shouldn't really be something that autovacuum can't deal with. I don't see why it's limited to a few dozen rows. Moderately busy web sites these days count their traffic in hundreds of page

Re: [HACKERS] idea: global temp tables

2009-04-30 Thread Kevin Grittner
Greg Stark st...@enterprisedb.com wrote: On Wed, Apr 29, 2009 at 6:42 PM, Tom Lane t...@sss.pgh.pa.us wrote: I agree with Kevin's objection that you are unfairly raising the bar for this feature by demanding a performance improvement to go along with a functionality change. I think

Re: [HACKERS] idea: global temp tables

2009-04-30 Thread Pavel Stehule
2009/4/29 Tom Lane t...@sss.pgh.pa.us: Greg Stark st...@enterprisedb.com writes: Well I claim it's not just a nice bonus but is the difference between implementing something which falls technically within the standard's rules but fails to actually be useful for the standard's intended

Re: [HACKERS] idea: global temp tables

2009-04-29 Thread Alvaro Herrera
Pavel Stehule escribió: But question? about MVCC? Is necessary to use MVCC on pg_statistic and some columns from pg_proc? Right now, we use MVCC everywhere because we have no choice. But there are some things that would be better off not using MVCC; and indeed we use a hack to make things

Re: [HACKERS] idea: global temp tables

2009-04-29 Thread Kevin Grittner
Greg Stark st...@enterprisedb.com wrote: On Wed, Apr 29, 2009 at 7:57 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: But the scaling issues are there already with temp tables. This actually makes it better, not worse, because the table can be materialized once per session, not once

Re: [HACKERS] idea: global temp tables

2009-04-29 Thread Tom Lane
Greg Stark st...@enterprisedb.com writes: I don't understand what you mean by a cost once per request. You only have to create the temporary table on the first request. If you can't tell which is the first request you only have to test whether it exists which doesn't incur the consequences

Re: [HACKERS] idea: global temp tables

2009-04-29 Thread Greg Stark
On Tue, Apr 28, 2009 at 6:18 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Tom Lane t...@sss.pgh.pa.us wrote: I was thinking in terms of throwing a warning in 8.4 and implementing new behavior in 8.5. If we're on that time frame with it, I guess it's not too early to suggest what we

Re: [HACKERS] idea: global temp tables

2009-04-29 Thread Kevin Grittner
Greg Stark st...@enterprisedb.com wrote: I've been thinking about Alvaro's idea of a separate smgr. If you had a single pg_class entry for all sessions but the smgr knew to store the actual data for it in a session-local file, either in a session-specific tablespace or using the same

Re: [HACKERS] idea: global temp tables

2009-04-29 Thread Kevin Grittner
Greg Stark st...@enterprisedb.com wrote: The whole point of having the schema declared in advance and then having each procedure execution have access to a private (or non-private) data store following that predefined schema is to avoid having to execute any catalog changes with all the

Re: [HACKERS] idea: global temp tables

2009-04-29 Thread Greg Stark
On Wed, Apr 29, 2009 at 8:59 PM, Tom Lane t...@sss.pgh.pa.us wrote: This is all based on utterly-unproven assumptions about relative costs. In particular, ISTM an additional network round trip or two associated with testing for/creating a temp table could easily swamp any costs associated

Re: [HACKERS] idea: global temp tables

2009-04-29 Thread Greg Stark
On Wed, Apr 29, 2009 at 7:57 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: But the scaling issues are there already with temp tables.  This actually makes it better, not worse, because the table can be materialized once per session, not once per request. Currently you have to issue

Re: [HACKERS] idea: global temp tables

2009-04-28 Thread Alvaro Herrera
Pavel Stehule escribió: Hello I am thinking about global temp tables. One possible solution is creating global temporary table like normal table and in planner stage check using this table. When some global temporary table is detected, then real temporary table is created and used in

Re: [HACKERS] idea: global temp tables

2009-04-28 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Maybe we could make this work by fiddling with a different smgr -- on it, smgr_sync would be a noop, as would smgr_immedsync, and we could kludge something up to truncate relations during recovery. Interesting thought but I think it falls down

Re: [HACKERS] idea: global temp tables

2009-04-28 Thread vacuum
Perhaps call them session tables instead of temp tables? regards, tom lane Or transient table ... Maybe we can define when such table lose data But in real - there is no need in this feature - databases are made to hold data, not to lose. If an application requires

Re: [HACKERS] idea: global temp tables

2009-04-28 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: we have already concluded that the spec's GLOBAL/LOCAL TEMP TABLE distinction is not related to cross-session persistence of the table definitions How do you reconcile that conclusion with the following, from ISO/IEC 9075-2:2003 (E), 4.14 Tables: The

Re: [HACKERS] idea: global temp tables

2009-04-28 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: it could be considered either a global or a local temp table per spec (without any module support you can't really say which it is). That seems bogus -- without modules it is clearly not LOCAL. What Pavel is requesting exactly matches the spec's

Re: [HACKERS] idea: global temp tables

2009-04-28 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: Tom Lane t...@sss.pgh.pa.us wrote: we have already concluded that the spec's GLOBAL/LOCAL TEMP TABLE distinction is not related to cross-session persistence of the table definitions How do you reconcile that conclusion with the following,

Re: [HACKERS] idea: global temp tables

2009-04-28 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: GLOBAL/LOCAL TEMP TABLE distinction is not related to cross-session persistence of the table definitions On a re-read, I think I see your point -- it is the DECLARE LOCAL TEMP TABLE versus CREATE { GLOBAL | LOCAL } TEMP TABLE which determines whether the

Re: [HACKERS] idea: global temp tables

2009-04-28 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: ... Both forms of CREATE TEMP TABLE should persist the definition if you go by the standard, so you don't want to muddy the waters by complying on one and not the other? Right. This goes back to our old principle of trying not to use

Re: [HACKERS] idea: global temp tables

2009-04-28 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: Tom Lane t...@sss.pgh.pa.us wrote: it could be considered either a global or a local temp table per spec (without any module support you can't really say which it is). That seems bogus -- without modules it is clearly not LOCAL. You could

Re: [HACKERS] idea: global temp tables

2009-04-28 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: One thing I just noticed is that the spec does not consider GLOBAL/LOCAL to be optional --- per spec you *must* write one or the other in front of TEMPORARY. Agreed. So we could adopt the view that omitting this keyword implies our current non-spec

Re: [HACKERS] idea: global temp tables

2009-04-28 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: Tom Lane t...@sss.pgh.pa.us wrote: However, if we're going to do that then we should start throwing warnings for use of the keywords, preferably before the release in which they actually start doing something different. We might actually

Re: [HACKERS] idea: global temp tables

2009-04-28 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: I was thinking in terms of throwing a warning in 8.4 and implementing new behavior in 8.5. If we're on that time frame with it, I guess it's not too early to suggest what we would implement in 8.5. I would suggest that it is clear that what Pavel is

[HACKERS] idea: global temp tables

2009-04-27 Thread Pavel Stehule
Hello I am thinking about global temp tables. One possible solution is creating global temporary table like normal table and in planner stage check using this table. When some global temporary table is detected, then real temporary table is created and used in execution plan. It's like: CREATE

Re: [HACKERS] idea: global temp tables

2009-04-27 Thread Kevin Grittner
Pavel Stehule pavel.steh...@gmail.com wrote: I am thinking about global temp tables. These would have some value to us. In case anyone doesn't know, this is a feature in the SQL standard. You have a permanent definition of the schema, but the table is materialized as a temporary table on

Re: [HACKERS] idea: global temp tables

2009-04-27 Thread Andrew Dunstan
Kevin Grittner wrote: Pavel Stehule pavel.steh...@gmail.com wrote: I am thinking about global temp tables. These would have some value to us. In case anyone doesn't know, this is a feature in the SQL standard. You have a permanent definition of the schema, but the table is

Re: [HACKERS] idea: global temp tables

2009-04-27 Thread Kevin Grittner
Andrew Dunstan and...@dunslane.net wrote: Or perhaps Pavel doesn't really mean global as the term is used in Postgres (c.f. the pg_database table)? I'd bet that he doesn't. He's taking terminology from the standard, where it means not limited to one SQL-client module. It just means it is

Re: [HACKERS] idea: global temp tables

2009-04-27 Thread A.M.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Apr 27, 2009, at 4:44 PM, Pavel Stehule wrote: Hello I am thinking about global temp tables. One possible solution is creating global temporary table like normal table and in planner stage check

Re: [HACKERS] idea: global temp tables

2009-04-27 Thread Dimitri Fontaine
Hi, Le 27 avr. 09 à 23:32, A.M. a écrit : When will postgresql offer global temporary tables with data which are shared among sessions? Such tables are great for transient data such as web session data where writing to the WAL is a waste. (On DB startup, the tables would simply be empty.)

Re: [HACKERS] idea: global temp tables

2009-04-27 Thread Kevin Grittner
A.M. age...@themactionfaction.com wrote: When will postgresql offer global temporary tables with data which are shared among sessions? Well, that would certainly be far different from what the standard calls a temporary table of any flavor. In the standard all temporary tables are

Re: [HACKERS] idea: global temp tables

2009-04-27 Thread A.M.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Apr 27, 2009, at 5:39 PM, Dimitri Fontaine wrote: Hi, Le 27 avr. 09 à 23:32, A.M. a écrit : When will postgresql offer global temporary tables with data which are shared among sessions? Such tables are great for transient data such as web

Re: [HACKERS] idea: global temp tables

2009-04-27 Thread Kevin Grittner
A.M. age...@themactionfaction.com wrote: On Apr 27, 2009, at 5:39 PM, Dimitri Fontaine wrote: Le 27 avr. 09 à 23:32, A.M. a écrit : When will postgresql offer global temporary tables with data which are shared among sessions? Such tables are great for transient data such as web session

Re: [HACKERS] idea: global temp tables

2009-04-27 Thread A.M.
On Apr 27, 2009, at 6:01 PM, Kevin Grittner wrote: A.M. age...@themactionfaction.com wrote: On Apr 27, 2009, at 5:39 PM, Dimitri Fontaine wrote: Le 27 avr. 09 à 23:32, A.M. a écrit : When will postgresql offer global temporary tables with data which are shared among sessions? Such tables

Re: [HACKERS] idea: global temp tables

2009-04-27 Thread Pavel Stehule
2009/4/27 Andrew Dunstan and...@dunslane.net: Kevin Grittner wrote: Pavel Stehule pavel.steh...@gmail.com wrote: I am thinking about global temp tables.  These would have some value to us.  In case anyone doesn't know, this is a feature in the SQL standard. You have a permanent

Re: [HACKERS] idea: global temp tables

2009-04-27 Thread Pavel Stehule
2009/4/28 Kevin Grittner kevin.gritt...@wicourts.gov: A.M. age...@themactionfaction.com wrote: On Apr 27, 2009, at 5:39 PM, Dimitri Fontaine wrote: Le 27 avr. 09 à 23:32, A.M. a écrit : When will postgresql offer global temporary tables with data which are shared among sessions? Such tables