Re: [HACKERS] scheduler in core

2010-02-21 Thread Dimitri Fontaine
Joshua D. Drake j...@commandprompt.com writes: On Sat, 2010-02-20 at 18:19 -0500, Tom Lane wrote: Dimitri Fontaine dfonta...@hi-media.com writes: What would it take to have it included in core, I don't think this really makes sense. There's basically no argument for having it in core

Re: [HACKERS] scheduler in core

2010-02-21 Thread Dimitri Fontaine
Greg Stark gsst...@mit.edu writes: It'll always be another (set of) processes even if it's in core. All it means to be in core is that it will be harder to make modifications and you'll be tied to the Postgres release cycle. Another set of processes all right, but that postmaster is

Re: [HACKERS] libpq PGresult object and encoding

2010-02-21 Thread Jeroen Vermeulen
Jeff Davis wrote: libpq has a PQclientEncoding() function that takes a connection object. However, the client encoding is, in some cases, a property of the result object. For instance, if your client_encoding changes, but you keep the result object around, you have no way to determine later

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-21 Thread Jeroen Vermeulen
Greg Stark wrote: So in principle I agree with this idea. I think a conservative value for the constant would be more like 100x though. If I told you we had an easy way to speed all your queries up by 10% by caching queries but were just choosing not to then I think you would be unhappy.

Re: [HACKERS] parallelizing subplan execution

2010-02-21 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: Probably. For one thing, you can't use fork(), because it won't work on Windows. [...] query. IOW, we're going to need, well, a connection pool in core. *ducks, runs for cover* Well, in fact, you're slowly getting to the interesting^W crazy part of

[HACKERS] Plans for 9.1, Grouping Sets, disabling multiqueries, contrib module for string, plpgpsm, preload dictionaries

2010-02-21 Thread Pavel Stehule
Hello, * Now I am working on migration of plpgpsm to plpgsql 9.0 base. I hope so I understand SQL/PSM well so I am able to write production quality implementation. If you like, I can integrate it to core. It can share about 40-50% code with plpgpsm. The behave of plpgpsm is same as plpgsql -

[HACKERS] getting to beta

2010-02-21 Thread Robert Haas
Now that PostgreSQL 9.0 alpha4 is bundled (though apparently not quite out the door yet), it seems like a good time to think about what we'll need to do to get to beta. Any thoughts? http://wiki.postgresql.org/wiki/PostgreSQL_8.5_Open_Items currently lists no open items, and the Hot Standby TODO

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-21 Thread Robert Haas
On Wed, Feb 17, 2010 at 5:52 PM, Jeroen Vermeulen j...@xs4all.nl wrote: I may have cut this out of my original email for brevity... my impression is that the planner's estimate is likely to err on the side of scalability, not best-case response time; and that this is more likely to happen than

Re: [HACKERS] parallelizing subplan execution (was: explain and PARAM_EXEC)

2010-02-21 Thread Greg Stark
On Sun, Feb 21, 2010 at 3:25 AM, Robert Haas robertmh...@gmail.com wrote: What kinds of things would be sensible to hand off in this way?  Well, you'd want to find nodes that are not likely to be repeatedly re-executed with different parameters, like subplans or inner-indexscans, because

Re: [HACKERS] scheduler in core

2010-02-21 Thread Andrew Dunstan
Pavel Stehule wrote: This reasoning just doesn't fly in the PostgreSQL world. PostgreSQL is designed to be extensible, not a monolithic product. We're not going to change that because some companies have insane corporate policies. The answer, as Jefferson said in another context, is to inform

Re: [HACKERS] Plans for 9.1, Grouping Sets, disabling multiqueries, contrib module for string, plpgpsm, preload dictionaries

2010-02-21 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes: * Last two months I spent some time with preparing workshops about SQL injection. PostgreSQL has only one issue related to this topic. It allows multi queries. With this feature any successful injection can have much more destructive impact. Now we

Re: [HACKERS] getting to beta

2010-02-21 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: http://wiki.postgresql.org/wiki/PostgreSQL_8.5_Open_Items currently lists no open items, um ... are we looking at the same page? I see 8 open items there, not counting the links to the separate HS and SR pages. My suspicion is that the real situation

Re: [HACKERS] scheduler in core

2010-02-21 Thread Bruce Momjian
Pavel Stehule wrote: 2010/2/21 Andrew Dunstan and...@dunslane.net: ? ?I believe that in core may be installed by default in case of ? ?the pgAgent or similar solution... ? ?Many big companies does not allow the developers to configure and ? ?install components we need to request

Re: [HACKERS] getting to beta

2010-02-21 Thread Bruce Momjian
Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: http://wiki.postgresql.org/wiki/PostgreSQL_8.5_Open_Items currently lists no open items, um ... are we looking at the same page? I see 8 open items there, not counting the links to the separate HS and SR pages. My suspicion is

Re: [HACKERS] scheduler in core

2010-02-21 Thread Lucas
2010/2/20 Andrew Dunstan and...@dunslane.net We're not going to change that because some companies have insane corporate policies. I agree, Andrew... This is an outside benefit... not a reason or justification... I believe that a general purpose scheduler is similar to the autovacuum... it

Re: [HACKERS] scheduler in core

2010-02-21 Thread Ron Mayer
Lucas wrote: I believe that in core may be installed by default in case of Those seem like totally orthogonal concepts to me. A feature may be in core but not installed by default (like many PLs). A feature might not be in core but installed by many installers (say postgis). It seems like

Re: [HACKERS] PGXS: REGRESS_OPTS=--load-language=plpgsql

2010-02-21 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Feb 20, 2010, at 10:56 PM, Tom Lane t...@sss.pgh.pa.us wrote: There is a very clear set of behaviors that CORL ought to have given the precedents of our other COR commands. If we don't make it do things that way then we are going to surprise users,

Re: [HACKERS] scheduler in core

2010-02-21 Thread Tom Lane
Ron Mayer rm...@cheapcomplexdevices.com writes: Is the real need here for a convenient way to enable and/or recommend packagers to install some non-core modules by default? It would certainly help us resist assorted requests to put everything including the kitchen sink into core.

[HACKERS] some issue in plpgsq - exec code?

2010-02-21 Thread Pavel Stehule
Hello I am looking on code in pl_exec.c file. I see one issue: /* -- * exec_run_select Execute a select query * -- */ static int exec_run_select(PLpgSQL_execstate *estate, PLpgSQL_expr *expr, long maxtuples, Portal

Re: [HACKERS] getting to beta

2010-02-21 Thread Robert Haas
On Sun, Feb 21, 2010 at 9:26 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: http://wiki.postgresql.org/wiki/PostgreSQL_8.5_Open_Items currently lists no open items, um ... are we looking at the same page?  I see 8 open items there, not counting the links to

Re: [HACKERS] scheduler in core

2010-02-21 Thread Robert Haas
On Sun, Feb 21, 2010 at 12:33 PM, Tom Lane t...@sss.pgh.pa.us wrote: Ron Mayer rm...@cheapcomplexdevices.com writes: Is the real need here for a convenient way to enable and/or recommend packagers to install some non-core modules by default? It would certainly help us resist assorted requests

[HACKERS] WAL-support for Pluggable Indexes

2010-02-21 Thread Simon Riggs
We've just rejected Knn-gist indexes as not enough time for 9.0, which is a considerable disappointment for many people. We already have a pluggable index API, but not one that supports recoverability. It is a simple patch to add recoverability to the index API, if we have the will to do so.

Re: [HACKERS] getting to beta

2010-02-21 Thread Stefan Kaltenbrunner
Robert Haas wrote: On Sun, Feb 21, 2010 at 9:26 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: http://wiki.postgresql.org/wiki/PostgreSQL_8.5_Open_Items currently lists no open items, um ... are we looking at the same page? I see 8 open items there, not

Re: [HACKERS] scheduler in core

2010-02-21 Thread Robert Haas
On Sun, Feb 21, 2010 at 10:17 AM, Lucas luca...@gmail.com wrote: I wonder if the scheduler already existed before the  implementation of the autovacuum, its implementation would  not be a function executed by the in-core scheduler? The real genius of autovacuum is that it works out when there

Re: [HACKERS] scheduler in core

2010-02-21 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Sun, Feb 21, 2010 at 10:17 AM, Lucas luca...@gmail.com wrote: I wonder if the scheduler already existed before the  implementation of the autovacuum, its implementation would  not be a function executed by the in-core scheduler? The real genius of

Re: [HACKERS] PGXS: REGRESS_OPTS=--load-language=plpgsql

2010-02-21 Thread Robert Haas
On Sun, Feb 21, 2010 at 12:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Feb 20, 2010, at 10:56 PM, Tom Lane t...@sss.pgh.pa.us wrote: There is a very clear set of behaviors that CORL ought to have given the precedents of our other COR commands.  If we

Re: [HACKERS] scheduler in core

2010-02-21 Thread Simon Riggs
On Sat, 2010-02-20 at 18:19 -0500, Tom Lane wrote: Dimitri Fontaine dfonta...@hi-media.com writes: Dave Page dp...@pgadmin.org writes: Why not just use pgAgent? It's far more flexible than the design you've suggested, and already exists. What would it take to have it included in core,

Re: [HACKERS] scheduler in core

2010-02-21 Thread Robert Haas
On Sun, Feb 21, 2010 at 1:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Sun, Feb 21, 2010 at 10:17 AM, Lucas luca...@gmail.com wrote: I wonder if the scheduler already existed before the  implementation of the autovacuum, its implementation would  not

Re: [HACKERS] scheduler in core

2010-02-21 Thread Robert Haas
On Sun, Feb 21, 2010 at 1:11 PM, Simon Riggs si...@2ndquadrant.com wrote: On Sat, 2010-02-20 at 18:19 -0500, Tom Lane wrote: Dimitri Fontaine dfonta...@hi-media.com writes: Dave Page dp...@pgadmin.org writes: Why not just use pgAgent? It's far more flexible than the design you've

Re: [HACKERS] WAL-support for Pluggable Indexes

2010-02-21 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: We already have a pluggable index API, but not one that supports recoverability. It is a simple patch to add recoverability to the index API, if we have the will to do so. I suggest you go re-read the archives before asserting this is a simple

Re: [HACKERS] WAL-support for Pluggable Indexes

2010-02-21 Thread Robert Haas
On Sun, Feb 21, 2010 at 12:54 PM, Simon Riggs si...@2ndquadrant.com wrote: We've just rejected Knn-gist indexes as not enough time for 9.0, which is a considerable disappointment for many people. We already have a pluggable index API, but not one that supports recoverability. It is a simple

Re: [HACKERS] PGXS: REGRESS_OPTS=--load-language=plpgsql

2010-02-21 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Well, I'm a big fan of CREATE OR REPLACE anything so I like the patch regardless of whether it solves the current problem, but having said that, I'm not clear on whether it does in fact solve the current problem. When PL/pgsql is installed by default,

Re: [HACKERS] PGXS: REGRESS_OPTS=--load-language=plpgsql

2010-02-21 Thread Robert Haas
On Sun, Feb 21, 2010 at 1:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Well, I'm a big fan of CREATE OR REPLACE anything so I like the patch regardless of whether it solves the current problem, but having said that, I'm not clear on whether it does in

[HACKERS] A thought on Index Organized Tables

2010-02-21 Thread Gokulakannan Somasundaram
Hi, As you all know, Index Organized tables are a way by which we can automatically cluster the data based on the primary key. While i was thinking about an implementation for postgres, it looks like an impossible with the current ideologies. In an IOT, if a record gets updated, we need to

Re: [HACKERS] PGXS: REGRESS_OPTS=--load-language=plpgsql

2010-02-21 Thread Bruce Momjian
Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Feb 20, 2010, at 10:56 PM, Tom Lane t...@sss.pgh.pa.us wrote: There is a very clear set of behaviors that CORL ought to have given the precedents of our other COR commands. If we don't make it do things that way then we are

Re: [HACKERS] scheduler in core

2010-02-21 Thread Dimitri Fontaine
Simon Riggs si...@2ndquadrant.com writes: There is currently no way to run a separate daemon process that runs user code as part of Postgres, so that the startup code gets run immediately we startup, re-run if we crash and shut down cleanly when the server does. If there were some way to run

Re: [HACKERS] PGXS: REGRESS_OPTS=--load-language=plpgsql

2010-02-21 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Sun, Feb 21, 2010 at 1:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: It will be owned by the bootstrap superuser, so the case is exactly the one that a non-superuser DBA would be faced with. Or even a superuser other than the bootstrap superuser, no?

Re: [HACKERS] PGXS: REGRESS_OPTS=--load-language=plpgsql

2010-02-21 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: Tom Lane wrote: Attached is a draft patch (no doc changes) that implements CREATE OR REPLACE LANGUAGE How is pg_migrator affected by this? It always loads the the dump as the super-user. How will the pg_dump use CREATE OR REPLACE LANGUAGE? pg_dump

Re: [HACKERS] PGXS: REGRESS_OPTS=--load-language=plpgsql

2010-02-21 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Tom Lane wrote: Attached is a draft patch (no doc changes) that implements CREATE OR REPLACE LANGUAGE How is pg_migrator affected by this? It always loads the the dump as the super-user. How will the pg_dump use CREATE OR

Re: [HACKERS] PGXS: REGRESS_OPTS=--load-language=plpgsql

2010-02-21 Thread David Fetter
On Thu, Feb 18, 2010 at 01:51:08PM -0800, David Fetter wrote: Folks, While hacking on PL/Parrot, I ran across an issue where when trying to load PL/pgsql, it's done unconditionally and fails. How do we fix pg_regress to be a little more subtle about this? For now, and for the archives,

[HACKERS] Time travel on the buildfarm

2010-02-21 Thread Tom Lane
It is currently 22:21:59 EST here. At 21:50 I committed a fix to copydir.c that cleaned up a couple of thinkos by Greg, including a misspelling that had been making all the builds fail for several hours. I went to see if any of the buildfarm had gone green yet, and indeed half a dozen members

Re: [HACKERS] [COMMITTERS] pgsql: Oops, don't forget to rewind the directory before scanning it to

2010-02-21 Thread Tom Lane
Fujii Masao masao.fu...@gmail.com writes: + Free(xldir); s/Free/FreeDir ? Yeah, that too. I think it's all good now, but please test. One thing I was wondering was whether the stat-wrong-file problem could explain the buildfarm failures that we thought were evidence of a portability

Re: [HACKERS] Streaming replication on win32, still broken

2010-02-21 Thread Fujii Masao
On Fri, Feb 19, 2010 at 7:54 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Heikki Linnakangas wrote: Magnus Hagander wrote: Well, it's going to make the process that reads the WAL cause actual physical I/O... That'll take a chunk out of your total available I/O, which is

Re: [HACKERS] A thought on Index Organized Tables

2010-02-21 Thread Heikki Linnakangas
Gokulakannan Somasundaram wrote: Hi, As you all know, Index Organized tables are a way by which we can automatically cluster the data based on the primary key. While i was thinking about an implementation for postgres, it looks like an impossible with the current ideologies. In an IOT, if