[HACKERS] Buffering GiST leaf pages too

2011-08-23 Thread Heikki Linnakangas
While looking at Alexander's GiST fastbuild patch, which adds buffers to internal nodes to avoid random I/O during index build, it occurred to me that inserting the tuples to the leaf pages one at a time is quite inefficient too, even if the leaf pages are in cache. There's still the overhead

Re: [HACKERS] OperationalError: FATAL: lock AccessShareLock on object 0/1260/0 is already

2011-08-23 Thread daveg
On Sun, Aug 14, 2011 at 12:16:39AM -0400, Robert Haas wrote: On Fri, Aug 12, 2011 at 7:19 PM, daveg da...@sonic.net wrote: This seems to be bug month for my client. Now there are seeing periods where all new connections fail immediately with the error:   FATAL:  lock AccessShareLock on

Re: [HACKERS] VIP: plpgsql - early embedded sql plan preparation

2011-08-23 Thread Merlin Moncure
On Tue, Aug 23, 2011 at 6:30 AM, Pavel Stehule pavel.steh...@gmail.com wrote: Hello I am returning back to discus two months ago about usage of plpgsql_lint. I integrated this module to plpgsql's core. This feature is controlled via plpgsql GUC variable prepare_plans and via plpgsql option.

Re: [HACKERS] VIP: plpgsql - early embedded sql plan preparation

2011-08-23 Thread Pavel Stehule
Is it possible to raise warnings at function creation time? when this code will be in core, then it is possible without one significant exceptions - triggers :( I am not able to specify a target table - and real type for NEW and OLD records. Regards Pavel merlin -- Sent via

Re: [HACKERS] PushActiveSnapshot(GetTransactionSnapshot())

2011-08-23 Thread Simon Riggs
On Mon, Aug 22, 2011 at 7:07 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Simon Riggs's message of dom ago 21 16:23:39 -0300 2011: In common cases of snapshot use we run GetSnapshotData() into a statically allocated snapshot, then immediately copy the static struct into a

Re: [HACKERS] PushActiveSnapshot(GetTransactionSnapshot())

2011-08-23 Thread Alvaro Herrera
Excerpts from Simon Riggs's message of mar ago 23 10:56:17 -0300 2011: On Mon, Aug 22, 2011 at 7:07 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Simon Riggs's message of dom ago 21 16:23:39 -0300 2011: In common cases of snapshot use we run GetSnapshotData() into a

[HACKERS] Deferred Snapshots

2011-08-23 Thread Simon Riggs
Back in 2009, I proposed the idea of Lazy Snapshots. The idea was to put off generating a snapshot until we need one. I have further ideas on this now. In general, this was a bad idea because... On Tue, 2009-08-18 at 10:18 -0400, Tom Lane wrote: Simon, this concept is completely broken, as far

Re: [HACKERS] cheaper snapshots redux

2011-08-23 Thread Simon Riggs
On Mon, Aug 22, 2011 at 10:25 PM, Robert Haas robertmh...@gmail.com wrote: I've been giving this quite a bit more thought, and have decided to abandon the scheme described above, at least for now. I liked your goal of O(1) snapshots and think you should go for that. I didn't realise you were

[HACKERS] Getting rid of pg_pltemplate

2011-08-23 Thread Tom Lane
One of my goals for the extensions feature has been that we should be able to get rid of the pg_pltemplate system catalog, moving all the information therein into languages' extension definition files. This would allow third-party procedural languages to be installed as easily as built-in ones.

Re: [HACKERS] Deferred Snapshots

2011-08-23 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: Back in 2009, I proposed the idea of Lazy Snapshots. The idea was to put off generating a snapshot until we need one. It was a broken idea then, and it has not become less so with the passage of time. However, that does not exclude a number of cases

Re: [HACKERS] cheaper snapshots redux

2011-08-23 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: With respect to the first problem, what I'm imagining is that we not do a complete rewrite of the snapshot in shared memory on every commit. Instead, when a transaction ends, we'll decide whether to (a) write a new snapshot or (b) just record the XIDs

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-08-23 Thread Robert Haas
On Mon, Aug 22, 2011 at 3:31 AM, daveg da...@sonic.net wrote: So far I've got:  - affects system tables  - happens very soon after process startup  - in 8.4.7 and 9.0.4  - not likely to be hardware or OS related  - happens in clusters for period of a few second to many minutes I'll work

Re: [HACKERS] cheaper snapshots redux

2011-08-23 Thread Robert Haas
On Tue, Aug 23, 2011 at 12:13 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm a bit concerned that this approach is trying to optimize the heavy contention situation at the cost of actually making things worse anytime that you're not bottlenecked by contention for access to this shared data

Re: [HACKERS] cheaper snapshots redux

2011-08-23 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: I think the real trick is figuring out a design that can improve concurrency. I'm far from familiar with the detailed concepts here, but allow me to comment. I have two open questions: - is it possible to use a distributed algorithm to produce XIDs,

[HACKERS] text search: restricting the number of parsed words in headline generation

2011-08-23 Thread Sushant Sinha
Given a document and a query, the goal of headline generation is to produce text excerpts in which the query appears. Currently the headline generation in postgres follows the following steps: 1. Tokenize the documents and obtain the lexemes 2. Decide on lexemes that should be the part of the

[HACKERS] Why doesn't psql use the information schema to get ACL description ?

2011-08-23 Thread Dave Cramer
Recently a bug was reported on the JDBC list regarding table privileges. It turns out that we have not been parsing the ACL tables correctly. This is partially due to having to track the catalogs through each version. So this breaks down into two questions 1) is it possible to get all of the ACL

Re: [HACKERS] Getting rid of pg_pltemplate

2011-08-23 Thread David E. Wheeler
On Aug 23, 2011, at 8:31 AM, Tom Lane wrote: One of my goals for the extensions feature has been that we should be able to get rid of the pg_pltemplate system catalog, moving all the information therein into languages' extension definition files. This would allow third-party procedural

Re: [HACKERS] cheaper snapshots redux

2011-08-23 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: That's certainly a fair concern, and it might even be worse than O(n^2). On the other hand, the current approach involves scanning the entire ProcArray for every snapshot, even if nothing has changed and 90% of the backends are sitting around playing

[HACKERS] Range Types

2011-08-23 Thread Jeff Davis
Attached is the latest version of the Range Types patch. I will get it into better shape before the commitfest, but wanted to put up a draft in case anyone had comments on the TODO items. Changes: * Uses BTree opclass rather than compare function. * Collation specified at type definition

[HACKERS] skip WAL on COPY patch

2011-08-23 Thread Steve Singer
The attached patch adds an option to the COPY command to skip writing WAL when the following conditions are all met: 1) The table is empty (zero size on disk) 2) The copy command can obtain an access exclusive lock on the table with out blocking. 3) The WAL isn't needed for replication For

Re: [HACKERS] SSI 2PC coverage

2011-08-23 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Committed. I removed the second expected output file, and marked the prepared-transactions tests in the schedule as ignore instead. That way if max_prepared_transactions=0, you get a notice that the test case failed, but

Re: [HACKERS] Getting rid of pg_pltemplate

2011-08-23 Thread Tom Lane
David E. Wheeler da...@kineticode.com writes: On Aug 23, 2011, at 8:31 AM, Tom Lane wrote: Having done that, we'd mark all the standard trusted PLs as dba_create, expand the existing definition scripts for the PL extensions so that they fully specify the languages and their support functions

Re: [HACKERS] Why doesn't psql use the information schema to get ACL description ?

2011-08-23 Thread Peter Eisentraut
On tis, 2011-08-23 at 12:48 -0400, Dave Cramer wrote: Recently a bug was reported on the JDBC list regarding table privileges. It turns out that we have not been parsing the ACL tables correctly. This is partially due to having to track the catalogs through each version. So this breaks down

Re: [HACKERS] text search: restricting the number of parsed words in headline generation

2011-08-23 Thread Tom Lane
Sushant Sinha sushant...@gmail.com writes: Given a document and a query, the goal of headline generation is to produce text excerpts in which the query appears. ... right ... Here is a simple patch that limits the number of words during the tokenization phase and puts an upper-bound on the

Re: [HACKERS] skip WAL on COPY patch

2011-08-23 Thread Tom Lane
Steve Singer ssin...@ca.afilias.info writes: The attached patch adds an option to the COPY command to skip writing WAL when the following conditions are all met: 1) The table is empty (zero size on disk) 2) The copy command can obtain an access exclusive lock on the table with out

Re: [HACKERS] Getting rid of pg_pltemplate

2011-08-23 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: We'll add a new boolean parameter to extension control files, called say dba_create (ideas for better names welcome). If it's missing or set to false, there's no change in behavior. When it's true, then (a) you must be superuser or owner of the current

Re: [HACKERS] text search: restricting the number of parsed words in headline generation

2011-08-23 Thread Alvaro Herrera
Excerpts from Tom Lane's message of mar ago 23 15:59:18 -0300 2011: Sushant Sinha sushant...@gmail.com writes: Given a document and a query, the goal of headline generation is to produce text excerpts in which the query appears. ... right ... Here is a simple patch that limits the

Re: [HACKERS] Getting rid of pg_pltemplate

2011-08-23 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Tom Lane t...@sss.pgh.pa.us writes: We'll add a new boolean parameter to extension control files, called say dba_create (ideas for better names welcome). If it's missing or set to false, there's no change in behavior. When it's true, then

Re: [HACKERS] Getting rid of pg_pltemplate

2011-08-23 Thread Christopher Browne
On Tue, Aug 23, 2011 at 3:09 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Tom Lane t...@sss.pgh.pa.us writes: We'll add a new boolean parameter to extension control files, called say dba_create (ideas for better names welcome).  If it's missing or set to false, there's no change in

[HACKERS] pg_dump --exclude-table-data

2011-08-23 Thread Andrew Dunstan
Attached is an undocumented patch that allows a user to have pg_dump exclude data but not DDL for a table. One use case for this is a very large table that changes infrequently, and for which dumping data frequently would be wasteful and unnecessary. This is especially useful in conjunction

Re: [HACKERS] skip WAL on COPY patch

2011-08-23 Thread Robert Haas
On Tue, Aug 23, 2011 at 3:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: Steve Singer ssin...@ca.afilias.info writes: The attached patch adds an option to the COPY command to skip writing WAL when the following conditions are all met: 1) The table is empty (zero size on disk) 2) The copy command

Re: [HACKERS] Getting rid of pg_pltemplate

2011-08-23 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: That's called sudo on linux. I propose that we stick to such a name. I'm not impressed with that name: it will mean nothing to Windows users, nor for that matter to many non-sysadmin types on Unix. Fair enough. Do we want a more general SUDO facility in

Re: [HACKERS] Getting rid of pg_pltemplate

2011-08-23 Thread Dimitri Fontaine
Christopher Browne cbbro...@gmail.com writes: Actually, this is somewhat more like UNIX setuid (2). When I first started using SECURITY DEFINER functions, I thought of it as being like sudo. But it's really like setuid. I see SECURITY DEFINER functions definitely as setuid for PostgreSQL,

Re: [HACKERS] skip WAL on COPY patch

2011-08-23 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: What I think would be really interesting is a way to make this work when the table *isn't* empty. In other words, have a COPY option that (1) takes an exclusive lock on the table, (2) writes the data being inserted into new pages beyond the old EOF,

Re: [HACKERS] skip WAL on COPY patch

2011-08-23 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mar ago 23 17:08:50 -0300 2011: What I think would be really interesting is a way to make this work when the table *isn't* empty. In other words, have a COPY option that (1) takes an exclusive lock on the table, (2) writes the data being inserted into

[HACKERS] pg_restore --no-post-data and --post-data-only

2011-08-23 Thread Andrew Dunstan
Attached is an undocumented patch that allows pg_restore to omit post-data items or omit all but post-data items. This has been discussed before, and Simon sent in a patch back on 2008, which has bitrotted some. I'm not sure why it was dropped at the time, but I think it's time to do this.

Re: [HACKERS] skip WAL on COPY patch

2011-08-23 Thread Steve Singer
On 11-08-23 04:17 PM, Tom Lane wrote: Robert Haasrobertmh...@gmail.com writes: What I think would be really interesting is a way to make this work when the table *isn't* empty. In other words, have a COPY option that (1) takes an exclusive lock on the table, (2) writes the data being inserted

Re: [HACKERS] skip WAL on COPY patch

2011-08-23 Thread Robert Haas
On Tue, Aug 23, 2011 at 4:17 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: What I think would be really interesting is a way to make this work when the table *isn't* empty.  In other words, have a COPY option that (1) takes an exclusive lock on the table, (2)

Re: [HACKERS] skip WAL on COPY patch

2011-08-23 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mar ago 23 17:43:13 -0300 2011: On Tue, Aug 23, 2011 at 4:17 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: What I think would be really interesting is a way to make this work when the table *isn't* empty.  In other

[HACKERS] Short document fix

2011-08-23 Thread Hitoshi Harada
In the CREATE DOMAIN reference page of the current HEAD, it says --- CREATE DOMAIN us_postal_code AS TEXT CHECK( VALUE ~ '^\\d{5}$' OR VALUE ~ '^\\d{5}-\\d{4}$' ); --- but I believe it should conform the standard string style now that the default is standard_conforming_strings = on. I didn't

Re: [HACKERS] 9.1rc1: TRAP: FailedAssertion(!(item_width 0), File: costsize.c, Line: 3274)

2011-08-23 Thread Tom Lane
Erik Rijkers e...@xs4all.nl writes: The below SQL causes: TRAP: FailedAssertion(!(item_width 0), File: costsize.c, Line: 3274) Thanks, I've applied a patch for this. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] skip WAL on COPY patch

2011-08-23 Thread Jeff Davis
On Tue, 2011-08-23 at 15:05 -0400, Tom Lane wrote: Steve Singer ssin...@ca.afilias.info writes: The attached patch adds an option to the COPY command to skip writing WAL when the following conditions are all met: 1) The table is empty (zero size on disk) 2) The copy command can obtain

[HACKERS] Another extensions bug

2011-08-23 Thread Tom Lane
Whilst testing the schema-creation-permissions scenario that Kaigai-san recently pointed out, I happened to do this: regression=# create schema c; CREATE SCHEMA regression=# create extension cube with schema c; CREATE EXTENSION regression=# drop schema c; DROP SCHEMA ... er, what? I was

[HACKERS] REGRESS_OPTS default

2011-08-23 Thread Tom Lane
I got burnt tonight by the fact that the earthdistance contrib module runs its regression test in database regression, not contrib_regression as I was expecting and as every other contrib module does. The reason turns out to be that earthdistance/Makefile does this: REGRESS_OPTS =

Re: [HACKERS] text search: restricting the number of parsed words in headline generation

2011-08-23 Thread Sushant Sinha
Here is a simple patch that limits the number of words during the tokenization phase and puts an upper-bound on the headline generation. Doesn't this force the headline to be taken from the first N words of the document, independent of where the match was? That seems rather unworkable,

Re: [HACKERS] Question: CREATE EXTENSION and create schema permission?

2011-08-23 Thread Tom Lane
Kohei KaiGai kai...@kaigai.gr.jp writes: The attached patch adds permission check at the scenario that I explained bellow. Instead of using this patch, I changed the code to call CreateSchemaCommand itself. The test that was still missing was the one to restrict the schema name to not start

Re: [HACKERS] text search: restricting the number of parsed words in headline generation

2011-08-23 Thread Tom Lane
Sushant Sinha sushant...@gmail.com writes: Doesn't this force the headline to be taken from the first N words of the document, independent of where the match was? That seems rather unworkable, or at least unhelpful. In headline generation function, we don't have any index or knowledge of

Re: [HACKERS] text search: restricting the number of parsed words in headline generation

2011-08-23 Thread Sushant Sinha
Actually, this code seems probably flat-out wrong: won't every successful call of hlCover() on a given document return exactly the same q value (end position), namely the last token occurrence in the document? How is that helpful? regards, tom lane There is a line

Re: [HACKERS] Another extensions bug

2011-08-23 Thread Tom Lane
I wrote: ... So I'm thinking this recursive call should just pass DEPFLAG_NORMAL in all cases: On further reflection, it seems more in keeping with the coding elsewhere in this module to treat this as a distinct dependency type, instead of confusing it with a NORMAL dependency. There's no