Re: [HACKERS] contrib/citext versus collations

2011-06-07 Thread David E. Wheeler
On Jun 6, 2011, at 4:35 PM, Tom Lane wrote: That sounds like a good idea. BTW, it struck me shortly after sending this that we'd already discussed the idea of a flag in pg_proc showing whether a function pays attention to collation. We could of course use that for this purpose. Seems like

Re: [HACKERS] gdb with postgres

2011-06-07 Thread HuangQi
On 6 June 2011 21:57, Kevin Grittner kevin.gritt...@wicourts.gov wrote: HuangQi huangq...@gmail.com wrote: (gdb) b qp_add_paths_to_joinrel Breakpoint 1 at 0x1a6744: file joinpath.c, line 67. (gdb) attach 23903 If I enter c, gdb will directly finish executing this process and current

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Dave Page
On Tue, Jun 7, 2011 at 12:29 AM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: On Mon, Jun 6, 2011 at 8:44 PM, Stephen Frost sfr...@snowman.net wrote: If we're going to start putting in changes like this, I'd suggest that we try and target something like September for

Re: [HACKERS] WALInsertLock tuning

2011-06-07 Thread Simon Riggs
On Mon, Jun 6, 2011 at 11:25 PM, Robert Haas robertmh...@gmail.com wrote: As to the question of whether it's safe, I think I'd agree that the chances of this backfiring are pretty remote.  I think that with the zeroing they are exactly zero, because (now that we start XLOG positions at 0/1)

Re: [HACKERS] WALInsertLock tuning

2011-06-07 Thread Heikki Linnakangas
On 07.06.2011 10:21, Simon Riggs wrote: On Mon, Jun 6, 2011 at 11:25 PM, Robert Haasrobertmh...@gmail.com wrote: It strikes me, though, that we could probably get nearly all of the benefit of this patch by being willing to zero the first sizeof(XLogRecord) bytes following a record, but not the

[HACKERS] Invalid byte sequence for encoding UTF8, caused due to non wide-char-aware downcase_truncate_identifier() function on WINDOWS

2011-06-07 Thread Jeevan Chalke
Hi Tom, Issue is on Windows: If you see in attached failure.out file, (after running failure.sql) we are getting ERROR: invalid byte sequence for encoding UTF8: 0xe59aff error. Please note that byte sequence we got from database is e5 9a ff, where as actual byte sequence for the wide character

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-07 Thread Heikki Linnakangas
On 06.06.2011 05:13, Kevin Grittner wrote: Kevin Grittner wrote: Maybe I should submit a patch without added complexity of the scheduled cleanup and we can discuss that as a separate patch? Here's a patch which adds the missing support for DDL. It makes me a bit uncomfortable to do catalog

Re: [HACKERS] WALInsertLock tuning

2011-06-07 Thread Simon Riggs
On Tue, Jun 7, 2011 at 8:27 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 07.06.2011 10:21, Simon Riggs wrote: On Mon, Jun 6, 2011 at 11:25 PM, Robert Haasrobertmh...@gmail.com  wrote: It strikes me, though, that we could probably get nearly all of the benefit of this

Re: [HACKERS] WALInsertLock tuning

2011-06-07 Thread Robert Haas
On Tue, Jun 7, 2011 at 3:21 AM, Simon Riggs si...@2ndquadrant.com wrote: It strikes me, though, that we could probably get nearly all of the benefit of this patch by being willing to zero the first sizeof(XLogRecord) bytes following a record, but not the rest of the buffer.  That would pretty

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-07 Thread Kevin Grittner
Heikki Linnakangas wrote: It makes me a bit uncomfortable to do catalog cache lookups while holding all the lwlocks. We've also already removed the reserved entry for scratch space while we do that - if a cache lookup errors out, we'll leave behind quite a mess. I guess it shouldn't fail,

Re: [HACKERS] WALInsertLock tuning

2011-06-07 Thread Simon Riggs
On Tue, Jun 7, 2011 at 1:24 PM, Robert Haas robertmh...@gmail.com wrote: One other thought is that I think that this patch might cause a user-visible behavior change.  Right now, when you hit the end of recovery, you most typically get a message saying - record with zero length.  Not always,

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-07 Thread Kevin Grittner
Kevin Grittner wrote: Heikki Linnakangas wrote: I think the logic is actually wrong at the moment: When you reindex a single index, DropAllPredicateLocksFromTableImpl() will transfer all locks belonging to any index on the same table, and any finer-granularity locks on the heap. It would

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Stephen Frost
* Alvaro Herrera (alvhe...@commandprompt.com) wrote: I note that if 2nd Quadrant is interested in having a game-changing platform without having to wait a full year for 9.2, they can obviously distribute a modified version of Postgres that integrates Robert's patch. Having thought about this,

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-07 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: It makes me a bit uncomfortable to do catalog cache lookups while holding all the lwlocks. We've also already removed the reserved entry for scratch space while we do that - if a cache lookup errors out, we'll leave behind quite

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-07 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: If you don't believe that a catcache lookup will ever fail, I will contract to break the patch. As you probably know by now by reaching the end of the thread, this code is going away based on Heikki's arguments; but for my understanding, so that I don't

Re: [HACKERS] Range Types and extensions

2011-06-07 Thread Merlin Moncure
On Mon, Jun 6, 2011 at 6:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: I vote for at minimum the type itself and ANYRANGE to be in core. From there you could make it like arrays where the range type is automatically generated for each POD type.  I would

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-07 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: Tom Lane t...@sss.pgh.pa.us wrote: If you don't believe that a catcache lookup will ever fail, I will contract to break the patch. As you probably know by now by reaching the end of the thread, this code is going away based on Heikki's

Re: [HACKERS] Range Types and extensions

2011-06-07 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: On Mon, Jun 6, 2011 at 6:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: By my count there are only about 20 datatypes in core for which it looks sensible to provide a range type (ie, it's a non-deprecated, non-composite type with a standard default btree

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-07 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: What am I missing? Out-of-memory. Query cancel. The attempted catalog access failing because it results in a detected deadlock. I could probably think of several more if I spent ten minutes on it; and

Re: [HACKERS] WALInsertLock tuning

2011-06-07 Thread Heikki Linnakangas
On 07.06.2011 10:55, Simon Riggs wrote: On Tue, Jun 7, 2011 at 8:27 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: You would only need to do it just before you write out the WAL. I guess you'd need to grab WALInsertLock in XLogWrite() to prevent more WAL records from being

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Joshua D. Drake
On 06/06/2011 04:43 PM, Robert Haas wrote: On Mon, Jun 6, 2011 at 6:53 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Robert Haas's message of vie jun 03 09:17:08 -0400 2011: I've now spent enough time working on this issue now to be convinced that the approach has merit,

Re: [HACKERS] WALInsertLock tuning

2011-06-07 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 07.06.2011 10:55, Simon Riggs wrote: How would that help? It doesn't matter whether the pages are zeroed while they sit in memory. And if you write a full page of WAL data, any wasted bytes at the end of the page don't

Re: [HACKERS] WALInsertLock tuning

2011-06-07 Thread Simon Riggs
On Tue, Jun 7, 2011 at 4:57 PM, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 07.06.2011 10:55, Simon Riggs wrote: How would that help? It doesn't matter whether the pages are zeroed while they sit in memory. And if you write a full page

Re: [HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

2011-06-07 Thread Greg Stark
On Jun 3, 2011 8:38 PM, Bruce Momjian br...@momjian.us wrote: I realize we just read the pages from the kernel to maintain sequential I/O, but do we actually read the contents of the page if we know it doesn't need vacuuming? If so, do we need to? I dont follow. What's your question? Tom's

Re: [HACKERS] [Pgbuildfarm-members] CREATE FUNCTION hang on test machine polecat on HEAD

2011-06-07 Thread Robert Creager
On Jun 6, 2011, at 7:29 PM, Andrew Dunstan and...@dunslane.net wrote: On 06/06/2011 07:30 PM, Robert Creager wrote: [4de65a8f.607a:1] LOG: connection received: host=[local] [4de65a8f.607a:2] LOG: connection authorized: user=Robert database=pl_regression [4de65a8f.607a:3] LOG:

[HACKERS] CREATE FUNCTION hang on test machine polecat on HEAD

2011-06-07 Thread Robert Creager
This is the second time I've had this happen in the last week or so. I have a 'regular' postgresql server running, and then test test setup (both llvm ang gcc). It's chewing up 1 core on my MBP, never completing.502 229 1 0 0:00.50 ?? 0:00.60 /Library/PostgreSQL/8.3/bin/postgres -D

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-07 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Tom Lane's message of lun jun 06 12:49:46 -0400 2011: Hmm, there's already a mechanism for closing temp FDs at the end of a query ... maybe blind writes could use temp-like FDs? I don't think it can be made to work exactly like

Re: [HACKERS] Re: [Pgbuildfarm-members] CREATE FUNCTION hang on test machine polecat on HEAD

2011-06-07 Thread Andrew Dunstan
On 06/07/2011 12:35 AM, Tom Lane wrote: Andrew Dunstanand...@dunslane.net writes: On 06/06/2011 07:30 PM, Robert Creager wrote: [4de65a8f.607a:3] LOG: statement: CREATE OR REPLACE FUNCTION bar() RETURNS integer AS $$ #die 'BANG!'; # causes server process to exit(2) # alternative - causes

Re: [HACKERS] Re: [Pgbuildfarm-members] CREATE FUNCTION hang on test machine polecat on HEAD

2011-06-07 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 06/07/2011 12:35 AM, Tom Lane wrote: You sure it's hung on that statement, and not the following one? The following one would be trying to load plperlu into a backend already using plperl, which is an area that it wouldn't exactly be surprising to

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Simon Riggs
On Mon, Jun 6, 2011 at 8:50 PM, Dave Page dp...@pgadmin.org wrote: On Mon, Jun 6, 2011 at 8:40 PM, Stefan Kaltenbrunner ste...@kaltenbrunner.cc wrote: On 06/06/2011 09:24 PM, Dave Page wrote: On Mon, Jun 6, 2011 at 8:12 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: So, to the question

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Robert Haas
On Tue, Jun 7, 2011 at 12:51 PM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, Jun 6, 2011 at 8:50 PM, Dave Page dp...@pgadmin.org wrote: On Mon, Jun 6, 2011 at 8:40 PM, Stefan Kaltenbrunner ste...@kaltenbrunner.cc wrote: On 06/06/2011 09:24 PM, Dave Page wrote: On Mon, Jun 6, 2011 at 8:12

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Robert Haas
On Tue, Jun 7, 2011 at 11:56 AM, Joshua D. Drake j...@commandprompt.com wrote: On 06/06/2011 04:43 PM, Robert Haas wrote: On Mon, Jun 6, 2011 at 6:53 PM, Alvaro Herrera alvhe...@commandprompt.com  wrote: Excerpts from Robert Haas's message of vie jun 03 09:17:08 -0400 2011: I've now spent

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-07 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: We've also already removed the reserved entry for scratch space This and Tom's concerns have me wondering if we should bracket the two sections of code where we use the reserved lock target entry with HOLD_INTERRUPTS() and

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-07 Thread Heikki Linnakangas
On 07.06.2011 20:03, Kevin Grittner wrote: Heikki Linnakangasheikki.linnakan...@enterprisedb.com wrote: We've also already removed the reserved entry for scratch space This and Tom's concerns have me wondering if we should bracket the two sections of code where we use the reserved lock

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: Please consider this as a serious proposal for tuning in 9.1. Look: it is at least four months too late for anything of the sort in 9.1. We should be fixing bugs, and nothing else, if we ever want to get 9.1 out the door. Performance improvements don't

Re: [HACKERS] [Pgbuildfarm-members] CREATE FUNCTION hang on test machine polecat on HEAD

2011-06-07 Thread Alex Hunsaker
On Mon, Jun 6, 2011 at 21:16, Robert Creager robert.crea...@oracle.com wrote: That's weird. Why it should hang there I have no idea. Did it hang at the same spot both times? Can you get a backtrace? I think so, but I didn't pay much attention :-( GNU gdb 6.3.50-20050815 (Apple version

Re: [HACKERS] Range Types and extensions

2011-06-07 Thread Jeff Davis
On Tue, 2011-06-07 at 11:15 -0400, Tom Lane wrote: Merlin Moncure mmonc...@gmail.com writes: right. hm -- can you have multiple range type definitions for a particular type? In principle, sure, if the type has multiple useful sort orderings. Right. Additionally, you might want to use

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Joshua Berkus
iew. The reason we usually skip the summer isn't actually a wholesale lack of people - it's because it's not so good from a publicity perspective, and it's hard to get all the packagers around at the same time. Actually, the summer is *excellent* from a publicity perspective ... at least,

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: ... I think at the next developer meeting we're going to get to hear Tom argue that overlapping the end of beta with the beginning of the next release cycle is a mistake and we should go back to the old system where we yell at everyone to shut up

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Joshua Berkus
Robert, Oh, I get that. I'm just dismayed that we can't have a discussion about the patch without getting sidetracked into a conversation about whether we should throw feature freeze out the window. That's not something you can change. Whatever the patch is, even if it's a psql

Re: [HACKERS] Range Types and extensions

2011-06-07 Thread Jeff Davis
On Mon, 2011-06-06 at 14:42 -0700, Darren Duncan wrote: On this note, here's a *big* thing that needs discussion ... [ refering to the concept of discrete versus continuous ranges ] Yes, there has been much discussion on this topic already. The solution right now is that they both behave like

9.1 release scheduling (was Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch)

2011-06-07 Thread Tom Lane
Joshua Berkus j...@agliodbs.com writes: Actually, the summer is *excellent* from a publicity perspective ... at least, June and July are. Both of those months are full of US conferences whose PR we can piggyback on to make a splash. August is really the only bad month from a PR

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Robert Haas
On Tue, Jun 7, 2011 at 1:27 PM, Joshua Berkus j...@agliodbs.com wrote: As long as we have solidarity of the committers that this is not allowed, however, this is not a real problem.  And it appears that we do.  In the future, it shouldn't even be necessary to discuss it. Solidarity? Simon -

Re: [HACKERS] [Pgbuildfarm-members] CREATE FUNCTION hang on test machine polecat on HEAD

2011-06-07 Thread Andrew Dunstan
On 06/07/2011 01:18 PM, Alex Hunsaker wrote: I don't suppose /dev/urandom blocks on OS X? Granted, I may have missed something in translation with the macro fest that is perl... I wondered if we were possibly exhausting some entropy pool. It seems like this would be just such a bad bug

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-07 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: It makes me a bit uncomfortable to do catalog cache lookups while holding all the lwlocks. I think I've caught up with the rest of the class on why this isn't sane in DropAllPredicateLocksFromTableImpl, but I wonder about

Re: 9.1 release scheduling (was Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch)

2011-06-07 Thread Thom Brown
On 7 June 2011 19:32, Tom Lane t...@sss.pgh.pa.us wrote: Joshua Berkus j...@agliodbs.com writes: Actually, the summer is *excellent* from a publicity perspective ... at least, June and July are.  Both of those months are full of US conferences whose PR we can piggyback on to make a splash.

Re: [HACKERS] [Pgbuildfarm-members] CREATE FUNCTION hang on test machine polecat on HEAD

2011-06-07 Thread Andres Freund
On Tuesday, June 07, 2011 19:40:21 Andrew Dunstan wrote: On 06/07/2011 01:18 PM, Alex Hunsaker wrote: I don't suppose /dev/urandom blocks on OS X? Granted, I may have missed something in translation with the macro fest that is perl... I wondered if we were possibly exhausting some entropy

Re: [HACKERS] [Pgbuildfarm-members] CREATE FUNCTION hang on test machine polecat on HEAD

2011-06-07 Thread Tom Lane
Alex Hunsaker bada...@gmail.com writes: On Mon, Jun 6, 2011 at 21:16, Robert Creager robert.crea...@oracle.com wrote: (gdb) bt #0  0x000100a505e4 in Perl_get_hash_seed () #1  0x000100a69b94 in perl_parse () I don't suppose /dev/urandom blocks on OS X? The man page for it avers

Re: [HACKERS] [Pgbuildfarm-members] CREATE FUNCTION hang on test machine polecat on HEAD

2011-06-07 Thread Christopher Browne
On Tue, Jun 7, 2011 at 5:40 PM, Andrew Dunstan and...@dunslane.net wrote: On 06/07/2011 01:18 PM, Alex Hunsaker wrote: I don't suppose /dev/urandom blocks on OS X?  Granted, I may have missed something in translation with the macro fest that is perl... I wondered if we were possibly

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Robert Haas
On Tue, Jun 7, 2011 at 1:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: ... I think at the next developer meeting we're going to get to hear Tom argue that overlapping the end of beta with the beginning of the next release cycle is a mistake and we should

Re: 9.1 release scheduling (was Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch)

2011-06-07 Thread Robert Haas
On Tue, Jun 7, 2011 at 1:45 PM, Thom Brown t...@linux.com wrote: Speaking of which, is it now safe to remove the NOT VALID constraints don't dump properly issue from the blocker list since the fix has been committed? I hope so, because I just did that (before noticing this email from you). --

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jun 7, 2011 at 1:27 PM, Joshua Berkus j...@agliodbs.com wrote: As long as we have solidarity of the committers that this is not allowed, however, this is not a real problem.  And it appears that we do.  In the future, it shouldn't even be

Re: [HACKERS] Range Types and extensions

2011-06-07 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: On Mon, 2011-06-06 at 14:42 -0700, Darren Duncan wrote: Can Pg be changed to support . in operator names as long as they don't just appear by themselves? What would this break to do so? Someone else would have to comment on that. DOT_DOT is already a

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-07 Thread Heikki Linnakangas
On 07.06.2011 20:42, Kevin Grittner wrote: Heikki Linnakangasheikki.linnakan...@enterprisedb.com wrote: It makes me a bit uncomfortable to do catalog cache lookups while holding all the lwlocks. I think I've caught up with the rest of the class on why this isn't sane in

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Simon Riggs
On Tue, Jun 7, 2011 at 6:33 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Jun 7, 2011 at 1:27 PM, Joshua Berkus j...@agliodbs.com wrote: As long as we have solidarity of the committers that this is not allowed, however, this is not a real problem.  And it appears that we do.  In the

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-07 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: I think I've caught up with the rest of the class on why this isn't sane in DropAllPredicateLocksFromTableImpl, but I wonder about CheckTableForSerializableConflictIn. We *do* expect to be throwing errors in here, and we need some way to tell

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-07 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Predicate locks on indexes are only needed to lock key ranges, to notice later insertions into the range, right? For locks on tuples that do exist, we have locks on the heap. If we're just about to delete every tuple in the heap,

Re: [HACKERS] [Pgbuildfarm-members] CREATE FUNCTION hang on test machine polecat on HEAD

2011-06-07 Thread Alex Hunsaker
On Tue, Jun 7, 2011 at 11:48, Tom Lane t...@sss.pgh.pa.us wrote: Alex Hunsaker bada...@gmail.com writes: On Mon, Jun 6, 2011 at 21:16, Robert Creager robert.crea...@oracle.com wrote: (gdb) bt #0  0x000100a505e4 in Perl_get_hash_seed () #1  0x000100a69b94 in perl_parse () I don't

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-07 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Just to answer the question (independently of Heikki's concern about whether this is needed at all): it depends on the information you have. If all you have is the index OID, then yeah a catcache lookup in pg_index is probably the best thing. If you have

Re: [HACKERS] [Pgbuildfarm-members] CREATE FUNCTION hang on test machine polecat on HEAD

2011-06-07 Thread Tom Lane
Alex Hunsaker bada...@gmail.com writes: Im looking at the raw perl 5.10.0 source... I wonder if apple is shipping a modified version? You could find out by digging around at http://www.opensource.apple.com/ polecat appears to be running OSX 10.6.7, so this is what you want:

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Stephen Frost
* Simon Riggs (si...@2ndquadrant.com) wrote: Before you arrived, it was quite normal to suggest tuning patches after feature freeze. I haven't been around as long as some, but I think I've been around longer than Robert, and I can say that I don't recall serious performance patches,

Re: [HACKERS] Range Types and extensions

2011-06-07 Thread Darren Duncan
Jeff Davis wrote: On Tue, 2011-06-07 at 11:15 -0400, Tom Lane wrote: Merlin Moncure mmonc...@gmail.com writes: right. hm -- can you have multiple range type definitions for a particular type? In principle, sure, if the type has multiple useful sort orderings. Right. Additionally, you might

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-07 Thread Heikki Linnakangas
On 07.06.2011 21:10, Kevin Grittner wrote: I think that leaves me with all the answers I need to get a new patch out this evening (U.S. Central Time). Great, I'll review it in my morning (in about 12h) -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via

Re: [HACKERS] Range Types and extensions

2011-06-07 Thread Darren Duncan
Jeff Davis wrote: On Mon, 2011-06-06 at 14:42 -0700, Darren Duncan wrote: Can Pg be changed to support . in operator names as long as they don't just appear by themselves? What would this break to do so? Someone else would have to comment on that. My feeling is that it might create problems

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Kevin Grittner
Simon Riggs si...@2ndquadrant.com wrote: Before you arrived, it was quite normal to suggest tuning patches after feature freeze. I've worn a lot of hats in the practical end of this industry, and regardless of which perspective I look at this from, I can't think of anything so destructive to

Re: [HACKERS] [Pgbuildfarm-members] CREATE FUNCTION hang on test machine polecat on HEAD

2011-06-07 Thread Alex Hunsaker
On Tue, Jun 7, 2011 at 12:22, Tom Lane t...@sss.pgh.pa.us wrote: Alex Hunsaker bada...@gmail.com writes: Im looking at the raw perl 5.10.0 source... I wonder if apple is shipping a modified version? You could find out by digging around at http://www.opensource.apple.com/ polecat appears to

Re: [HACKERS] [Pgbuildfarm-members] CREATE FUNCTION hang on test machine polecat on HEAD

2011-06-07 Thread Andrew Dunstan
On 06/07/2011 02:22 PM, Tom Lane wrote: Alex Hunsakerbada...@gmail.com writes: Im looking at the raw perl 5.10.0 source... I wonder if apple is shipping a modified version? You could find out by digging around at http://www.opensource.apple.com/ polecat appears to be running OSX 10.6.7, so

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Robert Haas
On Tue, Jun 7, 2011 at 2:06 PM, Simon Riggs si...@2ndquadrant.com wrote: On Tue, Jun 7, 2011 at 6:33 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Jun 7, 2011 at 1:27 PM, Joshua Berkus j...@agliodbs.com wrote: As long as we have solidarity of the committers that this is not allowed,

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: Before you arrived, it was quite normal to suggest tuning patches after feature freeze. *Low risk* tuning patches make sense at this stage, yes. Fooling with the lock mechanisms doesn't qualify as low risk in my book. The probability of undetected

Re: [HACKERS] heap vacuum cleanup locks

2011-06-07 Thread Greg Stark
On Mon, Jun 6, 2011 at 11:30 PM, Simon Riggs si...@2ndquadrant.com wrote: But I think you've hit the important point here. The problem is not whether VACUUM waits for the pin, its that the pins can be held for extended periods. Yes It makes more sense to try to limit pin hold times than it

Re: [HACKERS] patch for new feature: Buffer Cache Hibernation

2011-06-07 Thread Greg Smith
On 06/05/2011 08:50 AM, Mitsuru IWASAKI wrote: It seems that I don't have enough time to complete this work. You don't need to keep cc'ing me, and I'm very happy if postgres to be the first DBMS which support buffer cache hibernation feature. Thanks for submitting the patch, and we'll see

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Jignesh Shah
On Mon, Jun 6, 2011 at 11:20 PM, Jignesh Shah jks...@gmail.com wrote: Okay I tried it out with sysbench read scaling test.. Note I had tried that earlier on 9.0 http://jkshah.blogspot.com/2010/11/postgresql-90-simple-select-scaling.html And on that test I found that doing that test on

Re: [HACKERS] heap vacuum cleanup locks

2011-06-07 Thread Simon Riggs
On Tue, Jun 7, 2011 at 8:24 PM, Greg Stark gsst...@mit.edu wrote: On Mon, Jun 6, 2011 at 11:30 PM, Simon Riggs si...@2ndquadrant.com wrote: But I think you've hit the important point here. The problem is not whether VACUUM waits for the pin, its that the pins can be held for extended periods.

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I plead guilty to taking my eye off the ball post-beta1. I busted my ass for two months stabilizing other people's code after CF4 was over, and then I moved on to other things. I will try to get my eye back on the ball - but actually I'm not sure

Re: [HACKERS] contrib/citext versus collations

2011-06-07 Thread Greg Stark
On Mon, Jun 6, 2011 at 9:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: The most workable alternative that I can see is to lobotomize citext so that it always does lower-casing according to the database's default collation, which would allow us to pretend that its notion of equality is not

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Simon Riggs
On Tue, Jun 7, 2011 at 7:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Before you arrived, it was quite normal to suggest tuning patches after feature freeze. *Low risk* tuning patches make sense at this stage, yes.  Fooling with the lock mechanisms

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: Moving on from that, I have proposed other solutions. Koichi, Jignesh and and then Robert have shown measurements of the huge contention in this area of our software. Robert's patch addresses the problems, as do Koichi's and my latest patch. I would

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Robert Haas
On Tue, Jun 7, 2011 at 3:44 PM, Jignesh Shah jks...@gmail.com wrote: On Mon, Jun 6, 2011 at 11:20 PM, Jignesh Shah jks...@gmail.com wrote: Okay I tried it out with sysbench read scaling test.. Note I had tried that earlier on 9.0

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Simon Riggs
On Tue, Jun 7, 2011 at 9:00 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Moving on from that, I have proposed other solutions. Koichi, Jignesh and and then Robert have shown measurements of the huge contention in this area of our software. Robert's patch

Re: [HACKERS] contrib/citext versus collations

2011-06-07 Thread Tom Lane
Greg Stark gsst...@mit.edu writes: On Mon, Jun 6, 2011 at 9:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: The most workable alternative that I can see is to lobotomize citext so that it always does lower-casing according to the database's default collation, which would allow us to pretend that its

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Robert Haas
On Tue, Jun 7, 2011 at 12:51 PM, Simon Riggs si...@2ndquadrant.com wrote: Stefan/Robert's observation that we perform a VirtualXactLockTableInsert() to no real benefit is a good one. It leads to the following simple patch to remove one lock table hit per transaction. It's a lot smaller impact

Re: [HACKERS] [Pgbuildfarm-members] CREATE FUNCTION hang on test machine polecat on HEAD

2011-06-07 Thread Tom Lane
Robert Creager robert.crea...@oracle.com writes: Another question worth asking here is whether PG is picking up perl 5.10.0 or 5.8.9, both of which are shipped in this OSX release. Hmm... This might be a problem: which perl /opt/local/bin/perl type -a perl /opt/local/bin/perl

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Simon Riggs
On Tue, Jun 7, 2011 at 9:52 PM, Robert Haas robertmh...@gmail.com wrote: If we were to fix ONLY the vxid issue in 9.1 as you were advocating Sensible debate is impossible when you don't read what I've written. --  Simon Riggs   http://www.2ndQuadrant.com/  PostgreSQL

Re: [HACKERS] [Pgbuildfarm-members] CREATE FUNCTION hang on test machine polecat on HEAD

2011-06-07 Thread Alvaro Herrera
Excerpts from Tom Lane's message of mar jun 07 14:22:13 -0400 2011: Alex Hunsaker bada...@gmail.com writes: Im looking at the raw perl 5.10.0 source... I wonder if apple is shipping a modified version? You could find out by digging around at http://www.opensource.apple.com/ polecat

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Robert Haas
On Tue, Jun 7, 2011 at 5:43 PM, Simon Riggs si...@2ndquadrant.com wrote: On Tue, Jun 7, 2011 at 9:52 PM, Robert Haas robertmh...@gmail.com wrote: If we were to fix ONLY the vxid issue in 9.1 as you were advocating Sensible debate is impossible when you don't read what I've written. I've read

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-07 Thread Josh Berkus
On 6/7/11 1:11 PM, Simon Riggs wrote: that appear low risk. I seriously doubt that I would consider *any* meaningful change in the locking area to be low risk. That's a shame. We'll fix it in 9.2 then. I will point out that we bounced Alvaro's FK patch, which *was* submitted in time for CF4,

Re: [HACKERS] heap vacuum cleanup locks

2011-06-07 Thread Robert Haas
On Tue, Jun 7, 2011 at 3:43 PM, Simon Riggs si...@2ndquadrant.com wrote: On Tue, Jun 7, 2011 at 8:24 PM, Greg Stark gsst...@mit.edu wrote: On Mon, Jun 6, 2011 at 11:30 PM, Simon Riggs si...@2ndquadrant.com wrote: But I think you've hit the important point here. The problem is not whether

Re: [HACKERS] Domains versus polymorphic functions, redux

2011-06-07 Thread Tom Lane
I wrote: Anyway, I think we're out of time to do anything about the issue for 9.1. I think what we'd better do is force a downcast in the context of matching to an ANYARRAY parameter, and leave the other cases to revisit later. Attached is a draft patch to do the above. It's only lightly

Re: [HACKERS] smallserial / serial2

2011-06-07 Thread Brar Piening
On Wed, 20 Apr 2011 21:27:27 -0400, Mike Pultz m...@mikepultz.com wrote: Can this be added? Probably not - since it's not a complete patch ;-) I tried to test this one but was unable to find a complete version of the patch in my local mail archives and in the official archives

Re: [HACKERS] SIREAD lock versus ACCESS EXCLUSIVE lock

2011-06-07 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 07.06.2011 21:10, Kevin Grittner wrote: I think that leaves me with all the answers I need to get a new patch out this evening (U.S. Central Time). Great, I'll review it in my morning (in about 12h) Attached. Passes all

[HACKERS] could not truncate directory pg_serial: apparent wraparound

2011-06-07 Thread Kevin Grittner
We had a report of the subject message during testing a while back and attempted to address the issue. It can result in a LOG level message and the accumulation of files in the pg_serial SLRU subdirectory. We haven't seen a recurrence, until I hit it during testing of the just-posted patch for

Re: [HACKERS] smallserial / serial2

2011-06-07 Thread Mike Pultz
Sorry, forgot the documentation- I guess that stuff doesn't magically happen! New patch attached. Mike From: Brar Piening [mailto:b...@gmx.de] Sent: Tuesday, June 07, 2011 6:58 PM To: Mike Pultz Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] smallserial / serial2 On Wed,

[HACKERS] reindex creates predicate lock on index root

2011-06-07 Thread Kevin Grittner
During testing of the SSI DDL changes I noticed that a REINDEX INDEX created a predicate lock on page 0 of the index. This is pretty harmless, but mildly annoying. There are a few other places where it would be good to suppress predicate locks; these are listed on the RD section of the Wiki. I

Re: [HACKERS] Domains versus polymorphic functions, redux

2011-06-07 Thread Robert Haas
On Tue, Jun 7, 2011 at 6:28 PM, Tom Lane t...@sss.pgh.pa.us wrote: I wrote: Anyway, I think we're out of time to do anything about the issue for 9.1.  I think what we'd better do is force a downcast in the context of matching to an ANYARRAY parameter, and leave the other cases to revisit

Re: [HACKERS] Invalid byte sequence for encoding UTF8, caused due to non wide-char-aware downcase_truncate_identifier() function on WINDOWS

2011-06-07 Thread Robert Haas
2011/6/7 Jeevan Chalke jeevan.cha...@enterprisedb.com: since we smash the identifier to lower case using downcase_truncate_identifier() function, the solution is to make this function should be wide-char aware, like LOWER() function functionality. I see some discussion related to

Re: 9.1 release scheduling (was Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch)

2011-06-07 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mar jun 07 13:53:23 -0400 2011: On Tue, Jun 7, 2011 at 1:45 PM, Thom Brown t...@linux.com wrote: Speaking of which, is it now safe to remove the NOT VALID constraints don't dump properly issue from the blocker list since the fix has been committed?

Re: [HACKERS] Domains versus polymorphic functions, redux

2011-06-07 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jun 7, 2011 at 6:28 PM, Tom Lane t...@sss.pgh.pa.us wrote: Note that I changed coerce_type's behavior for both ANYARRAY and ANYENUM targets, but the latter behavioral change is unreachable since the other routines in parse_coerce.c will not

Re: [HACKERS] Domains versus polymorphic functions, redux

2011-06-07 Thread Robert Haas
On Tue, Jun 7, 2011 at 9:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jun 7, 2011 at 6:28 PM, Tom Lane t...@sss.pgh.pa.us wrote: Note that I changed coerce_type's behavior for both ANYARRAY and ANYENUM targets, but the latter behavioral change is

Re: [HACKERS] reindex creates predicate lock on index root

2011-06-07 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: During testing of the SSI DDL changes I noticed that a REINDEX INDEX created a predicate lock on page 0 of the index. This is pretty harmless, but mildly annoying. There are a few other places where it would be good to suppress predicate

Re: [HACKERS] WALInsertLock tuning

2011-06-07 Thread Fujii Masao
On Tue, Jun 7, 2011 at 9:54 PM, Simon Riggs si...@2ndquadrant.com wrote: On Tue, Jun 7, 2011 at 1:24 PM, Robert Haas robertmh...@gmail.com wrote: One other thought is that I think that this patch might cause a user-visible behavior change.  Right now, when you hit the end of recovery, you

  1   2   >