Re: [HACKERS] EXPLAIN verbose?

2006-05-11 Thread Qingqing Zhou
"Cristiano Duarte" <[EMAIL PROTECTED]> wrote > Does anyone know how to discover the tables (with schemas or oid) > involved > on a select statement? > I tried EXPLAIN but it only reports the table names, not schemas or > oid. > In general, this is a pretty difficult problem - thinking you may d

[HACKERS] Continuing "pgstat update timeout" failures in buildfarm

2006-05-11 Thread Tom Lane
The reliability of that recent pgstat patch is clearly completely unacceptable. Please revert it until it's fixed. I'm tired of looking at intermittent buildfarm failures. regards, tom lane ---(end of broadcast)--- TIP 1: i

Re: [PERFORM] [HACKERS] Big IN() clauses etc : feature proposal

2006-05-11 Thread Jim C. Nasby
On Thu, May 11, 2006 at 06:08:36PM -0400, Tom Lane wrote: > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > I'd hope that wasn't what's happening... is the backend smart enough to > > know not to fsync anything involved with the temp table? > > The catalog entries required for it have to be fsync'd

Re: [PERFORM] [HACKERS] Big IN() clauses etc : feature proposal

2006-05-11 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > I'd hope that wasn't what's happening... is the backend smart enough to > know not to fsync anything involved with the temp table? The catalog entries required for it have to be fsync'd, unless you enjoy putting your entire database at risk (a bad block

Re: [HACKERS] Compressing table images

2006-05-11 Thread Jim C. Nasby
On Thu, May 11, 2006 at 05:05:26PM -0400, Alvaro Herrera wrote: > Joshua D. Drake wrote: > > Brian Hurt wrote: > > >My apologies if this subject has already been hashed to death, or if > > >this is the wrong list, but I was wondering if people had seen this paper: > > >http://www.cwi.nl/htbin/ins1

Re: [PERFORM] [HACKERS] Big IN() clauses etc : feature proposal

2006-05-11 Thread PFC
> 0.450 ms INSERT INTO tmp SELECT * FROM bookmarks ORDER BY annonce_id DESC > LIMIT 20 > 0.443 ms ANALYZE tmp > 0.365 ms SELECT * FROM tmp > 0.310 ms DROP TABLE tmp > 32.918 ms COMMIT The 32 seconds for commit can hardly be catalog related. It seems the file is fsynced before it is dropped.

Re: [HACKERS] hashagg, statistisics and excessive memory allocation

2006-05-11 Thread Jim C. Nasby
On Thu, May 11, 2006 at 08:36:25PM +0200, Stefan Kaltenbrunner wrote: > Hi! > > on irc somebody complained yesterday that a simple group by on a 25M > integer row caused his backend to exhaust the 3GB process limit on his > 32bit built(one a box with 16GB Ram). > Some testing showed that the plann

Re: [HACKERS] Compressing table images

2006-05-11 Thread Alvaro Herrera
Joshua D. Drake wrote: > Brian Hurt wrote: > >My apologies if this subject has already been hashed to death, or if > >this is the wrong list, but I was wondering if people had seen this paper: > >http://www.cwi.nl/htbin/ins1/publications?request=intabstract&key=ZuHeNeBo:ICDE:06 > > > > > > > >Bas

Re: [PERFORM] [HACKERS] Big IN() clauses etc : feature proposal

2006-05-11 Thread Jim C. Nasby
On Thu, May 11, 2006 at 09:55:15AM +0200, Zeugswetter Andreas DCP SD wrote: > > > Something else worth considering is not using the normal > > catalog methods > > for storing information about temp tables, but hacking that together > > would probably be a rather large task. > > But the timings s

Re: [HACKERS] sblock state on FreeBSD 6.1

2006-05-11 Thread Jim C. Nasby
On Thu, May 11, 2006 at 09:50:14PM +0200, Martijn van Oosterhout wrote: > On Thu, May 11, 2006 at 02:39:14PM -0500, Jim C. Nasby wrote: > > On Thu, May 11, 2006 at 07:39:23PM +0200, Martijn van Oosterhout wrote: > > > This is an idle backend waiting for the user. > > > > So why would that be waiti

Re: [HACKERS] Compressing table images

2006-05-11 Thread Joshua D. Drake
Brian Hurt wrote: My apologies if this subject has already been hashed to death, or if this is the wrong list, but I was wondering if people had seen this paper: http://www.cwi.nl/htbin/ins1/publications?request=intabstract&key=ZuHeNeBo:ICDE:06 Basically it describes a compression algorithm f

Re: [HACKERS] BEGIN inside transaction should be an error

2006-05-11 Thread Simon Riggs
On Wed, 2006-05-10 at 21:24 -0400, Tom Lane wrote: > Martijn van Oosterhout writes: > > How do other database deal with this? Either they nest BEGIN/COMMIT or > > they probably throw an error without aborting the transaction, which is > > pretty much what we do. Is there a database that actually a

[HACKERS] Compressing table images

2006-05-11 Thread Brian Hurt
My apologies if this subject has already been hashed to death, or if this is the wrong list, but I was wondering if people had seen this paper: http://www.cwi.nl/htbin/ins1/publications?request=intabstract&key=ZuHeNeBo:ICDE:06 Basically it describes a compression algorithm for tables of a datab

Re: [HACKERS] BEGIN inside transaction should be an error

2006-05-11 Thread Jim C. Nasby
On Thu, May 11, 2006 at 08:05:57AM +0200, Tommi Maekitalo wrote: > I'd vote for breaking broken applications and leave the > database-administrator > reactivate this currently broken behavior of postgresql via GUC. +1... As for whether this should or shouldn't abort the current transaction, I'd

Re: [PERFORM] [HACKERS] Big IN() clauses etc : feature proposal

2006-05-11 Thread Jim C. Nasby
On Thu, May 11, 2006 at 08:43:46PM +0200, Martijn van Oosterhout wrote: > On Thu, May 11, 2006 at 11:35:34AM -0400, Greg Stark wrote: > > I can say that I've seen plenty of instances where the ability to create > > temporary tables very quickly with no overhead over the original query would > > be

Re: [HACKERS] sblock state on FreeBSD 6.1

2006-05-11 Thread Alvaro Herrera
Jim C. Nasby wrote: > Yeah, my suspicion is that those processes had moved past waiting on the > socket lock by the time gdb got to them. Any idea of how you could tell > what state (as reported by top) the process was in when gdb stopped it? I think you could send SIGSTOP to all backends at once

Re: [HACKERS] [PERFORM] Big IN() clauses etc : feature proposal

2006-05-11 Thread Jim C. Nasby
On Thu, May 11, 2006 at 08:03:19PM +0200, Martijn van Oosterhout wrote: > On Thu, May 11, 2006 at 12:18:06PM -0500, Jim C. Nasby wrote: > > > Yes, because there can be more than one active snapshot within a single > > > transaction (think about volatile functions in particular). > > > > Any docume

Re: [HACKERS] XLOG_BLCKSZ vs. wal_buffers table

2006-05-11 Thread Mark Wong
Simon Riggs wrote: On Wed, 2006-05-10 at 09:55 -0700, Mark Wong wrote: Simon Riggs wrote: Could you turn full_page_writes = off and do a few more tests? I think the full page writes is swamping the xlog and masking the performance we might see for normal small xlog writes. I'd try XLOG_BLCKSZ =

Re: [HACKERS] sblock state on FreeBSD 6.1

2006-05-11 Thread Martijn van Oosterhout
On Thu, May 11, 2006 at 02:39:14PM -0500, Jim C. Nasby wrote: > On Thu, May 11, 2006 at 07:39:23PM +0200, Martijn van Oosterhout wrote: > > This is an idle backend waiting for the user. > > So why would that be waiting to lock the socket? My understanding is > that nothing else should be contendin

Re: [HACKERS] sblock state on FreeBSD 6.1

2006-05-11 Thread Jim C. Nasby
On Thu, May 11, 2006 at 07:39:23PM +0200, Martijn van Oosterhout wrote: > On Thu, May 11, 2006 at 12:09:56PM -0500, Jim C. Nasby wrote: > > Unfortunately, I suspect some of these were grabbed after the process > > had already moved past whatever was holding it in sblock. > > > > Here's the traces

Re: [HACKERS] XLOG_BLCKSZ vs. wal_buffers table

2006-05-11 Thread Simon Riggs
On Wed, 2006-05-10 at 09:55 -0700, Mark Wong wrote: > Simon Riggs wrote: > > Could you turn full_page_writes = off and do a few more tests? I think > > the full page writes is swamping the xlog and masking the performance we > > might see for normal small xlog writes. > > I'd try XLOG_BLCKSZ = 4096

Re: [HACKERS] hashagg, statistisics and excessive memory allocation

2006-05-11 Thread Josh Berkus
Stefan, > Some testing showed that the planner was seriously underestimating the > number of distinct rows in the table (with the default statistic target > it estimated ~150k rows while there are about 19M distinct values) and > chosing a hashagg for the aggregate. > uping the statistics target t

[HACKERS] hashagg, statistisics and excessive memory allocation

2006-05-11 Thread Stefan Kaltenbrunner
Hi! on irc somebody complained yesterday that a simple group by on a 25M integer row caused his backend to exhaust the 3GB process limit on his 32bit built(one a box with 16GB Ram). Some testing showed that the planner was seriously underestimating the number of distinct rows in the table (with th

Re: [PERFORM] [HACKERS] Big IN() clauses etc : feature proposal

2006-05-11 Thread Martijn van Oosterhout
On Thu, May 11, 2006 at 11:35:34AM -0400, Greg Stark wrote: > I can say that I've seen plenty of instances where the ability to create > temporary tables very quickly with no overhead over the original query would > be useful. I wonder if this requires what the standard refers to as a global tempo

Re: [HACKERS] [PERFORM] Big IN() clauses etc : feature proposal

2006-05-11 Thread Martijn van Oosterhout
On Thu, May 11, 2006 at 12:18:06PM -0500, Jim C. Nasby wrote: > > Yes, because there can be more than one active snapshot within a single > > transaction (think about volatile functions in particular). > > Any documentation on how snapshot's work? They're a big mystery to me. > :( A snapshot is a

Re: [HACKERS] [PERFORM] Big IN() clauses etc : feature proposal

2006-05-11 Thread Scott Marlowe
On Thu, 2006-05-11 at 12:18, Jim C. Nasby wrote: > On Wed, May 10, 2006 at 08:31:54PM -0400, Tom Lane wrote: > > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > > On Tue, May 09, 2006 at 03:13:01PM -0400, Tom Lane wrote: > > >> PFC <[EMAIL PROTECTED]> writes: > > >>> Fun thing is, the rowcount from

Re: [HACKERS] sblock state on FreeBSD 6.1

2006-05-11 Thread Martijn van Oosterhout
On Thu, May 11, 2006 at 12:09:56PM -0500, Jim C. Nasby wrote: > Unfortunately, I suspect some of these were grabbed after the process > had already moved past whatever was holding it in sblock. > > Here's the traces that we captured... > > Got 2 of these: > #0 0x00080135bd2c in recvfrom () f

Re: [HACKERS] [PERFORM] Big IN() clauses etc : feature proposal

2006-05-11 Thread Jim C. Nasby
On Wed, May 10, 2006 at 08:31:54PM -0400, Tom Lane wrote: > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > On Tue, May 09, 2006 at 03:13:01PM -0400, Tom Lane wrote: > >> PFC <[EMAIL PROTECTED]> writes: > >>> Fun thing is, the rowcount from a temp table (which is the problem here) > >>> should be

Re: [HACKERS] sblock state on FreeBSD 6.1

2006-05-11 Thread Jim C. Nasby
On Wed, May 10, 2006 at 05:23:29PM -0500, Jim C. Nasby wrote: > On Thu, May 11, 2006 at 12:09:32AM +0200, Martijn van Oosterhout wrote: > > > > This stack trace doesn't make any sense. ClosePostmasterPorts is not > > > > calling PostgresMain. And pq_getbyte is not calling TouchSocketFile, > > > >

Re: [HACKERS] Bug in signal handler

2006-05-11 Thread Martijn van Oosterhout
On Thu, May 11, 2006 at 10:11:00AM -0400, Tom Lane wrote: > Douglas McNaught <[EMAIL PROTECTED]> writes: > > I don't disagree with your reasoning, but does POSIX actually say > > this? > > The fact remains that the postmaster has *always* been coded like that, > and we have *never* seen any proble

Re: [PERFORM] [HACKERS] Big IN() clauses etc : feature proposal

2006-05-11 Thread Greg Stark
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > Perhaps it would be worth creating a class of temporary tables that used > a tuplestore, although that would greatly limit what could be done with > that temp table. I can say that I've seen plenty of instances where the ability to create temporary ta

[HACKERS] Upcoming releases

2006-05-11 Thread Tom Lane
Per discussion among pgsql-packagers, we'll release 8.1.4 and updates in the back branches next week; tentative plan is tarballs wrapped Monday, public release Wednesday. Get those last-minute fixes in now ... regards, tom lane ---(end of broadcast

Re: [HACKERS] Bug in signal handler

2006-05-11 Thread Tom Lane
Douglas McNaught <[EMAIL PROTECTED]> writes: > Martijn van Oosterhout writes: >> Running unsafe functions within a signal handler is not unsafe per-se. >> It's only unsafe if the main program could also be running unsafe >> functions. > I don't disagree with your reasoning, but does POSIX actuall

Re: [HACKERS] Bug in signal handler

2006-05-11 Thread Martijn van Oosterhout
On Thu, May 11, 2006 at 08:24:02AM -0400, Douglas McNaught wrote: > Martijn van Oosterhout writes: > > > Running unsafe functions within a signal handler is not unsafe per-se. > > It's only unsafe if the main program could also be running unsafe > > functions. > > I don't disagree with your reas

Re: [HACKERS] [COMMITTERS] pgsql: Clean up code associated with updating

2006-05-11 Thread Bruce Momjian
Tom Lane wrote: > Alvaro Herrera <[EMAIL PROTECTED]> writes: > > Minor comment: since you removed the - lines from the index_create > > header comment, I think pgindent will mess up the alignment for that > > comment, which probably will not be very nice. Same in > > index_update_stats. Other

Re: [HACKERS] [COMMITTERS] pgsql: Clean up code associated with updating pg_class statistics

2006-05-11 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Minor comment: since you removed the - lines from the index_create > header comment, I think pgindent will mess up the alignment for that > comment, which probably will not be very nice. Same in > index_update_stats. Otherwise, nice work! I will u

Re: [HACKERS] Bug in signal handler

2006-05-11 Thread Douglas McNaught
Martijn van Oosterhout writes: > Running unsafe functions within a signal handler is not unsafe per-se. > It's only unsafe if the main program could also be running unsafe > functions. I don't disagree with your reasoning, but does POSIX actually say this? -Doug ---(end

Re: [HACKERS] Bug in signal handler [Was: [TODO] Allow commenting

2006-05-11 Thread Martijn van Oosterhout
On Thu, May 11, 2006 at 01:59:46PM +0200, Zdenek Kotala wrote: > Decision is that Postgres uses signal dangerous functions (fopen, ...) > and its signal handler is not save and should generate unpredictable > behavior after signal processing. I would like to fix it, but there is > some waiting p

Re: [HACKERS] Bug in signal handler [Was: [TODO] Allow commenting

2006-05-11 Thread Zdenek Kotala
Alvaro Herrera wrote: Zdenek Kotala wrote: I performed some investigation and I found that signal handler (SIGHUP_handler) contents a big code and contents signal nonsafe functions. It should generate deadlock or damage some internal data structure in the standard c library. See h

Re: [PERFORM] [HACKERS] Big IN() clauses etc : feature proposal

2006-05-11 Thread Martijn van Oosterhout
On Thu, May 11, 2006 at 09:55:15AM +0200, Zeugswetter Andreas DCP SD wrote: > > 0.101 ms BEGIN > > 1.451 ms CREATE TEMPORARY TABLE tmp ( a INTEGER NOT NULL, b INTEGER > NOT > > NULL, c TIMESTAMP NOT NULL, d INTEGER NOT NULL ) ON COMMIT DROP > > 1.4 seconds is not great for create table, is that

Re: [HACKERS] intarray internals

2006-05-11 Thread Volkan YAZICI
Hi, First, thanks so much for your reply. On May 10 04:01, Teodor Sigaev wrote: > > Again, in g_int_decompress(), I couldn't figure out the functionality of > > below lines: > > gist__int_ops use rangeset compression technique, read about in "THE > RD-TREE: AN INDEX STRUCTURE FOR SETS", Joseph

Re: [PERFORM] [HACKERS] Big IN() clauses etc : feature proposal

2006-05-11 Thread Zeugswetter Andreas DCP SD
> Something else worth considering is not using the normal > catalog methods > for storing information about temp tables, but hacking that together > would probably be a rather large task. But the timings suggest, that it cannot be the catalogs in the worst case he showed. > 0.101 ms BEGIN > 1.

Re: [HACKERS] BEGIN inside transaction should be an error

2006-05-11 Thread Marko Kreen
On 5/11/06, Martijn van Oosterhout wrote: On Wed, May 10, 2006 at 04:03:51PM -0500, Jim C. Nasby wrote: > If the existing application is trying to start a new transaction from > within an existing one, I'd say it's already broken and we're just > hiding that fact. Well maybe, except the extra B

Re: [HACKERS] .pgpass file and unix domain sockets

2006-05-11 Thread Martijn van Oosterhout
On Wed, May 10, 2006 at 09:34:38PM -0400, Tom Lane wrote: > I'm not sure if that's a bug or not. Arguably, different socket paths > might point to different servers for which you need different passwords. > If we did want unix-socket connections to search for "localhost" > regardless of socket pat