Re: [HACKERS] slow IN() clause for many cases

2005-10-11 Thread Jonah H. Harris
true dat :) On 10/12/05, Tom Lane <[EMAIL PROTECTED]> wrote: "Ilia Kantor" <[EMAIL PROTECTED]> writes:> Bitmap Heap Scan on objects_hier  (cost=60.29..179.57 rows=80 width=600)> (actual time=0.835..1.115 rows=138 loops=1) vs>  Merge Join  (cost=62.33..576.80 rows=1117 width=600) (actual> time=0.54

Re: [HACKERS] slow IN() clause for many cases

2005-10-11 Thread Tom Lane
"Ilia Kantor" <[EMAIL PROTECTED]> writes: > Bitmap Heap Scan on objects_hier (cost=60.29..179.57 rows=80 width=600) > (actual time=0.835..1.115 rows=138 loops=1) vs > Merge Join (cost=62.33..576.80 rows=1117 width=600) (actual > time=0.542..2.898 rows=138 loops=1) Hmm, sure looks from here li

Re: [HACKERS] slow IN() clause for many cases

2005-10-11 Thread Ilia Kantor
>> It is bitmap-OR on multiple index(PK) lookups. > Describing it doesn't help. We need an *actual* EXPLAIN ANALYZE. Sure, why not.. 6ms for Bitmap Heap Scan on objects_hier (cost=60.29..179.57 rows=80 width=600) (actual time=0.835..1.115 rows=138 loops=1) Recheck Cond: ((id = 1) OR (id =

Re: [HACKERS] How TODO prevent PQfnumber() from lowercasing?

2005-10-11 Thread Bruce Momjian
Volkan YAZICI wrote: > Hi, > > Which way do you suggest to "Prevent libpq's PQfnumber() from > lowercasing the column name" (which is listed as a TODO item). If > column name has quotes around it we're just removing the quotes and > comparing with the related column name. Else, lowercasing the col

Re: [HACKERS] pg_dump option to dump only functions

2005-10-11 Thread Bruce Momjian
Tino Wildenhain wrote: > > Proposed TODO entries for pg_dump: > > > > * Allow selection of individual object(s) of all types, not just tables > > * In a selective dump, allow dumping of all dependencies of the objects > > May I suggest the implementation of -l / -L like pg_restore has? > So you c

Re: [HACKERS] database vacuum from cron hanging

2005-10-11 Thread Tom Lane
I wrote: > "Kevin Grittner" <[EMAIL PROTECTED]> writes: >> (gdb) p BufferDescriptors[781] >> $1 = {tag = {rnode = {spcNode = 1663, dbNode = 16385, relNode = 2666}, >> blockNum = 1}, flags = 70, usage_count = 5, refcount = 4294967294, >> wait_backend_pid = 748, buf_hdr_lock = 0 '\0', buf_id = 781,

Re: [HACKERS] PG 8.1beta3 out soon

2005-10-11 Thread Andrew Dunstan
Greg Sabino Mullane wrote: That would probably work, but it would ONLY deal with the issue for $_TD. In your function $event will still hit this problem. Well, fixing $_TD would pretty much fix all the problems I've been having. As far as "$event", that is in my control and easily fixed

Re: [HACKERS] pg_dump option to dump only functions

2005-10-11 Thread Bruce Momjian
Tom Lane wrote: > Josh Berkus writes: > >> I was wonderring, because I create a lot of server side utility functions, > >> whether adding an option to pg_dump to just dump functions has been > >> considered. I did a quick perusal of the code, and noted that there is a > >> separate section within

[HACKERS] Beta3 Bundled

2005-10-11 Thread Marc G. Fournier
Sizes look right compared to beta2 ... please check it over and make sure there are no outstanding issues ... will announce over the next 24-48 hrs, once Dave has had a change to get the pgInstaller up to date ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.o

Re: [HACKERS] database vacuum from cron hanging

2005-10-11 Thread Tom Lane
"Kevin Grittner" <[EMAIL PROTECTED]> writes: > (gdb) p BufferDescriptors[781] > $1 = {tag = {rnode = {spcNode = 1663, dbNode = 16385, relNode = 2666}, > blockNum = 1}, flags = 70, usage_count = 5, refcount = 4294967294, > wait_backend_pid = 748, buf_hdr_lock = 0 '\0', buf_id = 781, freeNext = -2

Re: [HACKERS] database vacuum from cron hanging

2005-10-11 Thread Kevin Grittner
(gdb) p BufferDescriptors[781] $1 = {tag = {rnode = {spcNode = 1663, dbNode = 16385, relNode = 2666}, blockNum = 1}, flags = 70, usage_count = 5, refcount = 4294967294, wait_backend_pid = 748, buf_hdr_lock = 0 '\0', buf_id = 781, freeNext = -2, io_in_progress_lock = 1615, content_lock = 1616}

Re: [HACKERS] database vacuum from cron hanging

2005-10-11 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Trivial observation: process 748 is a manually-issued VACUUM (manually, > by cron), it's holding locks other VACUUMs are waiting for, and is > waiting on LockBufferForCleanup. I guess this means it lost a signal, > or somebody else is holding a pin on t

Re: [HACKERS] slower merge join on sorted data chosen over

2005-10-11 Thread Kevin Grittner
You don't get to read part of a page, but you may be dealing with probabilities. For example, consider a case where there are ten data pages, and you are going to read 15% of the tuples. There is a 50% chance that your scan will start in the first half of a leaf page and only need two leaf pages.

Re: [HACKERS] avoid pulling up subquerys that contain volatile functions?

2005-10-11 Thread Jaime Casanova
On 10/9/05, Tom Lane <[EMAIL PROTECTED]> wrote: > Jaime Casanova <[EMAIL PROTECTED]> writes: > > On 10/8/05, Tom Lane <[EMAIL PROTECTED]> wrote: > >> This is exactly the same example discussed in previous threads on this > >> issue. Do you think it will change anyone's mind? > > > in any case, i s

Re: [HACKERS] Spinlocks and CPU Architectures

2005-10-11 Thread Tom Lane
Martijn van Oosterhout writes: > On Tue, Oct 11, 2005 at 02:28:02PM -0400, Tom Lane wrote: >> One thought that comes to mind is that these decisions are probably >> comparable to those made by gcc conditional on -march flags. Do we >> get access to the -march setting by means of predefined symbol

[HACKERS] beta2 no longer builds with MSVC?

2005-10-11 Thread Magnus Hagander
Has anybody tried building beta2 or later with MSVC? It doesn'ät work for me - it builds fine, but whenever I run with it I get a coredump from it whenever I try to connect. If I revert it to the 8.0 version of port/getaddrinfo.c, things work again. The problem shows itself in that conn->addr_c

Re: [HACKERS] database vacuum from cron hanging

2005-10-11 Thread Alvaro Herrera
Kevin Grittner wrote: > This may or may not be related to previous threads regarding vacuum problems. > Following the last thread, we built the development snapshot of Oct 6 with > --enable-integer-datetimes and --enable-debug. We have had the autovacuum > running every ten seconds. The only tabl

Re: [HACKERS] Spinlocks and CPU Architectures

2005-10-11 Thread Peter Eisentraut
Simon Riggs wrote: > Tom is suggesting having different behaviour for x86 and x86_64. The > x86 will still run on x86_64 architecture would it not? So we'll have > two binaries for each OS, yes? A quick glance around tells me that most free operating systems are treating x86 and x86_64 as separat

[HACKERS] database vacuum from cron hanging

2005-10-11 Thread Kevin Grittner
This may or may not be related to previous threads regarding vacuum problems. Following the last thread, we built the development snapshot of Oct 6 with --enable-integer-datetimes and --enable-debug. We have had the autovacuum running every ten seconds. The only table which will meet the autovacu

Re: [HACKERS] Spinlocks and CPU Architectures

2005-10-11 Thread Martijn van Oosterhout
On Tue, Oct 11, 2005 at 02:28:02PM -0400, Tom Lane wrote: > One thought that comes to mind is that these decisions are probably > comparable to those made by gcc conditional on -march flags. Do we > get access to the -march setting by means of predefined symbols? > If so we could compile different

Re: [HACKERS] slow IN() clause for many cases

2005-10-11 Thread Josh Berkus
Ilia, > It is bitmap-OR on multiple index(PK) lookups. Describing it doesn't help. We need an *actual* EXPLAIN ANALYZE. -- --Josh Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster

Re: [HACKERS] slow IN() clause for many cases

2005-10-11 Thread Ilia Kantor
>Please post an explain analyze on your query with a 20-30 item IN clause so that we can see what plan is being generated. It is bitmap-OR on multiple index(PK) lookups. ---(end of broadcast)--- TIP 4: Have you searched our list archives?

Re: [HACKERS] Spinlocks and CPU Architectures

2005-10-11 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Tue, 2005-10-11 at 18:45 +0200, Peter Eisentraut wrote: >> A number of packages in the video area (and perhaps others) do compile >> "sub-architecture" specific variants. This could be done for >> PostgreSQL, but you'd probably need to show some prett

Re: [HACKERS] Spinlocks and CPU Architectures

2005-10-11 Thread Dann Corbit
As an aside, here is a package that has recently been BSD re-licensed: http://sourceforge.net/projects/libltx/ It is a lightweight memory transaction package. It comes with a paper entitled "Cache Sensitive Software Transactional Memory" by Robert Ennals. In the paper, Robert Ennals suggests th

Re: [HACKERS] Spinlocks and CPU Architectures

2005-10-11 Thread Simon Riggs
On Tue, 2005-10-11 at 18:45 +0200, Peter Eisentraut wrote: > Tom Lane wrote: > > This seems pretty unworkable from a packaging standpoint. Even if > > you teach autoconf how to tell which model it's running on, there's > > no guarantee that the resulting executables will be used on that same > > m

Re: [HACKERS] PG 8.1beta3 out soon

2005-10-11 Thread David Fetter
On Tue, Oct 11, 2005 at 08:51:01AM -0400, Andrew Dunstan wrote: > > > Andrew Dunstan wrote: > > >Tom Lane wrote: > > > >>Andrew Dunstan <[EMAIL PROTECTED]> writes: > >> > >>>My take: we should document this better, but it ain't broke so it > >>>don't need fixing, > >>> > >>Actually, my take on y

Re: [HACKERS] Spinlocks and CPU Architectures

2005-10-11 Thread Peter Eisentraut
Tom Lane wrote: > This seems pretty unworkable from a packaging standpoint. Even if > you teach autoconf how to tell which model it's running on, there's > no guarantee that the resulting executables will be used on that same > machine. A number of packages in the video area (and perhaps others)

Re: [HACKERS] Spinlocks and CPU Architectures

2005-10-11 Thread Martijn van Oosterhout
On Tue, Oct 11, 2005 at 11:12:46AM -0400, Tom Lane wrote: > This seems pretty unworkable from a packaging standpoint. Even if you > teach autoconf how to tell which model it's running on, there's no > guarantee that the resulting executables will be used on that same > machine. We would have to m

Re: [HACKERS] Spinlocks and CPU Architectures

2005-10-11 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > The long history of spinlock issues has recently been attacked > significantly by Tom, but I wanted to get a status on this issue before > we release 8.1 I'd still like to do something more with that before we release, but exactly what is TBD. > The concl

Re: [HACKERS] slow IN() clause for many cases

2005-10-11 Thread Jonah H. Harris
Please post an explain analyze on your query with a 20-30 item IN clause so that we can see what plan is being generated. On 10/11/05, Ilia Kantor <[EMAIL PROTECTED]> wrote: When in clause becomes large enough (>20-30 cases),It is much better to use "join" way of processing..I mean,"SELECT * FROM t

Re: [HACKERS] Scan Direction not part of ScanState?

2005-10-11 Thread Tom Lane
Martijn van Oosterhout writes: > I notice that the IndexScan code looks up the scan direction each > invocation by looking up the direction of the plan. Does this mean the > direction can change in the middle of a scan? Is this how MOVE > FORWARD/BACKWARD works? Yes, yes.

Re: [HACKERS] NZ mirror

2005-10-11 Thread Dave Page
-Original Message- From: [EMAIL PROTECTED] on behalf of Andrej Ricnik-Bay Sent: Mon 10/10/2005 11:34 PM To: pgsql-hackers@postgresql.org Subject: [HACKERS] NZ mirror > Not sure whether this is the right place to ask this, but > who is looking after the varied mirrors? The NZ mirror >

Re: [HACKERS] PG 8.1beta3 out soon

2005-10-11 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 > That would probably work, but it would ONLY deal with the issue for > $_TD. In your function $event will still hit this problem. Well, fixing $_TD would pretty much fix all the problems I've been having. As far as "$event", that is in my control a

Re: [HACKERS] PG 8.1beta3 out soon

2005-10-11 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: >>> Actually, my take on your analysis is that there should be a way to get >>> at "use warnings" (I assume that's disallowed in trusted plperl). >> >> Yes, we can't allow "use" in trusted code. But we could turn it on in >> plperl.c, just as we can turn

Re: [HACKERS] slow IN() clause for many cases

2005-10-11 Thread Ilia Kantor
When in clause becomes large enough (>20-30 cases), It is much better to use "join" way of processing.. I mean, "SELECT * FROM table WHERE field IN (1,2...30)" will be slower than "SELECT * FROM table JOIN (SRF returning 1...30) USING(field)" I'm not quite sure, where the difference starts, but

Re: [HACKERS] PG 8.1beta3 out soon

2005-10-11 Thread Andrew Dunstan
Greg Sabino Mullane wrote: Hmm...what if we did this?: Index: plperl.c === RCS file: /projects/cvsroot/pgsql/src/pl/plperl/plperl.c,v retrieving revision 1.92 diff -r1.92 plperl.c 671c671 < XPUSHs(sv_2mortal(newSVpv("my $_

Re: [HACKERS] Spinlocks and CPU Architectures

2005-10-11 Thread Emil Briggs
> > Do other people reach the same conclusions? > > Can we make a list of those architectures for which 8.1 is known to > perform reasonably well, with reasonable SMP scalability? I suggest that > we record this list somewhere in the release notes, but with a comment > to say we run on other archit

Re: [HACKERS] Need A Suggestion

2005-10-11 Thread Ilia Kantor
> Is there a simple, user-accessible mechanism to schedule a function to > be run at query commit ? CONSTRAINT TRIGGER (DEFERRABLE) It is kinda hack, because CONSTRAINT TRIGGERs are not indended for such use, But there are no other "ON COMMIT" triggers in postgresql. -

Re: [HACKERS] relational class vs partitioned table (was

2005-10-11 Thread Simon Riggs
On Mon, 2005-10-10 at 19:58 -0700, Trent Shipley wrote: > Of course, there is no reason a relation in a relational class might not be > huge. Well, as a designer, I would make it so. > Orthoganal partion rules would be created for the class. The rules would be > applied to each member relati

Re: [HACKERS] PG 8.1beta3 out soon

2005-10-11 Thread Andrew Dunstan
Andrew Dunstan wrote: Tom Lane wrote: Andrew Dunstan <[EMAIL PROTECTED]> writes: My take: we should document this better, but it ain't broke so it don't need fixing, Actually, my take on your analysis is that there should be a way to get at "use warnings" (I assume that's disallowed in

Re: [HACKERS] PG 8.1beta3 out soon

2005-10-11 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 NotDashEscaped: You need GnuPG to verify this message > I don't think it's really a bug - it's a well known perl effect that has > caught many people over the years, especially unwary users of > Apache::Registry who fail to recognise that their scrip

[HACKERS] Spinlocks and CPU Architectures

2005-10-11 Thread Simon Riggs
The long history of spinlock issues has recently been attacked significantly by Tom, but I wanted to get a status on this issue before we release 8.1 My understanding of the problems of spinlocking has been greatly enhanced by two recent articles: Linux Journal, discussing linux SMP portability i

[HACKERS] Scan Direction not part of ScanState?

2005-10-11 Thread Martijn van Oosterhout
Hi, I notice that the IndexScan code looks up the scan direction each invocation by looking up the direction of the plan. Does this mean the direction can change in the middle of a scan? Is this how MOVE FORWARD/BACKWARD works? I've traced the code and it *appears* this is what's happening, but c

Re: [HACKERS] Need A Suggestion

2005-10-11 Thread Hannu Krosing
On E, 2005-10-10 at 16:32 -0400, Tom Lane wrote: > "Jonah H. Harris" <[EMAIL PROTECTED]> writes: > > In the past, I've just written a C-based function that calls out to system. > > Use pltclu, plpythonu, or plperlu, according to taste. They all have > pre-existing solutions for this. > > Whether

Re: [HACKERS] slower merge join on sorted data chosen over

2005-10-11 Thread Simon Riggs
On Mon, 2005-10-10 at 15:14 -0500, Kevin Grittner wrote: > We are looking at doing much more with PostgreSQL over the > next two years, and it seems likely that this issue will come up > again where it is more of a problem. It sounded like there was > some agreement on HOW this was to be fixed, ye

Re: [HACKERS] LDAP Authentication?

2005-10-11 Thread Magnus Hagander
> > > > You can do this today using PAM authenication, but this is > > > not always > > > > possible. Notably it's never possible on Windows, and there are > > > > several unix platforms/distros that don't support it > > > without a lot of > > > > work. > > > > > > Or you port PAM to Windows, and

[HACKERS] Implementing Multi-relation/column keyword indices

2005-10-11 Thread Paresh Bafna
For implementing "Multi-relation/column keyword indices", which modules of postgres might be required to change or look into? BTW anyone has clear idea about "Multi-relation/column keyword indices"? Cheers, ---(end of broadcast)--- TIP 4: Have you