Re: [PATCHES] [HACKERS] Index creation takes for ever

2004-03-17 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Tom Lane wrote: > >> Where was it posted anyway? > > > Found it: > > > > > http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&selm=200312010450.hB14ovH16330%40candle.pha.pa.us&rnum=8 > > Thanks. The original patch is much olde

Re: [PATCHES] [HACKERS] Index creation takes for ever

2004-03-17 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Where was it posted anyway? > Found it: > > http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&selm=200312010450.hB14ovH16330%40candle.pha.pa.us&rnum=8 Thanks. The original patch is much older than I thought --- I was looking

Re: [PATCHES] [HACKERS] Index creation takes for ever

2004-03-16 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > If I remember correctly, you didn't like the index routines reading the > > tuple information, or something like that, but there was a performance > > benefit for duplicate keys, so I think we should re-investigate this. > > I don't s

Re: [PATCHES] [HACKERS] Index creation takes for ever

2004-03-16 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > If I remember correctly, you didn't like the index routines reading the > tuple information, or something like that, but there was a performance > benefit for duplicate keys, so I think we should re-investigate this. I don't see the actual patch either i

Re: [PATCHES] [HACKERS] Index creation takes for ever

2004-03-16 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Where are we on this? It seems like a win to me. > > I thought it was a bad idea, although I no longer remember the details. If I remember correctly, you didn't like the index routines reading the tuple information, or something li

Re: [PATCHES] [HACKERS] Index creation takes for ever

2004-03-16 Thread Bruce Momjian
Here is more detail on the patch. --- Manfred Koizar wrote: > On Mon, 1 Dec 2003 00:02:54 -0500 (EST), Bruce Momjian > <[EMAIL PROTECTED]> wrote: > >Tom Lane wrote: > >> Bruce Momjian <[EMAIL PROTECTED]> writes: > >> >> An

Re: [PATCHES] [HACKERS] Index creation takes for ever

2003-12-02 Thread Manfred Koizar
On Mon, 01 Dec 2003 13:32:10 -0500, Tom Lane <[EMAIL PROTECTED]> wrote: >Manfred Koizar <[EMAIL PROTECTED]> writes: >> comparetup_index() compares two IndexTuples. The structure >> IndexTupleData consists basically of not much more than an ItemPointer, >> and the patch is not much more than adding

Re: [PATCHES] [HACKERS] Index creation takes for ever

2003-12-01 Thread Tom Lane
Manfred Koizar <[EMAIL PROTECTED]> writes: > comparetup_index() compares two IndexTuples. The structure > IndexTupleData consists basically of not much more than an ItemPointer, > and the patch is not much more than adding a comparison of two > ItemPointers. So how does the patch introduce a new

Re: [PATCHES] [HACKERS] Index creation takes for ever

2003-12-01 Thread Manfred Koizar
On Mon, 1 Dec 2003 00:02:54 -0500 (EST), Bruce Momjian <[EMAIL PROTECTED]> wrote: >Tom Lane wrote: >> Bruce Momjian <[EMAIL PROTECTED]> writes: >> >> And if it doesn't help index >> >> creation speed, at least the resulting index has better correlation. ... which has been shown by the example in

Re: [PATCHES] [HACKERS] Index creation takes for ever

2003-11-30 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > >> If qsort is to blame, then maybe this patch could help. It sorts > >> equal key values on item pointer. And if it doesn't help index > >> creation speed, at least the resulting index has better correlation. > > > I will try to appl

Re: [PATCHES] [HACKERS] Index creation takes for ever

2003-11-30 Thread Bruce Momjian
Patch removed from queue. --- Manfred Koizar wrote: > On Mon, 01 Sep 2003 08:46:09 -0400, Tom Lane <[EMAIL PROTECTED]> > wrote: > >[EMAIL PROTECTED] writes: > >> it took 69 minutes to finish, 75% of this time was devoted to

Re: [PATCHES] [HACKERS] Index creation takes for ever

2003-11-30 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: >> If qsort is to blame, then maybe this patch could help. It sorts >> equal key values on item pointer. And if it doesn't help index >> creation speed, at least the resulting index has better correlation. > I will try to apply it within the next 48 hour

Re: [PATCHES] [HACKERS] Index creation takes for ever

2003-11-30 Thread Bruce Momjian
Your patch has been added to the PostgreSQL unapplied patches list at: http://momjian.postgresql.org/cgi-bin/pgpatches I will try to apply it within the next 48 hours. --- Manfred Koizar wrote: > On Mon, 01 Sep 20

Re: [HACKERS] Index creation takes for ever

2003-09-08 Thread Manfred Koizar
On Mon, 8 Sep 2003 11:31:05 +0200, "Zeugswetter Andreas SB SD" <[EMAIL PROTECTED]> wrote: >> As Tom mentioned, we might not want to keep the tid's in order after the >> index is created because he wants the most recent tid's first, so the >> expired ones migrate to the end. > >But on average this a

Re: [HACKERS] Index creation takes for ever

2003-09-08 Thread Zeugswetter Andreas SB SD
> > I don't think so, because the patch does nothing to keep the sort > > order once the index is initially created. > > As Tom mentioned, we might not want to keep the tid's in order after the > index is created because he wants the most recent tid's first, so the > expired ones migrate to the e

Re: [HACKERS] Index creation takes for ever

2003-09-07 Thread Bruce Momjian
Manfred Koizar wrote: > On Sun, 7 Sep 2003 11:43:42 -0400 (EDT), Bruce Momjian > <[EMAIL PROTECTED]> wrote: > >I assume this completes this TODO: > > > > * Order duplicate index entries by tid for faster heap lookups > > I don't think so, because the patch does nothing to keep the sort > order

Re: [PATCHES] [HACKERS] Index creation takes for ever

2003-09-07 Thread Manfred Koizar
On Sun, 07 Sep 2003 12:23:28 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >Maybe so, but it would degrade the performance in the unique-index case >if we do it as the TODO is worded. The patch would only hurt with a unique index, if there are lots of duplicate tuples at CREATE INDEX time. >My own o

Re: [HACKERS] Index creation takes for ever

2003-09-07 Thread Manfred Koizar
On Sun, 7 Sep 2003 11:43:42 -0400 (EDT), Bruce Momjian <[EMAIL PROTECTED]> wrote: >I assume this completes this TODO: > > * Order duplicate index entries by tid for faster heap lookups I don't think so, because the patch does nothing to keep the sort order once the index is initially created

Re: [PATCHES] [HACKERS] Index creation takes for ever

2003-09-07 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Tom Lane wrote: > >>> * Order duplicate index entries by tid for faster heap lookups > > > >> I don't know why that TODO entry exists, but I think the idea is > >> counterproductive. > > > I assume you are talking about a unique index

Re: [PATCHES] [HACKERS] Index creation takes for ever

2003-09-07 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >>> * Order duplicate index entries by tid for faster heap lookups > >> I don't know why that TODO entry exists, but I think the idea is >> counterproductive. > I assume you are talking about a unique index that probably only has a > few

Re: [PATCHES] [HACKERS] Index creation takes for ever

2003-09-07 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > I assume this completes this TODO: > > * Order duplicate index entries by tid for faster heap lookups > > I don't know why that TODO entry exists, but I think the idea is > counterproductive. The existing btree code will tend to

Re: [HACKERS] Index creation takes for ever

2003-09-07 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > I assume this completes this TODO: > * Order duplicate index entries by tid for faster heap lookups I don't know why that TODO entry exists, but I think the idea is counterproductive. The existing btree code will tend to put newer versions of a ro

Re: [HACKERS] Index creation takes for ever

2003-09-07 Thread Bruce Momjian
I assume this completes this TODO: * Order duplicate index entries by tid for faster heap lookups and you will submit it for 7.5? If you want to post it now, I can get it into the 7.5 queue so we don't forget it. -

Re: [HACKERS] Index creation takes for ever

2003-09-07 Thread Manfred Koizar
On Mon, 01 Sep 2003 08:46:09 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >[EMAIL PROTECTED] writes: >> it took 69 minutes to finish, 75% of this time was devoted to create 2 >> indexes on varchar(2) with value being 'O', 'N' or null; > >I still say it's either strcoll or qsort's fault. If qsort is

Re: [HACKERS] Index creation takes for ever

2003-09-01 Thread Tom Lane
[EMAIL PROTECTED] writes: > I've made some tests with your qsort and it DEFINITIVLY help > ~3 mn instead of 69. > However this is for 7.3.4 I've got no probs with 7.4b. > Did something change in btree creation? Hmm, I wouldn't have thought so, but perhaps we did change something that would affect

Re: [HACKERS] Index creation takes for ever

2003-09-01 Thread ohp
lt;[EMAIL PROTECTED]> > To: [EMAIL PROTECTED] > Cc: pgsql-hackers list <[EMAIL PROTECTED]> > Subject: Re: [HACKERS] Index creation takes for ever > > [EMAIL PROTECTED] writes: > > it took 69 minutes to finish, 75% of this time was devoted to create 2 > > indexes

Re: [HACKERS] Index creation takes for ever

2003-09-01 Thread Tom Lane
[EMAIL PROTECTED] writes: > it took 69 minutes to finish, 75% of this time was devoted to create 2 > indexes on varchar(2) with value being 'O', 'N' or null; > I wonder if it's a configuration matter. I still say it's either strcoll or qsort's fault. Try swapping in our own version of qsort to se

Re: [HACKERS] Index creation takes for ever

2003-09-01 Thread ohp
tion took 100% of 1 CPU. Regards On Thu, 28 Aug 2003, Tom Lane wrote: > Date: Thu, 28 Aug 2003 10:13:21 -0400 > From: Tom Lane <[EMAIL PROTECTED]> > To: [EMAIL PROTECTED] > Cc: pgsql-hackers list <[EMAIL PROTECTED]> > Subject: Re: [HACKERS] Index creation takes fo

Re: [HACKERS] Index creation takes for ever

2003-08-30 Thread ohp
omjian <[EMAIL PROTECTED]> > To: Tom Lane <[EMAIL PROTECTED]> > Cc: [EMAIL PROTECTED], pgsql-hackers list <[EMAIL PROTECTED]> > Subject: Re: [HACKERS] Index creation takes for ever > > Tom Lane wrote: > > [EMAIL PROTECTED] writes: > > > This is on unixware 7

Re: [HACKERS] Index creation takes for ever

2003-08-30 Thread Bruce Momjian
Tom Lane wrote: > [EMAIL PROTECTED] writes: > > This is on unixware 7 (both 7.3.4 and 7.4b) > > > I'm on the FR language (I'll re-initdb whith lang=C to see what happens) > > Okay. If you find it's still slow in C locale, the next thing to try > would be forcing use of our own qsort, as we alrea

Re: [HACKERS] Index creation takes for ever

2003-08-28 Thread ohp
-0400 > From: Tom Lane <[EMAIL PROTECTED]> > To: [EMAIL PROTECTED] > Cc: pgsql-hackers list <[EMAIL PROTECTED]> > Subject: Re: [HACKERS] Index creation takes for ever > > [EMAIL PROTECTED] writes: > > I've reinitdb (on 7.4b) with LANG=C and it worked. >

Re: [HACKERS] Index creation takes for ever

2003-08-28 Thread Tom Lane
[EMAIL PROTECTED] writes: > I've reinitdb (on 7.4b) with LANG=C and it worked. > So I reinitDB with LANG=FR and used LANG=C to psql -f xxx.sql template1 to > recreate the db and it worked too... That's weird. I don't understand why an initdb in the same locale would make the problem go away. > I

Re: [HACKERS] Index creation takes for ever

2003-08-28 Thread ohp
-hackers list <[EMAIL PROTECTED]> > Subject: Re: [HACKERS] Index creation takes for ever > > [EMAIL PROTECTED] writes: > > This is on unixware 7 (both 7.3.4 and 7.4b) > > > I'm on the FR language (I'll re-initdb whith lang=C to see what happens) > > Okay.

Re: [HACKERS] Index creation takes for ever

2003-08-28 Thread Tom Lane
[EMAIL PROTECTED] writes: > This is on unixware 7 (both 7.3.4 and 7.4b) > I'm on the FR language (I'll re-initdb whith lang=C to see what happens) Okay. If you find it's still slow in C locale, the next thing to try would be forcing use of our own qsort, as we already do for Solaris. You'd need

Re: [HACKERS] Index creation takes for ever

2003-08-28 Thread ohp
ll process took only 5 Min. On Thu, 28 Aug 2003, Tom Lane wrote: > Date: Thu, 28 Aug 2003 10:13:21 -0400 > From: Tom Lane <[EMAIL PROTECTED]> > To: [EMAIL PROTECTED] > Cc: pgsql-hackers list <[EMAIL PROTECTED]> > Subject: Re: [HACKERS] Index creation takes for ever > &

Re: [HACKERS] Index creation takes for ever

2003-08-28 Thread Tom Lane
[EMAIL PROTECTED] writes: > I've then pg_dump'ed the database and recreate an other both on 7.3.4 and > 7.4b > Both are still running after more than 30 minutes of CPU (100% cpu taken) > creating the levt_lu_ligne_evt_key. That's hard to believe. I get regression=# SELECT levt_lu,count(*) from

[HACKERS] Index creation takes for ever

2003-08-28 Thread ohp
Hi every one, I've tried to reindex one of my customer's table to gain some disk space. I had to stop after 90 m cpu... I've then pg_dump'ed the database and recreate an other both on 7.3.4 and 7.4b Both are still running after more than 30 minutes of CPU (100% cpu taken) creating the levt_lu_l