Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-06 Thread Tatsuo Ishii
> I'm running this on a Dual-PIII 450 Server, 512Meg of RAM, zero > swap space being used ... the database has its indices on one hard drive, > the tables themselves are on a second one ... its PgSQL 7.0.2 (Tom, > anything in v7.0.3 that might improve this?) and startup is as: > > #!/bin/tc

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread Bruce Momjian
> Bruce Momjian <[EMAIL PROTECTED]> writes: > > Well, I think it would be helpful to catch the most obvious things > > people forget, but if no one thinks its a good idea, I will yank it. > > If you've got an idea *how* to do it in any sort of reliable fashion, > I'm all ears. But it sounds more

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Well, I think it would be helpful to catch the most obvious things > people forget, but if no one thinks its a good idea, I will yank it. If you've got an idea *how* to do it in any sort of reliable fashion, I'm all ears. But it sounds more like pie-in

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread Bruce Momjian
> so, we are gonna have an AI built into the database now too? my > experience to date is that each scenario is different for what can be done > to fix something ... as my last problem shows. I could remove the index, > since it isn't used anywhere else that I'm aware of, or, as philip pointed >

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread The Hermit Hacker
On Sun, 5 Nov 2000, Bruce Momjian wrote: > > Bruce Momjian <[EMAIL PROTECTED]> writes: > > > I am adding a new TODO item: > > > * Add SET PERFORMANCE_TIPS option to suggest INDEX, VACUUM, VACUUM > > > ANALYZE, and CLUSTER > > > Seems we should be able to emit NOTICE messages suggesting perf

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread Bruce Momjian
> Bruce Momjian <[EMAIL PROTECTED]> writes: > > I am adding a new TODO item: > > * Add SET PERFORMANCE_TIPS option to suggest INDEX, VACUUM, VACUUM > > ANALYZE, and CLUSTER > > Seems we should be able to emit NOTICE messages suggesting performance > > improvements. > > This would be tar

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > I am adding a new TODO item: > * Add SET PERFORMANCE_TIPS option to suggest INDEX, VACUUM, VACUUM > ANALYZE, and CLUSTER > Seems we should be able to emit NOTICE messages suggesting performance > improvements. This would be targeted to hel

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread Bruce Momjian
I am adding a new TODO item: * Add SET PERFORMANCE_TIPS option to suggest INDEX, VACUUM, VACUUM ANALYZE, and CLUSTER Seems we should be able to emit NOTICE messages suggesting performance improvements. > Philip Warner <[EMAIL PROTECTED]> writes: > > Could you trick it into n

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread The Hermit Hacker
On Mon, 6 Nov 2000, Philip Warner wrote: > At 23:12 5/11/00 -0400, The Hermit Hacker wrote: > > > >Except, if we are telling it to get rid of using the index, may as well > >get rid of it altogether, as updates/inserts would be slowed down by > >having to update that too ... > > > > So long as y

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread The Hermit Hacker
On Sun, 5 Nov 2000, Tom Lane wrote: > The Hermit Hacker <[EMAIL PROTECTED]> writes: > > On Mon, 6 Nov 2000, Philip Warner wrote: > >> At 21:59 5/11/00 -0500, Tom Lane wrote: > Looks like a great kluge to me ;-) > >> > >> Hmph. I prefer to think of it as a 'user-defined optimizer hint'. ;-}

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread Ron Chmara
The Hermit Hacker wrote: > I'm tryin to figure out how to speed up udmsearch when run under > postgresql, and am being hit by atrocious performance when using a LIKE > query ... the query looks like: > SELECT ndict.url_id,ndict.intag > FROM ndict,url > WHERE ndict.word_id=1971739852 >AND ur

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread Philip Warner
At 23:12 5/11/00 -0400, The Hermit Hacker wrote: > >Except, if we are telling it to get rid of using the index, may as well >get rid of it altogether, as updates/inserts would be slowed down by >having to update that too ... > So long as you don't ever need the index for anything else, then getti

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread Tom Lane
The Hermit Hacker <[EMAIL PROTECTED]> writes: > On Mon, 6 Nov 2000, Philip Warner wrote: >> At 21:59 5/11/00 -0500, Tom Lane wrote: Looks like a great kluge to me ;-) >> >> Hmph. I prefer to think of it as a 'user-defined optimizer hint'. ;-} > Except, if we are telling it to get rid of usi

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread The Hermit Hacker
On Mon, 6 Nov 2000, Philip Warner wrote: > At 21:59 5/11/00 -0500, Tom Lane wrote: > > > >Looks like a great kluge to me ;-) > > > > Hmph. I prefer to think of it as a 'user-defined optimizer hint'. ;-} Except, if we are telling it to get rid of using the index, may as well get rid of it altoge

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread Philip Warner
At 21:59 5/11/00 -0500, Tom Lane wrote: > >Looks like a great kluge to me ;-) > Hmph. I prefer to think of it as a 'user-defined optimizer hint'. ;-} Philip Warner| __---_ Albatross Consulting Pty. Ltd.

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread The Hermit Hacker
yowch ... removing that one index makes my 'test' search (mvcc) come back as: [97366] SQL 0.05s: SELECT ndict.url_id,ndict.intag FROM ndict,url WHERE ndict.word_id=572517542 AND url.rec_id=ndict.url_id AND (url.url LIKE 'http://www.postgresql.org/%') vs what we were doing before ... now, let

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread Tom Lane
Philip Warner <[EMAIL PROTECTED]> writes: > Could you trick it into not using the index (AND using the other strategy?) > by using a calculation: >AND ( (url.url || ' ') LIKE 'http://www.postgresql.org/% '); > it's a bit nasty. Looks like a great kluge to me ;-) reg

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread Bruce Momjian
Yes, I am waiting to hear back on that. > At 21:47 5/11/00 -0500, Tom Lane wrote: > >It's ye same olde nonuniform-distribution > >problem; until we have better statistics, there's not much hope > >for a non-kluge solution. > > Wasn't somebody trying to do something with that a few weeks back? >

Re: [HACKERS] How to get around LIKE inefficiencies?]

2000-11-05 Thread Bruce Momjian
> Bruce Momjian <[EMAIL PROTECTED]> writes: > > Sorry to be getting in here late. Have you tried CLUSTER? > > Prolly won't help much. I think what he's getting burnt by > is that the planner thinks that an indexscan based on the > LIKE 'http://www.postgresql.org/%' condition will be extremely >

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread Philip Warner
At 21:47 5/11/00 -0500, Tom Lane wrote: >It's ye same olde nonuniform-distribution >problem; until we have better statistics, there's not much hope >for a non-kluge solution. Wasn't somebody trying to do something with that a few weeks back? -

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread Philip Warner
At 21:28 5/11/00 -0500, Tom Lane wrote: >A brute-force answer would be to remove the url_url index ;-) >dunno if that would slow down other queries, however. Could you trick it into not using the index (AND using the other strategy?) by using a calculation: SELECT ndict.url_id,ndict.intag FRO

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Sorry to be getting in here late. Have you tried CLUSTER? Prolly won't help much. I think what he's getting burnt by is that the planner thinks that an indexscan based on the LIKE 'http://www.postgresql.org/%' condition will be extremely selective ---

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread Bruce Momjian
Sorry to be getting in here late. Have you tried CLUSTER? If it is using an index scan, and it is slow, cluster often helps, especially when there are several duplicate matches, as there is with LIKE. Let me know how that works. > A brute-force answer would be to remove the url_url index ;-) >

Re: [HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread Tom Lane
A brute-force answer would be to remove the url_url index ;-) dunno if that would slow down other queries, however. regards, tom lane

[HACKERS] How to get around LIKE inefficiencies?

2000-11-05 Thread The Hermit Hacker
I'm tryin to figure out how to speed up udmsearch when run under postgresql, and am being hit by atrocious performance when using a LIKE query ... the query looks like: SELECT ndict.url_id,ndict.intag FROM ndict,url WHERE ndict.word_id=1971739852 AND url.rec_id=ndict.url_id AND (ur