Re: Poor query optimizer choices is making Derby unusable for large tables

2005-11-14 Thread Kevin Hore
Hi Jeffrey, Thank you for your response. I think you are probably right about Derby not having a strategy for doing multiple scans. I think DERBY-47 probably covers this, but I'll perhaps add a note to that covering my circumstances. I had thought of using UNION but, unfortunately the IN clause

Re: Poor query optimizer choices is making Derby unusable for large tables

2005-11-14 Thread Kevin Hore
Hi Mamta, Thank you for your suggestion, but this is for a production system and so, as you suspected, we don't really want to use an alpha version of Derby. Kind regards, Kevin Hore Mamta Satoor wrote: Hi Kevin, I haven't investigated Derby-47 to know how to fix the problem but I do have an

Re: Poor query optimizer choices is making Derby unusable for large tables

2005-11-14 Thread Kevin Hore
Hi Satheesh Satheesh Bandaram wrote: Hi Kevin, Kevin Hore wrote: i) Does anyone have any plans to fix this problem? Can you file an enhancement request for this? I think Derby could improve it's handling of OR/IN clauses. Many databases don't optimize OR clauses as much as possible, thoug

Re: Poor query optimizer choices is making Derby unusable for large tables

2005-11-14 Thread Kevin Hore
Hi Rick, Thanks for your suggested re-write, but I'm really looking for a single query. Unfortunately, we have quite a number of queries affected by this and it would be significant work to re-written them all in this way. In addition, some are considerably more complex than the example I gave, a

Re: Poor query optimizer choices is making Derby unusable for large tables

2005-11-14 Thread Kevin Hore
Hi Michael, Thank you for yours suggestions. You've obviously very kindly given it quite a bit a thought. I'd already tried a bunch of the things that you have suggested. Defining a composite key and removing the existing indexes did nothing to improve performance. Re-writing using IN clause pro

Re: Poor query optimizer choices is making Derby unusable for large tables

2005-11-14 Thread Michael Segel
On Saturday 12 November 2005 16:04, Michael Segel wrote: > On Saturday 12 November 2005 10:28, Michael Segel wrote: > > CREATE TABLE tblSearchDictionary > > ( > > objectId INT NOT NULL, > > objectType INT NOT NULL, > > wordLocation

Re: Poor query optimizer choices is making Derby unusable for large tables

2005-11-12 Thread Michael Segel
On Saturday 12 November 2005 10:28, Michael Segel wrote: > CREATE TABLE tblSearchDictionary > ( >objectId INT NOT NULL, >objectType INT NOT NULL, >wordLocation INT NOT NULL, >word VARCHAR(64)

Re: Poor query optimizer choices is making Derby unusable for large tables

2005-11-12 Thread Michael Segel
On Friday 11 November 2005 08:59, Kevin Hore wrote: Hi, Before complaining about the query optimizer, lets look at your design first. Yeah, I know that Derby, like every other database is not perfect and could always use improvements. But the reality is that the first place to look for improveme

Re: Poor query optimizer choices is making Derby unusable for large tables

2005-11-11 Thread Rick Hillegas
Hi Kevin, You might also try using a temporary table to split your scan up into a series of optimzable queries. Regards-Rick declare global temporary table session.accumulator ( ObjectId int NOT NULL, WordLocation int NOT NULL ) not logged; insert into session.accumulator SELECT Object

Re: Poor query optimizer choices is making Derby unusable for large tables

2005-11-11 Thread Satheesh Bandaram
Hi Kevin, Kevin Hore wrote: > i) Does anyone have any plans to fix this problem? Can you file an enhancement request for this? I think Derby could improve it's handling of OR/IN clauses. Many databases don't optimize OR clauses as much as possible, though some do better than others. It would be

Re: Poor query optimizer choices is making Derby unusable for large tables

2005-11-11 Thread Jeffrey Lichtman
I've described the problem in detail below, and I'd appreciate any assistance. Specifically: i) Does anyone have any plans to fix this problem? I believe the real problem is that Derby doesn't have any strategy for doing multiple scans for OR/IN clauses. This is a useful feature, but I don

Re: Poor query optimizer choices is making Derby unusable for large tables

2005-11-11 Thread Mamta Satoor
Hi Kevin,   I haven't investigated Derby-47 to know how to fix the problem but I do have an optimizer overrides patch waiting for review on the derby developer list which will let user specify their own optimizer properties to help the optimizer pick a specific plan. The JIRA entry for optimizer ov

Poor query optimizer choices is making Derby unusable for large tables

2005-11-11 Thread Kevin Hore
The Derby query optimizer (this is with 10.1.1.0) decides to perform an expensive table scan in certain circumstances, when it could make use of available indexes. The resulting poor performance is rendering Derby useless for our application. I believe that this behaviour might be related to D