Re: [rt-users] RT 3.6.4 poor query performance

2008-03-25 Thread Richard Ellis
Hi Ruslan, I have changed those indexes this morning and will monitor the results today. I created the second new index as CGM2 to prevent the system complaining about duplicate keys. Thanks Richard Ruslan Zakirov wrote: 4 seconds is still slow, but better than 100-400. About your indexes

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-25 Thread Thomas Hecker
this helps. >> Torsten >> >> >> >> Kühne + Nagel (AG & Co.) KG, Geschäftsleitung: Hans-Georg Brinkmann (Vors.), >> Uwe Bielang (Stellv.), Bruno Mang, Alfred Manke, Thorsten Meincke, Mark >> Reinhardt (Stellv.), Jens Wollesen, Rainer Wunn, Sit

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-20 Thread Ruslan Zakirov
4 seconds is still slow, but better than 100-400. About your indexes. You can and I really suggest to delete the following indexes on CGM table: * DROP INDEX GrouMem ON CachedGroupMembers; * DROP INDEX group1 ON CachedGroupMembers; * DROP INDEX member1 ON CachedGroupMembers; And instead create in

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-19 Thread Jeff Voskamp
Jesse Vincent wrote: > > On Wed, Mar 19, 2008 at 07:42:30PM +0300, Ruslan Zakirov wrote: > >> Jesse, I know that they both have index on CachedGroupMembers table >> that starts from 'MemberId' column. And it does mess up optimizer and >> doesn't matter if it's one column or multiple like in (Mem

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-19 Thread Jeff Voskamp
Ruslan Zakirov wrote: > Jeff, always Cc the list. > > Version of your mysql server? > > As far as I can see you suffer from mysql bug, output from your server > is equal in both cases what is really wrong and mysql must use new > index in those test queries I sent to the list. > > There are several

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-19 Thread Richard Ellis
Hi Ruslan, You are a genius. Response time for the Query Builder is now back to 4 seconds which is good enough for me :0. Thanks to all your team for all the efforts to work out what was wrong. Thanks Richard Ruslan Zakirov wrote: Hey, Rechard, the latest results suggest me that we've sad

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-19 Thread Ruslan Zakirov
Jeff, always Cc the list. Version of your mysql server? As far as I can see you suffer from mysql bug, output from your server is equal in both cases what is really wrong and mysql must use new index in those test queries I sent to the list. There are several options: 1) Delete any indexes on Ca

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-19 Thread Ruslan Zakirov
Hey, Rechard, the latest results suggest me that we've saddled this beast :) at least that what explain says and I hope it's correct. You can check that query again and it should be fast. Wanna try? You can use SELECT SQL_NO_CACHE ... to make sure it's reproducible and is not cache hit. On Wed

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-19 Thread Jesse Vincent
On Wed, Mar 19, 2008 at 07:42:30PM +0300, Ruslan Zakirov wrote: > Jesse, I know that they both have index on CachedGroupMembers table > that starts from 'MemberId' column. And it does mess up optimizer and > doesn't matter if it's one column or multiple like in (MemberId, > GroupId, Disabled) in

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-19 Thread Ruslan Zakirov
Jesse, I know that they both have index on CachedGroupMembers table that starts from 'MemberId' column. And it does mess up optimizer and doesn't matter if it's one column or multiple like in (MemberId, GroupId, Disabled) index (Jeff created such thing). We really need such index in the core on CGM

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-19 Thread Jesse Vincent
On Wed, Mar 19, 2008 at 04:38:12PM +, Richard Ellis wrote: > Hi Jesse, > > Thanks. To the best of my knowledge nobody has added any indexes to the > database on anything except what RT patches apply on each upgrade. This > DB was originally 3.0 and has been upgraded more times than I want

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-19 Thread Richard Ellis
Hi Jesse, Thanks. To the best of my knowledge nobody has added any indexes to the database on anything except what RT patches apply on each upgrade. This DB was originally 3.0 and has been upgraded more times than I want to think about over the years to 3.6.6 now :) Richard Jesse Vincent wro

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-19 Thread Jesse Vincent
On Wed, Mar 19, 2008 at 04:22:46PM +, Richard Ellis wrote: > Hi Ruslan, > > here's the two sets of results. FWIW, from your response to ruslan, it _does_ look like your hand-added "group1" index was messing up the query planner. It's on GroupId, while we already had an index on GroupId, Me

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-19 Thread Richard Ellis
Hi Ruslan, here's the two sets of results. Thanks Richard Ruslan Zakirov wrote: Ok, I have an idea how to fix that problem Here is new file for testing that will give me more info to find the best way to fixing this. We're really close. You can run it using: mysql -t -u root -ppassword rt3

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-19 Thread Ruslan Zakirov
Ok, I have an idea how to fix that problem Here is new file for testing that will give me more info to find the best way to fixing this. We're really close. You can run it using: mysql -t -u root -ppassword rt3 <../search_possible_owners.mysql.sql >test.res As a first step to fix it you can crea

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-19 Thread Mathew
The vast amount of data could be due to spam. I set up a script which regularly (daily) searches for spam and the users automatically created by it. It then runs Shredder on the list I built. This is made easier by creating a spam only queue into which all incoming spam is placed. Additional

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-18 Thread Ruslan Zakirov
Hello, Richard. Jesse asked me to look into this problem. Could you please run SQL commands from the attachment. It's a list of COUNT and EXPLAIN queries. I hope that will help us identify problems. On Tue, Mar 18, 2008 at 7:31 PM, Richard Ellis <[EMAIL PROTECTED]> wrote: > > Hi Jesse, > > We a

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-18 Thread Jesse Vincent
[Please keep ccing the list. Free support is a lot more valuable when the community's involved] What version of mysql is this again? My explain output looks like this: +--++- +-+-+---+--

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-18 Thread Jesse Vincent
So, you have a query that ran for 400+ seconds an examined fifteen million rows. That seems..wrong. What does this say? EXPLAIN SELECT DISTINCT main.* FROM Users main CROSS JOIN ACL ACL_4 JOIN Principals Principals_1 ON ( Principals_1.id = main.id ) JOIN CachedGroupMembers CachedGroupMembers_2

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-17 Thread Kenneth Crocker
Mathew, You mentioned needing "SeeQueue" for "Unprivileged" users and you already have it for "Privileged" users. Why not grant "CreateTicket", "SeeQueue" Globally for "Everyone" (saves redundant granting) and "ShowTicket" Globally for "Requestors"? That way any requestor can always "

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-17 Thread Richard Ellis
Hi Jesse, That's worrying because according to the master list of users I have, their should only be 88 users who have permissions to own tickets. Thats probably not the cause of the performance issue though, so I'll shelve that for now to look at later. I'll turn slow queries on now and sen

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-17 Thread Jesse Vincent
On Mar 17, 2008, at 12:26 PM, Richard Ellis wrote: mysql> SELECT * from ACL, CachedGroupMembers, Groups where ACL.RightName = 'OwnTicket' and ACL.PrincipalId = Groups.id and Groups.id = CachedGroupMembers.GroupId; Ok. Those results tell me there should be 290 names in your "SelectOwner" d

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-17 Thread Jesse Vincent
On Mon, Mar 17, 2008 at 04:08:52PM +, Richard Ellis wrote: > ok, that doesn't look good > > > mysql> select * from ACL, CachedGroupMembers, Groups where ACL.RightName > = 'OwnTicket' and ACL.PrincipalId = Groups.id and Groups.id = > CachedGroupMembers.GroupId and (CachedGroupMembers.Memb

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-17 Thread Jesse Vincent
On Mar 17, 2008, at 11:48 AM, Richard Ellis wrote: Hi Jesse, Ok. Next up: select * from ACL, CachedGroupMembers, Groups where ACL.RightName = 'OwnTicket' and ACL.PrincipalId = Groups.id and Groups.id = CachedGroupMembers.GroupId and (CachedGroupMembers.MemberId = '3' or CachedGroupMem

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-17 Thread Jesse Vincent
On Mar 17, 2008, at 8:29 AM, Richard Ellis wrote: Hi Guys, I'm seeing the same thing. A lot more users in that dropdown than should be there. Richard, I'd like to help get to the bottom of this. Can you send the output of: SELECT * from ACL where RightName = 'OwnTicket' PGP.sig D

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-17 Thread Richard Ellis
Hi Guys, I'm seeing the same thing. A lot more users in that dropdown than should be there. I'll do the quick and dirty hack for now to get this working again. Thanks Richard Mathew wrote: We have Everyone: CreateTicket on two public facing queues Privileged: CommentOnTicket, CreateTick

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-17 Thread Mathew
We have Everyone: CreateTicket on two public facing queues Privileged: CommentOnTicket, CreateTicket, SeeQueue, ShowOutgoingEmail, ShowTicket, ShowTicketComments, Watch on all queues Unprivileged: CreateTicket, SeeQueue on one public facing queue (now that I think about it CreateTicket here se

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-17 Thread Ham MI-ID, Torsten Brumm
üngliche Nachricht- Von: Mathew [mailto:[EMAIL PROTECTED] Gesendet: Montag, 17. März 2008 12:20 An: Ham MI-ID, Torsten Brumm Cc: Richard Ellis; rt-users@lists.bestpractical.com Betreff: Re: AW: [rt-users] RT 3.6.4 poor query performance You shouldn't have had to write a patch to fix the imme

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-17 Thread Mathew
Contern/Luxemburg > Geschäftsführender Verwaltungsrat: Klaus-Michael Kühne > > > -Ursprüngliche Nachricht- > Von: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Im Auftrag von Mathew > Gesendet: Montag, 17. März 2008 11:06 > An: Richard Ellis > Cc: rt-users@lists.be

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-17 Thread Ham MI-ID, Torsten Brumm
us-Michael Kühne -Ursprüngliche Nachricht- Von: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Im Auftrag von Mathew Gesendet: Montag, 17. März 2008 11:06 An: Richard Ellis Cc: rt-users@lists.bestpractical.com Betreff: Re: [rt-users] RT 3.6.4 poor query performance Well, in that case, I reco

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-17 Thread Mathew
Well, in that case, I recommend witchcraft. ;) Mathew Richard Ellis wrote: > Hi All, > > We have upgraded to 3.6.6 over the weekend and also run some > optimisation against the database but performance is still very poor. > > I have looked at RTx::RightMatrix and Everyone definately does not h

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-17 Thread Richard Ellis
ery performance (Jesse Vincent) 2. Re: RT 3.6.4 poor query performance (Toby Darling) -- Message: 1 Date: Thu, 13 Mar 2008 11:45:53 -0400 From: Jesse Vincent <[EMAIL PROTECTED]> Subject: Re: [rt-users] RT 3.6.4 poor

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-13 Thread Graham Dunn
Toby Darling wrote: > Hi Richard > >> That crashes out with a divide by zero error in the scripting, but it >> looks really cool. >> >> [--] Status: +Archive -BDB -Federated +InnoDB -ISAM -NDBCluster >> [--] Data in MyISAM tables: 2M (Tables: 39) >> [--] Data in InnoDB tables: 1015M (Tables: 20)

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-13 Thread Toby Darling
Hi Richard > That crashes out with a divide by zero error in the scripting, but it > looks really cool. > > [--] Status: +Archive -BDB -Federated +InnoDB -ISAM -NDBCluster > [--] Data in MyISAM tables: 2M (Tables: 39) > [--] Data in InnoDB tables: 1015M (Tables: 20) > Use of uninitialized value

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-13 Thread Jesse Vincent
On Mar 13, 2008, at 8:46 AM, Richard Ellis wrote: Hi All, We have recently updated our RT instance from 3.4.6 running on Solaris 9 to a new server running Solaris 10 and 3.6.4. You probably want RT 3.6.6 if you're on MySQL. Ruz did some serious query optimization. But my first guess i

Re: [rt-users] RT 3.6.4 poor query performance

2008-03-13 Thread Toby Darling
Let mysqltuner (http://rackerhacker.com/mysqltuner/) have a look at your database. Lots of useful stuff. We went from 180 second full text queries, to 7 seconds, just be tweaking innodb_buffer_pool_size. Cheers Toby Richard Ellis wrote: > Hi All, > > We have recently updated our RT instance fr

[rt-users] RT 3.6.4 poor query performance

2008-03-13 Thread Richard Ellis
Hi All, We have recently updated our RT instance from 3.4.6 running on Solaris 9 to a new server running Solaris 10 and 3.6.4. Everything works perfectly and we have removed all of the customisations that we used to use for a virtually vanilla 3.6.4 install. However, when opening the Query Bui