RE: Need help optimizing query
Hello Victor, > What version of MySQL are you using? Have you > checked the cardinality on > these tables? Problem solved! Optimizing the table brought the query time down to 17 secs Wow! Thanks for the input Victor and merci to Arnaud for the quick fix. Patrick __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
RE: Need help optimizing query
What version of MySQL are you using? Have you checked the cardinality on these tables? -Original Message- From: Patrick Drouin To: [EMAIL PROTECTED] Sent: 7/15/04 8:27 AM Subject: Need help optimizing query Hello everyone, I'm having a hard time with the following query. It retrieves about 3K rows from a few tables. One of them contains over 40M rows. When run on a 3Ghz server with 1G of RAM it returns the rows in more than 1 mini. I don't think that's normal. Here's the output of EXPLAIN: mysql> explain SELECT ti.posi, ti.docid, d.filename, ti.id, c.name FROM corpus_documents cd, corpus c, documents d, tokens_ins ti, tokens t WHERE c.corpusid=4 AND cd.corpusid=c.corpusid AND cd.docid=d.docid AND t.docid=d.docid AND ti.id=t.id AND t.word='police' AND t.docid=ti.docid\g; He +---++-- +---+-+---+--+-- +| table | type | possible_keys | key | key_len | ref | rows | Extra |+---++- -+---+-+---+--+- -+| c | ref | PRIMARY,corpus_corpusid | PRIMARY | 3 | const | 1 | Using where; Using index || t | ref | PRIMARY,tokens_id,tokens_docid,tokens_word,tokens_word_docid | PRIMARY | 30 | const | 24 | Using where || ti | ref | PRIMARY,tokens_ins_id,tokens_ins_docid | tokens_ins_id | 4 | t.id | 96 | Using where || d | ref | PRIMARY,documents_docid | PRIMARY | 3 | t.docid | 3 | Using index || cd | eq_ref | PRIMARY | PRIMARY | 6 | const,d.docid | 1 | Using where; Using index |+---++- -+---+-+---+--+- -+ 5 rows in set (0.00 sec) It seems to be using indexes as expected and it does not seem to look at that many rows. Here's tthe query chewed up and nicely displayed: SELECT ti.posi, ti.docid, d.filename, ti.id, c.name FROM corpus_documents cd, corpus c, documents d, tokens_ins ti, tokens t WHERE c.corpusid=4 AND cd.corpusid=c.corpusid AND cd.docid=d.docid AND t.docid=d.docid AND ti.id=t.id AND t.word='police' AND t.docid=ti.docid; ... 3791 rows in set (1 min 29.78 sec) Here are descriptions of the tables at play : mysql> desc tokens_ins; +--+---+--+-+-+---+ | Field | Type | Null | Key | Default | Extra | +--+---+--+-+-+---+ | id | int(11) | | MUL | 0 | | | posi | int(11) | | PRI | 0 | | | docid | mediumint(20) | | PRI | 0 | | | originalspelling | varchar(30) | | | | | +--+---+--+-+-+---+ 4 rows in set (0.02 sec) mysql> desc tokens; +-++--+-+-+---+ | Field | Type | Null | Key | Default | Extra | +-++--+-+-+---+ | id | int(11) | | MUL | 0 | | | docid | mediumint(20) | | PRI | 0 | | | word | varchar(30) binary | | PRI | | | | pos | varchar(10) | | PRI | 0 | | | absfreq | mediumint(20) | | MUL | 0 | | +-++--+-+-+---+ 5 rows in set (0.00 sec) mysql> desc corpus; +--+---+--+-+-++ | Field | Type | Null | Key | Default | Extra | +--+---+--+-+-++ | corpusid | mediumint(20) | | PRI | NULL | auto_increment | | name | varchar(30) | | PRI |+-+-+--+-+-+---+ | Field | Type | Null | Key | Default | Extra | +-+-+--+-+-+---+ | docid | mediumint(20) | | PRI | 0 | | | filename | varchar(30) | | PRI | | | | language | char(3) | | | | | | description | varchar(255) binary | YES | | NULL | | +-+-+--+-+-+---+ 4 rows in set (0.00 sec) mysql> desc corpus_documents; +--+---+--+-+-+---+ | Field | Type | Null | Key | Default | Extra | +--+---+--+-+-+---+ | corpusid | mediumint(20) | | PRI | 0 | | | docid | mediumint(20) | | PRI | 0 | | +--+---+--+-+-+---+ 2 rows in set (0.00 sec) Can anybody give me a hand speeding up this ting? I'm running out of ideas. Thanks, P | | | language | char(3) | | MUL | | | +--+---+--+-+-++ 3 rows in set (0.00 sec) mysql> desc documents; __ Do you Yahoo!? Yahoo! Mail Address AutoComplete - You start. We finish. http://promotions.yahoo.com/new_mail -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED] --
Re: Need help optimizing query
Bonjour Arnaud, --- Arnaud <[EMAIL PROTECTED]> wrote: > On 15 Jul 2004 at 6:27, Patrick Drouin > Your indexes look good, but I see that you have some > varchar fields. > Maybe > you could run an "optimize table" on these tables? I'm running it at the moment, I will follow-up on the list when it's done. It could take a while I guess. > Also, you are > talking > about a 40M rows table. If it is a read only MyISAM > table, I might > try > compression. It's mainly read-only, I sometimes batch load some data but users don't update at all. I'll look into compression and see what it is about. Thanks, Patrick __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: Need help optimizing query
On 15 Jul 2004 at 6:27, Patrick Drouin <[EMAIL PROTECTED]> wrote: > I'm having a hard time with the following query. It > retrieves about 3K rows from a few tables. One of them > contains over 40M rows. When run on a 3Ghz server with > 1G of RAM it returns the rows in more than 1 mini. I > don't think that's normal. Your indexes look good, but I see that you have some varchar fields. Maybe you could run an "optimize table" on these tables? Also, you are talking about a 40M rows table. If it is a read only MyISAM table, I might try compression. Arnaud -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Need help optimizing query
Hello everyone, I'm having a hard time with the following query. It retrieves about 3K rows from a few tables. One of them contains over 40M rows. When run on a 3Ghz server with 1G of RAM it returns the rows in more than 1 mini. I don't think that's normal. Here's the output of EXPLAIN: mysql> explain SELECT ti.posi, ti.docid, d.filename, ti.id, c.name FROM corpus_documents cd, corpus c, documents d, tokens_ins ti, tokens t WHERE c.corpusid=4 AND cd.corpusid=c.corpusid AND cd.docid=d.docid AND t.docid=d.docid AND ti.id=t.id AND t.word='police' AND t.docid=ti.docid\g; He +---++--+---+-+---+--+--+| table | type | possible_keys | key | key_len | ref | rows | Extra |+---++--+---+-+---+--+--+| c | ref | PRIMARY,corpus_corpusid | PRIMARY | 3 | const | 1 | Using where; Using index || t | ref | PRIMARY,tokens_id,tokens_docid,tokens_word,tokens_word_docid | PRIMARY | 30 | const | 24 | Using where || ti | ref | PRIMARY,tokens_ins_id,tokens_ins_docid | tokens_ins_id | 4 | t.id | 96 | Using where || d | ref | PRIMARY,documents_docid | PRIMARY | 3 | t.docid | 3 | Using index || cd | eq_ref | PRIMARY | PRIMARY | 6 | const,d.docid | 1 | Using where; Using index |+---++--+---+-+---+--+--+ 5 rows in set (0.00 sec) It seems to be using indexes as expected and it does not seem to look at that many rows. Here's tthe query chewed up and nicely displayed: SELECT ti.posi, ti.docid, d.filename, ti.id, c.name FROM corpus_documents cd, corpus c, documents d, tokens_ins ti, tokens t WHERE c.corpusid=4 AND cd.corpusid=c.corpusid AND cd.docid=d.docid AND t.docid=d.docid AND ti.id=t.id AND t.word='police' AND t.docid=ti.docid; ... 3791 rows in set (1 min 29.78 sec) Here are descriptions of the tables at play : mysql> desc tokens_ins; +--+---+--+-+-+---+ | Field | Type | Null | Key | Default | Extra | +--+---+--+-+-+---+ | id | int(11) | | MUL | 0 | | | posi | int(11) | | PRI | 0 | | | docid | mediumint(20) | | PRI | 0 | | | originalspelling | varchar(30) | | | | | +--+---+--+-+-+---+ 4 rows in set (0.02 sec) mysql> desc tokens; +-++--+-+-+---+ | Field | Type | Null | Key | Default | Extra | +-++--+-+-+---+ | id | int(11) | | MUL | 0 | | | docid | mediumint(20) | | PRI | 0 | | | word | varchar(30) binary | | PRI | | | | pos | varchar(10) | | PRI | 0 | | | absfreq | mediumint(20) | | MUL | 0 | | +-++--+-+-+---+ 5 rows in set (0.00 sec) mysql> desc corpus; +--+---+--+-+-++ | Field | Type | Null | Key | Default | Extra | +--+---+--+-+-++ | corpusid | mediumint(20) | | PRI | NULL | auto_increment | | name | varchar(30) | | PRI |+-+-+--+-+-+---+ | Field | Type | Null | Key | Default | Extra | +-+-+--+-+-+---+ | docid | mediumint(20) | | PRI | 0 | | | filename | varchar(30) | | PRI | | | | language | char(3) | | | | | | description | varchar(255) binary | YES | | NULL | | +-+-+--+-+-+---+ 4 rows in set (0.00 sec) mysql> desc corpus_documents; +--+---+--+-+-+---+ | Field | Type | Null | Key | Default | Extra | +--+---+--+-+-+---+ | corpusid | mediumint(20) | | PRI | 0 | | | docid | mediumint(20) | | PRI | 0 | | +--+---+--+-+-+---+ 2 rows in set (0.00 sec) Can anybody give me a hand speeding up this ting? I'm running out of ideas. Thanks, P | | | language | char(3) | | MUL | | | +--+---+--+-+-++ 3 rows in set (0.00 sec) mysql> desc documents; __ Do you Yahoo!? Yahoo! Mail Address AutoComplete - You start. We finish. http://promotions.yahoo.com/new_mail -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: Need help optimizing query, awfully slow on only 20000 records
Absolutely! I have multiple indexes. I think it might be a problem with ODBC > Are your tables indexed? http://www.mysql.com/doc/en/MySQL_indexes.html > > Saqib Ali > - > http://www.xml-dev.com > > - Visit CARMEL MUSIC & ENTERTAINMENT website http://carmelme.com -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: Need help optimizing query, awfully slow on only 20000 records
Are your tables indexed? http://www.mysql.com/doc/en/MySQL_indexes.html Saqib Ali - http://www.xml-dev.com -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
RE: Need help optimizing query, awfully slow on only 20000 records
What does EXPLAIN SELECT show? Have you read the chapter in the manual on optimizing queries? Do you have all the proper indices set up? --Michael > -Original Message- > From: Apollo (Carmel Entertainment) [mailto:[EMAIL PROTECTED] > Sent: Wednesday, August 20, 2003 2:52 PM > To: [EMAIL PROTECTED] > Cc: [EMAIL PROTECTED] > Subject: RE: Need help optimizing query, awfully slow on only > 2 records > > > When following query is pulled up, it takes about whole 1-2 > minutes to come up. "inquiries" table has 17000 records, > "contacts" has about 7000, "individual_contacts" has about > 16000. It has gotten worse once I upgraded to 4.0 and latest > MyODBC. Clients are separate machines (mix of Win98 and > WinXP). Those 20K records is what feeds the Access97 form, > pull down list filters out some and pulls up about 3K and > people just start typing a name and then (since there > multiple inquiries for some clients) pull down the list to > pick inquiry they want. > > > What are you defining as a huge performance hit? > > > > Is the result set 20K records, or the base tables? How big are the > > base tables? > > > > Are the client and server on the same machine? Pulling 20K records > > across the network could take some time. Formatting 20K > records into a > > pull -down list in access will also take a long time. > Anyway who reads > > a 20K list? > > > > Which parts of the process are slow? How does the query > perform from > > the mysql command line? > > > > > -Original Message- > > > From: Apollo (Carmel Entertainment) [mailto:[EMAIL PROTECTED] > > > Sent: 19 August 2003 17:29 > > > To: [EMAIL PROTECTED] > > > Subject: Need help optimizing query, awfully slow on only 2 > > > records > > > > > > > > > 1.2Ghz Pentium, with 1/2Gig of ram, 4.0.14MySQL, RedHat9.0 > > > > > > I have about 20K records that result from the following > query. Front > > > end for the database is ACCESS97 and pulling up 20K > records makes a > > > huge performance hit. > > > For the form in question I am using PASSTHROUGH type query > > > (the one that just > > > passes everything straight to server without ODBC). > > > NOTE: souce_for_inquiries_form is the join table and is > > > searchable in the from > > > (it feeds a pull-down list). > > > > > > SELECT inquiries.inquiry_id, inquiries.contact_id, > > > inquiries.indiv_contact_id, inquiries.phone, inquiries.fax, > > > inquiries.agent_id, inquiries.inquiry_date, > > > inquiries.event_type, inquiries.letter_type, inquiries.event_date, > > > inquiries.event_date_general, inquiries.event_location, > > > inquiries.guests, > > > inquiries.hours, inquiries.budget, inquiries.event_description, > > > inquiries.talent_description, inquiries.past_use, > > > inquiries.referred_by, > > > inquiries.date_sent, inquiries.end_user, inquiries.event_id, > > > inquiries.notes, > > > source_for_inquiries_form.organization, > > > source_for_inquiries_form.mailing_address_1, > > > source_for_inquiries_form.mailing_address_2, > > > source_for_inquiries_form.city, > > > source_for_inquiries_form.state, source_for_inquiries_form.zip, > > > source_for_inquiries_form.contact_type, > > > individual_contacts.contact_name_first, > > > individual_contacts.contact_name_last, > > > individual_contacts.contact_prefix, > > > individual_contacts.contact_title, individual_contacts.email > > > FROM inquiries > > > LEFT JOIN individual_contacts ON inquiries.indiv_contact_id = > > > individual_contacts.indiv_contact_id > > > INNER JOIN contacts AS source_for_inquiries_form ON > > > inquiries.contact_id = > > > source_for_inquiries_form.contact_id > > > ORDER BY inquiries.inquiry_id DESC; > > > > > > - > > > > - > Visit CARMEL MUSIC & ENTERTAINMENT website http://carmelme.com > > -- > MySQL General Mailing List > For list archives: http://lists.mysql.com/mysql > To unsubscribe: > http://lists.mysql.com/mysql?> [EMAIL PROTECTED] > > -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
RE: Need help optimizing query, awfully slow on only 20000 records
That doesn't look too bad. Is the query that slow when you use the command-line client alone (connected directly to the server), or is it just when accessing the database through the ODBC tunnel? --Michael > -Original Message- > From: Apollo (Carmel Entertainment) [mailto:[EMAIL PROTECTED] > Sent: Wednesday, August 20, 2003 3:14 PM > To: Michael S. Fischer > Cc: [EMAIL PROTECTED] > Subject: RE: Need help optimizing query, awfully slow on only > 2 records > > > > Result of EXPLAIN is: > table|type|possible_keys|key|key_len|ref|rows|Extra > inquiries|ALL|contact_id| | | |8253|Using filesort > individual_contacts|eq_ref|PRIMARY,indiv_contact_id|PRIMARY|3| > inquiries.indiv_contact_id|1 > > source_for_inquiries_form|eq_ref|PRIMARY,contact_id|PRIMARY|3| > inquiries.contact_id|1 > > > > > What does EXPLAIN SELECT show? Have you read the > chapter in the > > manual on optimizing queries? Do you have all the proper > indices set > > up? > > > > --Michael > > > > > -Original Message- > > > From: Apollo (Carmel Entertainment) [mailto:[EMAIL PROTECTED] > > > Sent: Wednesday, August 20, 2003 2:52 PM > > > To: [EMAIL PROTECTED] > > > Cc: [EMAIL PROTECTED] > > > Subject: RE: Need help optimizing query, awfully slow on only > > > 2 records > > > > > > > > > When following query is pulled up, it takes about whole 1-2 > > > minutes to come up. "inquiries" table has 17000 records, > > > "contacts" has about 7000, "individual_contacts" has about > > > 16000. It has gotten worse once I upgraded to 4.0 and latest > > > MyODBC. Clients are separate machines (mix of Win98 and > > > WinXP). Those 20K records is what feeds the Access97 form, > > > pull down list filters out some and pulls up about 3K and > > > people just start typing a name and then (since there > > > multiple inquiries for some clients) pull down the list to > > > pick inquiry they want. > > > > > > > What are you defining as a huge performance hit? > > > > > > > > Is the result set 20K records, or the base tables? How > big are the > > > > base tables? > > > > > > > > Are the client and server on the same machine? Pulling > 20K records > > > > across the network could take some time. Formatting 20K > > > records into a > > > > pull -down list in access will also take a long time. > > > Anyway who reads > > > > a 20K list? > > > > > > > > Which parts of the process are slow? How does the query > > > perform from > > > > the mysql command line? > > > > > > > > > -Original Message- > > > > > From: Apollo (Carmel Entertainment) > [mailto:[EMAIL PROTECTED] > > > > > Sent: 19 August 2003 17:29 > > > > > To: [EMAIL PROTECTED] > > > > > Subject: Need help optimizing query, awfully slow on > only 2 > > > > > records > > > > > > > > > > > > > > > 1.2Ghz Pentium, with 1/2Gig of ram, 4.0.14MySQL, RedHat9.0 > > > > > > > > > > I have about 20K records that result from the following > > > query. Front > > > > > end for the database is ACCESS97 and pulling up 20K > > > records makes a > > > > > huge performance hit. > > > > > For the form in question I am using PASSTHROUGH type query > > > > > (the one that just > > > > > passes everything straight to server without ODBC). > > > > > NOTE: souce_for_inquiries_form is the join table and is > > > > > searchable in the from > > > > > (it feeds a pull-down list). > > > > > > > > > > SELECT inquiries.inquiry_id, inquiries.contact_id, > > > > > inquiries.indiv_contact_id, inquiries.phone, inquiries.fax, > > > > > inquiries.agent_id, inquiries.inquiry_date, > > > > > inquiries.event_type, inquiries.letter_type, > inquiries.event_date, > > > > > inquiries.event_date_general, inquiries.event_location, > > > > > inquiries.guests, > > > > > inquiries.hours, inquiries.budget, > inquiries.event_description, > > > > > inquiries.talent_description, inquiries.past_use, > > > > > inquiries.referred_by,
RE: Need help optimizing query, awfully slow on only 20000 records
When I send the query throuh comman line, it works perfect 3-4 seconds, but when I do Access97 pass-through query, that is when it runs into 1-2 minutes. It is almost as slow as using Access97 native query that goes through MyODBC, so... > That doesn't look too bad. Is the query that slow when you use the > command-line client alone (connected directly to the server), or is it > just when accessing the database through the ODBC tunnel? > > --Michael > > > -Original Message- > > From: Apollo (Carmel Entertainment) [mailto:[EMAIL PROTECTED] > > Sent: Wednesday, August 20, 2003 3:14 PM > > To: Michael S. Fischer > > Cc: [EMAIL PROTECTED] > > Subject: RE: Need help optimizing query, awfully slow on only > > 2 records > > > > > > > > Result of EXPLAIN is: > > table|type|possible_keys|key|key_len|ref|rows|Extra > > inquiries|ALL|contact_id| | | |8253|Using filesort > > individual_contacts|eq_ref|PRIMARY,indiv_contact_id|PRIMARY|3| > > inquiries.indiv_contact_id|1 > > > > source_for_inquiries_form|eq_ref|PRIMARY,contact_id|PRIMARY|3| > > inquiries.contact_id|1 > > > > > > > > > What does EXPLAIN SELECT show? Have you read the > > chapter in the > > > manual on optimizing queries? Do you have all the proper > > indices set > > > up? > > > > > > --Michael > > > > > > > -Original Message- > > > > From: Apollo (Carmel Entertainment) [mailto:[EMAIL PROTECTED] > > > > Sent: Wednesday, August 20, 2003 2:52 PM > > > > To: [EMAIL PROTECTED] > > > > Cc: [EMAIL PROTECTED] > > > > Subject: RE: Need help optimizing query, awfully slow on only > > > > 2 records > > > > > > > > > > > > When following query is pulled up, it takes about whole 1-2 > > > > minutes to come up. "inquiries" table has 17000 records, > > > > "contacts" has about 7000, "individual_contacts" has about > > > > 16000. It has gotten worse once I upgraded to 4.0 and latest > > > > MyODBC. Clients are separate machines (mix of Win98 and > > > > WinXP). Those 20K records is what feeds the Access97 form, > > > > pull down list filters out some and pulls up about 3K and > > > > people just start typing a name and then (since there > > > > multiple inquiries for some clients) pull down the list to > > > > pick inquiry they want. > > > > > > > > > What are you defining as a huge performance hit? > > > > > > > > > > Is the result set 20K records, or the base tables? How > > big are the > > > > > base tables? > > > > > > > > > > Are the client and server on the same machine? Pulling > > 20K records > > > > > across the network could take some time. Formatting 20K > > > > records into a > > > > > pull -down list in access will also take a long time. > > > > Anyway who reads > > > > > a 20K list? > > > > > > > > > > Which parts of the process are slow? How does the query > > > > perform from > > > > > the mysql command line? > > > > > > > > > > > -Original Message- > > > > > > From: Apollo (Carmel Entertainment) > > [mailto:[EMAIL PROTECTED] > > > > > > Sent: 19 August 2003 17:29 > > > > > > To: [EMAIL PROTECTED] > > > > > > Subject: Need help optimizing query, awfully slow on > > only 2 > > > > > > records > > > > > > > > > > > > > > > > > > 1.2Ghz Pentium, with 1/2Gig of ram, 4.0.14MySQL, RedHat9.0 > > > > > > > > > > > > I have about 20K records that result from the following > > > > query. Front > > > > > > end for the database is ACCESS97 and pulling up 20K > > > > records makes a > > > > > > huge performance hit. > > > > > > For the form in question I am using PASSTHROUGH type query > > > > > > (the one that just > > > > > > passes everything straight to server without ODBC). > > > > > > NOTE: souce_for_inquiries_form is the join table and is > > > > > > searchable in the from > > > > > > (it feeds a pull-down list). > > > > > > > > > > >
Re: Need help optimizing query, awfully slow on only 20000 records
Query takes 3.4 seconds to run on the server, but it takes 1-2minutes to run via MyODBC 3.51.06 using passthrough (Access97 is the front end, but it has query type that allows bypass of Access interpretation. > Two Questions: > > Is the same query running directly on the linux server thru mysql is also > very slow ? > > Have you done a explain plan on the query ? > > Marc. > > - Visit CARMEL MUSIC & ENTERTAINMENT website http://carmelme.com -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
RE: Need help optimizing query, awfully slow on only 20000 records
Result of EXPLAIN is: table|type|possible_keys|key|key_len|ref|rows|Extra inquiries|ALL|contact_id| | | |8253|Using filesort individual_contacts|eq_ref|PRIMARY,indiv_contact_id|PRIMARY|3|inquiries.indiv_contact_id|1 source_for_inquiries_form|eq_ref|PRIMARY,contact_id|PRIMARY|3|inquiries.contact_id|1 > What does EXPLAIN SELECT show? Have you read the chapter in the > manual on optimizing queries? Do you have all the proper indices set > up? > > --Michael > > > -Original Message- > > From: Apollo (Carmel Entertainment) [mailto:[EMAIL PROTECTED] > > Sent: Wednesday, August 20, 2003 2:52 PM > > To: [EMAIL PROTECTED] > > Cc: [EMAIL PROTECTED] > > Subject: RE: Need help optimizing query, awfully slow on only > > 2 records > > > > > > When following query is pulled up, it takes about whole 1-2 > > minutes to come up. "inquiries" table has 17000 records, > > "contacts" has about 7000, "individual_contacts" has about > > 16000. It has gotten worse once I upgraded to 4.0 and latest > > MyODBC. Clients are separate machines (mix of Win98 and > > WinXP). Those 20K records is what feeds the Access97 form, > > pull down list filters out some and pulls up about 3K and > > people just start typing a name and then (since there > > multiple inquiries for some clients) pull down the list to > > pick inquiry they want. > > > > > What are you defining as a huge performance hit? > > > > > > Is the result set 20K records, or the base tables? How big are the > > > base tables? > > > > > > Are the client and server on the same machine? Pulling 20K records > > > across the network could take some time. Formatting 20K > > records into a > > > pull -down list in access will also take a long time. > > Anyway who reads > > > a 20K list? > > > > > > Which parts of the process are slow? How does the query > > perform from > > > the mysql command line? > > > > > > > -Original Message- > > > > From: Apollo (Carmel Entertainment) [mailto:[EMAIL PROTECTED] > > > > Sent: 19 August 2003 17:29 > > > > To: [EMAIL PROTECTED] > > > > Subject: Need help optimizing query, awfully slow on only 2 > > > > records > > > > > > > > > > > > 1.2Ghz Pentium, with 1/2Gig of ram, 4.0.14MySQL, RedHat9.0 > > > > > > > > I have about 20K records that result from the following > > query. Front > > > > end for the database is ACCESS97 and pulling up 20K > > records makes a > > > > huge performance hit. > > > > For the form in question I am using PASSTHROUGH type query > > > > (the one that just > > > > passes everything straight to server without ODBC). > > > > NOTE: souce_for_inquiries_form is the join table and is > > > > searchable in the from > > > > (it feeds a pull-down list). > > > > > > > > SELECT inquiries.inquiry_id, inquiries.contact_id, > > > > inquiries.indiv_contact_id, inquiries.phone, inquiries.fax, > > > > inquiries.agent_id, inquiries.inquiry_date, > > > > inquiries.event_type, inquiries.letter_type, inquiries.event_date, > > > > inquiries.event_date_general, inquiries.event_location, > > > > inquiries.guests, > > > > inquiries.hours, inquiries.budget, inquiries.event_description, > > > > inquiries.talent_description, inquiries.past_use, > > > > inquiries.referred_by, > > > > inquiries.date_sent, inquiries.end_user, inquiries.event_id, > > > > inquiries.notes, > > > > source_for_inquiries_form.organization, > > > > source_for_inquiries_form.mailing_address_1, > > > > source_for_inquiries_form.mailing_address_2, > > > > source_for_inquiries_form.city, > > > > source_for_inquiries_form.state, source_for_inquiries_form.zip, > > > > source_for_inquiries_form.contact_type, > > > > individual_contacts.contact_name_first, > > > > individual_contacts.contact_name_last, > > > > individual_contacts.contact_prefix, > > > > individual_contacts.contact_title, individual_contacts.email > > > > FROM inquiries > > > > LEFT JOIN individual_contacts ON inquiries.indiv_contact_id = > > > > individual_contacts.indiv_contact_id > > > > INNER JOIN contacts AS source_for_inquiries_form ON > > > > inquiries.contact_id = > &g
RE: Need help optimizing query, awfully slow on only 20000 records
When following query is pulled up, it takes about whole 1-2 minutes to come up. "inquiries" table has 17000 records, "contacts" has about 7000, "individual_contacts" has about 16000. It has gotten worse once I upgraded to 4.0 and latest MyODBC. Clients are separate machines (mix of Win98 and WinXP). Those 20K records is what feeds the Access97 form, pull down list filters out some and pulls up about 3K and people just start typing a name and then (since there multiple inquiries for some clients) pull down the list to pick inquiry they want. > What are you defining as a huge performance hit? > > Is the result set 20K records, or the base tables? How big are the base > tables? > > Are the client and server on the same machine? Pulling 20K records across > the network could take some time. Formatting 20K records into a pull -down > list in access will also take a long time. Anyway who reads a 20K list? > > Which parts of the process are slow? How does the query perform from the > mysql command line? > > > -Original Message- > > From: Apollo (Carmel Entertainment) [mailto:[EMAIL PROTECTED] > > Sent: 19 August 2003 17:29 > > To: [EMAIL PROTECTED] > > Subject: Need help optimizing query, awfully slow on only > > 2 records > > > > > > 1.2Ghz Pentium, with 1/2Gig of ram, 4.0.14MySQL, RedHat9.0 > > > > I have about 20K records that result from the following > > query. Front end for the > > database is ACCESS97 and pulling up 20K records makes a huge > > performance hit. > > For the form in question I am using PASSTHROUGH type query > > (the one that just > > passes everything straight to server without ODBC). > > NOTE: souce_for_inquiries_form is the join table and is > > searchable in the from > > (it feeds a pull-down list). > > > > SELECT inquiries.inquiry_id, inquiries.contact_id, > > inquiries.indiv_contact_id, > > inquiries.phone, inquiries.fax, inquiries.agent_id, > > inquiries.inquiry_date, > > inquiries.event_type, inquiries.letter_type, inquiries.event_date, > > inquiries.event_date_general, inquiries.event_location, > > inquiries.guests, > > inquiries.hours, inquiries.budget, inquiries.event_description, > > inquiries.talent_description, inquiries.past_use, > > inquiries.referred_by, > > inquiries.date_sent, inquiries.end_user, inquiries.event_id, > > inquiries.notes, > > source_for_inquiries_form.organization, > > source_for_inquiries_form.mailing_address_1, > > source_for_inquiries_form.mailing_address_2, > > source_for_inquiries_form.city, > > source_for_inquiries_form.state, source_for_inquiries_form.zip, > > source_for_inquiries_form.contact_type, > > individual_contacts.contact_name_first, > > individual_contacts.contact_name_last, > > individual_contacts.contact_prefix, > > individual_contacts.contact_title, individual_contacts.email > > FROM inquiries > > LEFT JOIN individual_contacts ON inquiries.indiv_contact_id = > > individual_contacts.indiv_contact_id > > INNER JOIN contacts AS source_for_inquiries_form ON > > inquiries.contact_id = > > source_for_inquiries_form.contact_id > > ORDER BY inquiries.inquiry_id DESC; > > > > - - Visit CARMEL MUSIC & ENTERTAINMENT website http://carmelme.com -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
RE: Need help optimizing query, awfully slow on only 20000 records
What are you defining as a huge performance hit? Is the result set 20K records, or the base tables? How big are the base tables? Are the client and server on the same machine? Pulling 20K records across the network could take some time. Formatting 20K records into a pull -down list in access will also take a long time. Anyway who reads a 20K list? Which parts of the process are slow? How does the query perform from the mysql command line? > -Original Message- > From: Apollo (Carmel Entertainment) [mailto:[EMAIL PROTECTED] > Sent: 19 August 2003 17:29 > To: [EMAIL PROTECTED] > Subject: Need help optimizing query, awfully slow on only > 2 records > > > 1.2Ghz Pentium, with 1/2Gig of ram, 4.0.14MySQL, RedHat9.0 > > I have about 20K records that result from the following > query. Front end for the > database is ACCESS97 and pulling up 20K records makes a huge > performance hit. > For the form in question I am using PASSTHROUGH type query > (the one that just > passes everything straight to server without ODBC). > NOTE: souce_for_inquiries_form is the join table and is > searchable in the from > (it feeds a pull-down list). > > SELECT inquiries.inquiry_id, inquiries.contact_id, > inquiries.indiv_contact_id, > inquiries.phone, inquiries.fax, inquiries.agent_id, > inquiries.inquiry_date, > inquiries.event_type, inquiries.letter_type, inquiries.event_date, > inquiries.event_date_general, inquiries.event_location, > inquiries.guests, > inquiries.hours, inquiries.budget, inquiries.event_description, > inquiries.talent_description, inquiries.past_use, > inquiries.referred_by, > inquiries.date_sent, inquiries.end_user, inquiries.event_id, > inquiries.notes, > source_for_inquiries_form.organization, > source_for_inquiries_form.mailing_address_1, > source_for_inquiries_form.mailing_address_2, > source_for_inquiries_form.city, > source_for_inquiries_form.state, source_for_inquiries_form.zip, > source_for_inquiries_form.contact_type, > individual_contacts.contact_name_first, > individual_contacts.contact_name_last, > individual_contacts.contact_prefix, > individual_contacts.contact_title, individual_contacts.email > FROM inquiries > LEFT JOIN individual_contacts ON inquiries.indiv_contact_id = > individual_contacts.indiv_contact_id > INNER JOIN contacts AS source_for_inquiries_form ON > inquiries.contact_id = > source_for_inquiries_form.contact_id > ORDER BY inquiries.inquiry_id DESC; > > - > Visit CARMEL MUSIC & ENTERTAINMENT website http://carmelme.com > > -- > MySQL General Mailing List > For list archives: http://lists.mysql.com/mysql > To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED] -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Need help optimizing query, awfully slow on only 20000 records
Two Questions: Is the same query running directly on the linux server thru mysql is also very slow ? Have you done a explain plan on the query ? Marc. -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Need help optimizing query, awfully slow on only 20000 records
1.2Ghz Pentium, with 1/2Gig of ram, 4.0.14MySQL, RedHat9.0 I have about 20K records that result from the following query. Front end for the database is ACCESS97 and pulling up 20K records makes a huge performance hit. For the form in question I am using PASSTHROUGH type query (the one that just passes everything straight to server without ODBC). NOTE: souce_for_inquiries_form is the join table and is searchable in the from (it feeds a pull-down list). SELECT inquiries.inquiry_id, inquiries.contact_id, inquiries.indiv_contact_id, inquiries.phone, inquiries.fax, inquiries.agent_id, inquiries.inquiry_date, inquiries.event_type, inquiries.letter_type, inquiries.event_date, inquiries.event_date_general, inquiries.event_location, inquiries.guests, inquiries.hours, inquiries.budget, inquiries.event_description, inquiries.talent_description, inquiries.past_use, inquiries.referred_by, inquiries.date_sent, inquiries.end_user, inquiries.event_id, inquiries.notes, source_for_inquiries_form.organization, source_for_inquiries_form.mailing_address_1, source_for_inquiries_form.mailing_address_2, source_for_inquiries_form.city, source_for_inquiries_form.state, source_for_inquiries_form.zip, source_for_inquiries_form.contact_type, individual_contacts.contact_name_first, individual_contacts.contact_name_last, individual_contacts.contact_prefix, individual_contacts.contact_title, individual_contacts.email FROM inquiries LEFT JOIN individual_contacts ON inquiries.indiv_contact_id = individual_contacts.indiv_contact_id INNER JOIN contacts AS source_for_inquiries_form ON inquiries.contact_id = source_for_inquiries_form.contact_id ORDER BY inquiries.inquiry_id DESC; - Visit CARMEL MUSIC & ENTERTAINMENT website http://carmelme.com -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Need help optimizing query
Greetings, I am struggling trying to get good performance
using indexes.
I'll spare the details and narrow down the situation. I have about
100,000 rows of data with the following fields:
title - vachar(64)
description - text
state - varchar(2)
priority - int
modified - date
People type in a search string, and I need the results ordered in
the following way:
1) by priority DESC
2) items matching the title
3) items matching the description
I've experimented with full-text search, but cannot get it to do
this effeciently all in one query because of the way I need
to order things. I tried adding two full-text indexes like:
SELECT title, MATCH(title) AGAINST ('query') AS intitle FROM mytable WHERE
MATCH(title, description) AGAINST ('query') [AND state = 'state'] ORDER by priority
desc, intitle desc, modified desc;
but this only uses the (title,description) index, an not the (title)
index or the state index.
It would be much faster if I could do only a full text search on
records with the state that is selected. (Any ideas on how to do this?)
Currently I do the following which which is really weak when there is
no state selected:
SELECT title, (title LIKE '%query%') AS intitle FROM mytable WHERE ((description LIKE
'%query%') OR (title LIKE '%query%')) [AND state='state'] ORDER BY priority desc,
intitle desc, modified desc;
Does anyone have any better ideas? I would consider doing multiple queries
using the different indexes but then I don't know how many rows matched
without taking out the duplicates which would be expensive, I think.
Thanks in advance,
John
-
Before posting, please check:
http://www.mysql.com/manual.php (the manual)
http://lists.mysql.com/ (the list archive)
To request this thread, e-mail <[EMAIL PROTECTED]>
To unsubscribe, e-mail <[EMAIL PROTECTED]>
Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
