Re: Solved Select query locks tables in Innodb
Just to close this off. Baron was correct in that the core problem was a bug in MySQL (I was using version 5.0.37.) There are some references to this bug in the MySQL bug stuff but they claim to have eliminated it in 5.0.30... apparently not. I ungraded to version 5.1.32 and the original problem disappeared and the selects behave as one would expect. Many thanks to all who offered advice. Carl - Original Message - From: Perrin Harkins per...@elem.com To: Carl c...@etrak-plus.com Cc: mysql@lists.mysql.com Sent: Friday, March 13, 2009 1:40 PM Subject: Re: Select query locks tables in Innodb 2009/3/12 Carl c...@etrak-plus.com: I am still a little puzzled about how we could have a relatively large set of records (100,000+) and yet not cause any table to be locked as the server has only 8GB of memory. What's the relationship you're implying between memory and locking? Multi-version concurrency doesn't necessarily mean the older versions that are being read from have to be entirely in memory. InnoDB will lock on a query that doesn't use an index. It shouldn't lock on a SELECT query, regardless of the indexes involved. - Perrin -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql?unsub=arch...@jab.org
Re: Select query locks tables in Innodb
2009/3/12 Carl c...@etrak-plus.com: I am still a little puzzled about how we could have a relatively large set of records (100,000+) and yet not cause any table to be locked as the server has only 8GB of memory. What's the relationship you're implying between memory and locking? Multi-version concurrency doesn't necessarily mean the older versions that are being read from have to be entirely in memory. InnoDB will lock on a query that doesn't use an index. It shouldn't lock on a SELECT query, regardless of the indexes involved. - Perrin -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql?unsub=arch...@jab.org
Re: Select query locks tables in Innodb
Brent, After a delay while I was busy killing alligators, I did as you suggested (added a composite index of date and organization_serial on journal_entry_master... in the spirit of your suggestion, anyway.) The results were interesting: 1. In my test environment, I could not force a locked file even though I opened the dates up to cover 2+ years and changed to an organization that had more records. The 'Explain' is attached as temp1.txt. You will note that it starts with 100,000+ records while the eventual set of records for the report is 60,000 because the 100,000+ number includes some journmal entries for refund/void/etc. transactions which we have no interest in. 2. I tried various combinations of indexes but couldn't seem to get any better than the composite one on the journal_entry_master. I did not check whether the other options would produce locked files. I am now going to put this into production and see if it will actually fly. I am still a little puzzled about how we could have a relatively large set of records (100,000+) and yet not cause any table to be locked as the server has only 8GB of memory. Thanks for all your help and Baron's suggestions also. Carl - Original Message - From: Brent Baisley brentt...@gmail.com To: Carl c...@etrak-plus.com Sent: Thursday, March 05, 2009 1:12 PM Subject: Re: Select query locks tables in Innodb Ok, so you have 687 unique organization serial numbers. That's not very unique, on average it will only narrow down the table to 1/687 of it's full size. This is probably the source of your locking problem and where you want to focus. InnoDB will lock on a query that doesn't use an index. It would have to lock every record anyway, so why not lock the table? 36,000 records still may be too large of a result set to do record versioning. But, optimizing your query is the only way to go. Your date_effective is a lot more granular, so you may want to focus on that. If you do a lot of these types of searches, you can try creating a compound index on organization_serial+date_effective. CREATE INDEX (org_date) ON journal_entry_master(organization_serial,date_effective) MySQL would/should then use that query, which will narrow things down quicker and better. It shouldn't have to try to do versioning on 56,000 records while it tries to get the subset of that (36,000). Brent On Thu, Mar 5, 2009 at 6:02 AM, Carl c...@etrak-plus.com wrote: Brent, The query returns about 36,000 rows. The 56,000 rows from the journal_entry_master table is all the entries for organization 16 (they span more than the dates I have asked for.) SHOW INDEX FROM journal_entry_master shows 1,554,000+ for the primary index (auto-increment), 687 for the organization_serial (the one I am using), 18 for the organization_shift_start (I tried this before, i.e., starting with the organization_shift, but it quickly got mired down) and 777,000+ for the date_effective. If I understand correctly, you have suggested using the date index. The difficulty is the data contains many organizations and so the date range query returns 163,000+ rows. Also, I would expect scaling a query where I had to programatically cut it up would 1) be difficult and 2) wouldn't really solve the problem but would rather just shorten the time of the locks. I am not suggesting that I might not end up there, only hoping for a better solution. Thanks for all your insight and feel free to suggest away. Carl - Original Message - From: Brent Baisley brentt...@gmail.com To: Carl c...@etrak-plus.com Sent: Wednesday, March 04, 2009 4:23 PM Subject: Re: Select query locks tables in Innodb Is the result of the query returning 56,000+ rows? How many rows are you expecting to be returned once the query is finished running? Your date range is over a year. You may actually get much better performance (and avoid locking) by running more queries with a narrower date range and linking them through a UNION. It's using the organization index rather than the date index. I don't know your dataset, but typically you want your query to use the date index since that narrows down the data set better. You can run SHOW INDEX FROM journal_entry_master to see the distribution of your data in the index. The cardinality column will indicate the uniqueness of your data. The higher the number, the more unique values. Brent 2009/3/4 Carl c...@etrak-plus.com: Under stress (having transaction entered), the query shows that it is still locking the tables. I rewrote the query and tested it step by step but could not tell whether tyhe partially complete query was locking tables because it ran so fast. However, when I had all the pieces in the query (copy attached), I could easily see it was locking tables using the Server Monitor in Navicat. Explain (copy as text and copy as Excel attached) seems to indicate that it is fairly good although the first step does get quite a few rows. Does anyone
Re: Select query locks tables in Innodb
The nice thing about InnnoDB is that it won't have to access the data portion of the file if it doesn't have to. So if all the information you are retrieving is contained in an index, it only accesses the index to get the information it needs. The data portion is never access, and thus never locked. Something like this is probably going on. All the information it needs for the 100,000 records is contained in the index, the the data portion is never accessed until it needs to retrieve the 60,000 records. That's a simplistic overview of what could be going on. But it sounds like your issue has been resolved. Interesting, your temp1 attached file shows mysql switched from using the org_date index to the organization index. Brent Baisley 2009/3/12 Carl c...@etrak-plus.com: Brent, After a delay while I was busy killing alligators, I did as you suggested (added a composite index of date and organization_serial on journal_entry_master... in the spirit of your suggestion, anyway.) The results were interesting: 1. In my test environment, I could not force a locked file even though I opened the dates up to cover 2+ years and changed to an organization that had more records. The 'Explain' is attached as temp1.txt. You will note that it starts with 100,000+ records while the eventual set of records for the report is 60,000 because the 100,000+ number includes some journmal entries for refund/void/etc. transactions which we have no interest in. 2. I tried various combinations of indexes but couldn't seem to get any better than the composite one on the journal_entry_master. I did not check whether the other options would produce locked files. I am now going to put this into production and see if it will actually fly. I am still a little puzzled about how we could have a relatively large set of records (100,000+) and yet not cause any table to be locked as the server has only 8GB of memory. Thanks for all your help and Baron's suggestions also. Carl - Original Message - From: Brent Baisley brentt...@gmail.com To: Carl c...@etrak-plus.com Sent: Thursday, March 05, 2009 1:12 PM Subject: Re: Select query locks tables in Innodb Ok, so you have 687 unique organization serial numbers. That's not very unique, on average it will only narrow down the table to 1/687 of it's full size. This is probably the source of your locking problem and where you want to focus. InnoDB will lock on a query that doesn't use an index. It would have to lock every record anyway, so why not lock the table? 36,000 records still may be too large of a result set to do record versioning. But, optimizing your query is the only way to go. Your date_effective is a lot more granular, so you may want to focus on that. If you do a lot of these types of searches, you can try creating a compound index on organization_serial+date_effective. CREATE INDEX (org_date) ON journal_entry_master(organization_serial,date_effective) MySQL would/should then use that query, which will narrow things down quicker and better. It shouldn't have to try to do versioning on 56,000 records while it tries to get the subset of that (36,000). Brent On Thu, Mar 5, 2009 at 6:02 AM, Carl c...@etrak-plus.com wrote: Brent, The query returns about 36,000 rows. The 56,000 rows from the journal_entry_master table is all the entries for organization 16 (they span more than the dates I have asked for.) SHOW INDEX FROM journal_entry_master shows 1,554,000+ for the primary index (auto-increment), 687 for the organization_serial (the one I am using), 18 for the organization_shift_start (I tried this before, i.e., starting with the organization_shift, but it quickly got mired down) and 777,000+ for the date_effective. If I understand correctly, you have suggested using the date index. The difficulty is the data contains many organizations and so the date range query returns 163,000+ rows. Also, I would expect scaling a query where I had to programatically cut it up would 1) be difficult and 2) wouldn't really solve the problem but would rather just shorten the time of the locks. I am not suggesting that I might not end up there, only hoping for a better solution. Thanks for all your insight and feel free to suggest away. Carl - Original Message - From: Brent Baisley brentt...@gmail.com To: Carl c...@etrak-plus.com Sent: Wednesday, March 04, 2009 4:23 PM Subject: Re: Select query locks tables in Innodb Is the result of the query returning 56,000+ rows? How many rows are you expecting to be returned once the query is finished running? Your date range is over a year. You may actually get much better performance (and avoid locking) by running more queries with a narrower date range and linking them through a UNION. It's using the organization index rather than the date index. I don't know your dataset, but typically you want your query to use the date index since
Re: Select query locks tables in Innodb
I really appreciate the time you have taken to help me with this problem.
I will be out of the office until around 1:00PM and will try your
suggestions.
I did attach a copy of the query but it may have been stripped somewhere
along the line so I have placed it in line below.
select *
from payment_to_fee_link_budget_account_detail_link, journal_entry_master,
journal_entry_type,
payment_to_fee_link_event, payment_to_fee_link, fees, fees_event,
fees_budget_account_detail_link, person, transactions
left join regs on regs.transactions_serial =
transactions.transactions_serial,
transaction_event, receipt_master, budget_account_detail,
budget_account_detail as ptfl_budget_account_detail, budget_account_master
where journal_entry_master.organization_serial = 16
and journal_entry_master.date_effective = '2008-01-01'
and journal_entry_master.date_effective '2009-03-31'
and journal_entry_type.journal_entry_type_serial =
journal_entry_master.journal_entry_type_serial
and
payment_to_fee_link_budget_account_detail_link.journal_entry_master_serial =
journal_entry_master.journal_entry_master_serial
and payment_to_fee_link_budget_account_detail_link.date_effective =
'2008-01-01'
and payment_to_fee_link_budget_account_detail_link.date_effective
'2009-03-31'
and payment_to_fee_link_event.payment_to_fee_link_event_serial =
payment_to_fee_link_budget_account_detail_link.payment_to_fee_link_event_serial
and payment_to_fee_link.payment_to_fee_link_serial =
payment_to_fee_link_event.payment_to_fee_link_serial
and transaction_event.transaction_event_serial =
payment_to_fee_link_event.transaction_event_serial
and fees.fees_serial = payment_to_fee_link.fees_serial
and transactions.transactions_serial = fees.transactions_serial
and person.person_serial = transactions.person_serial
and receipt_master.receipt_serial = transaction_event.receipt_serial
and fees_event.fees_serial = payment_to_fee_link.fees_serial
and ( fees_event.transaction_event_description_serial = 13 or
fees_event.transaction_event_description_serial = 2 )
and fees_budget_account_detail_link.fees_event_serial =
fees_event.fees_event_serial
and budget_account_detail.budget_account_detail_serial =
fees_budget_account_detail_link.budget_account_detail_serial
and ptfl_budget_account_detail.budget_account_detail_serial =
payment_to_fee_link_budget_account_detail_link.budget_account_detail_serial
and budget_account_master.budget_account_serial =
budget_account_detail.budget_account_serial
and budget_account_master.budget_account_type_serial = 5001
TIA,
Carl
- Original Message -
From: Baron Schwartz ba...@xaprb.com
To: Carl c...@etrak-plus.com
Cc: mysql@lists.mysql.com
Sent: Wednesday, March 04, 2009 8:11 PM
Subject: Re: Select query locks tables in Innodb
I don't think it locks the tables. The behavior may be similar, but I
seriously doubt that's what's happening. Take a snapshot of SHOW
INNODB STATUS while this is going on. And use mysqladmin debug and
check the error log. Then put those in some pastebin and send us the
link. And realize that you've only given us bits and snippets of
information about this -- you still haven't given us SHOW CREATE TABLE
or even shown us the query that's running. There's not a lot I can do
to really help you with this other than assume that you are wrong :)
Your version is definitely affected by that bug, which I can't find --
I am using the wrong search terms and can't find the right ones to
find the bug.
5.0.37 is a very buggy version and I would upgrade regardless if I
were you, to the latest 5.0 release. You might be surprised at how
much that changes things.
Baron
On Wed, Mar 4, 2009 at 3:33 PM, Carl c...@etrak-plus.com wrote:
Baron,
I am using 5.0.37.
While it may be true that there is a bug that shows tables as being locked
when they really aren't, I do not think that applies here. I do know that
when a table shows a status of 'Locked' in the Navicat Server Monitor that
the transaction which created and is processing the query comes to a
complete stop until the report query (the one I am trying to straighten
out
or understand) is finished. For example, the report query is reading from
several files, e.g., receipt_master, if a user tries to check out (which
requires an insert into the receipt_master table), they are stopped until
the report query finishes and query on that table shows in Navicat as
waiting for lock ('Locked'.)
Since the report query is only reading data, I am puzzled why it locks the
tables. Any ideas?
TIA,
Carl
- Original Message - From: Baron Schwartz ba...@xaprb.com
To: Carl c...@etrak-plus.com
Cc: mysql@lists.mysql.com
Sent: Wednesday, March 04, 2009 2:29 PM
Subject: Re: Select query locks tables in Innodb
Carl,
Locked status in SHOW PROCESSLIST and a table being locked are
different. There is a bug in MySQL that shows Locked status for
queries accessing InnoDB tables in some cases. What version of MySQL
are you
Re: Select query locks tables in Innodb
Thanks to all of you.
The key was the 107488 rows. I restructured the query so that it started
with something smaller and it 1) runs faster (I'm guessing the reduced use
of temp space) and 2) did not seem to cause any locking problems (I will
test this under load today.)
I have attached a copy of the query which has been simplified in a couple of
ways (I don't really want every field from every row selected from every
table.) Also, the constants like organization_serial (16) and dates are
variables in the real version.
The explain now shows:
idtable typepossible_keys
keylenref rows
1organization_shiftrefPRIMARY, organizationorganization
4const5
1organization_shift_start ref PRIMARY, organization_shift
organization_shift4 organization_shift_serial295
1journal_entry_masterrefPRIMARY, organization_shift_start
organization_shift_start 5 organization_shift_start_serial 52
Note that it now starts with 5 row, expands to 295 rows, etc. not the
100,000+ from before.
Again, thanks for all your help.
Carl
- Original Message -
From: Baron Schwartz ba...@xaprb.com
To: Brent Baisley brentt...@gmail.com
Cc: Carl c...@etrak-plus.com; mysql@lists.mysql.com
Sent: Tuesday, March 03, 2009 5:50 PM
Subject: Re: Select query locks tables in Innodb
On Tue, Mar 3, 2009 at 12:35 PM, Brent Baisley brentt...@gmail.com
wrote:
A SELECT will/can lock a table. It almost always does in MyISAM (no
insert/updates), almost never does in InnoDB. There is an exception to
every rule. The problem is most likely in the 107488 rows part of the
query. That's too many rows for InnoDB to keep a version history on so
it's likely just locking the table.
InnoDB does not do lock escalation a la SQL Server etc. I'd look at
Perrin's suggestions, I think they are likely to be the problem.
More importantly, what is the query? :-)
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/mysql?unsub=c...@etrak-plus.com
# sales from collections
(select *
from organization_shift, organization_shift_start, transaction_event,payment_to_fee_link_event,payment_to_fee_link, payment_to_fee_link_budget_account_detail_link,fees_budget_account_detail_link, fees_event, budget_account_detail,
payments, budget_account_detail as ptfl_budget_account_detail, budget_account_master, journal_entry_master, journal_entry_type, receipt_master, person, transactions
left join regs on regs.transactions_serial =
transactions.transactions_serial
where organization_shift.organization_serial = 16
and organization_shift_start.organization_shift_serial =
organization_shift.organization_shift_serial
and organization_shift_start.date_effective = '2008-01-01'
and organization_shift_start.date_effective '2009-03-31'
#$P!{organizationShiftStartQuery}
and journal_entry_master.organization_shift_start_serial =
organization_shift_start.organization_shift_start_serial
and receipt_master.receipt_serial =
transaction_event.receipt_serial
and transactions.transactions_serial =
transaction_event.transactions_serial
and transactions.organization_serial = organization_shift.organization_serial
#$P!{itemSerials}
and person.person_serial = transactions.person_serial
and payment_to_fee_link_event.transaction_event_serial =
transaction_event.transaction_event_serial
and
payment_to_fee_link_budget_account_detail_link.payment_to_fee_link_event_serial
= payment_to_fee_link_event.payment_to_fee_link_event_serial
and
payment_to_fee_link_budget_account_detail_link.cash_basis_reporting_flag = 'Y'
and payment_to_fee_link.payment_to_fee_link_serial =
payment_to_fee_link_event.payment_to_fee_link_serial
and payments.payments_serial =
payment_to_fee_link.payments_serial
and payment_to_fee_link_budget_account_detail_link.date_effective
= '2008-01-01'
and payment_to_fee_link_budget_account_detail_link.date_effective
'2009-03-31'
and (payments.payment_type_code_serial in (
1,2,3,4,5,8,24,6,7,12,13,23,25 )# 1,2,3,4,5,8,24,6,7,12,13,23,25
or
payment_to_fee_link_budget_account_detail_link.description='Apply available
credit to customer accounts receivable')
and fees_event.fees_serial = payment_to_fee_link.fees_serial
and ( fees_event.transaction_event_description_serial = 13 or
fees_event.transaction_event_description_serial = 2 )
and fees_budget_account_detail_link.fees_event_serial =
fees_event.fees_event_serial
and
fees_budget_account_detail_link.budget_account_detail_serial
Re: Select query locks tables in Innodb
Under stress (having transaction entered), the query shows that it is still locking the tables. I rewrote the query and tested it step by step but could not tell whether tyhe partially complete query was locking tables because it ran so fast. However, when I had all the pieces in the query (copy attached), I could easily see it was locking tables using the Server Monitor in Navicat. Explain (copy as text and copy as Excel attached) seems to indicate that it is fairly good although the first step does get quite a few rows. Does anyone have any ideas? TIA, Carl - Original Message - From: Baron Schwartz ba...@xaprb.com To: Brent Baisley brentt...@gmail.com Cc: Carl c...@etrak-plus.com; mysql@lists.mysql.com Sent: Tuesday, March 03, 2009 5:50 PM Subject: Re: Select query locks tables in Innodb On Tue, Mar 3, 2009 at 12:35 PM, Brent Baisley brentt...@gmail.com wrote: A SELECT will/can lock a table. It almost always does in MyISAM (no insert/updates), almost never does in InnoDB. There is an exception to every rule. The problem is most likely in the 107488 rows part of the query. That's too many rows for InnoDB to keep a version history on so it's likely just locking the table. InnoDB does not do lock escalation a la SQL Server etc. I'd look at Perrin's suggestions, I think they are likely to be the problem. More importantly, what is the query? :-) -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql?unsub=c...@etrak-plus.com select * from payment_to_fee_link_budget_account_detail_link, journal_entry_master, journal_entry_type, payment_to_fee_link_event, payment_to_fee_link, fees, fees_event, fees_budget_account_detail_link, person, transactions left join regs on regs.transactions_serial = transactions.transactions_serial, transaction_event, receipt_master, budget_account_detail, budget_account_detail as ptfl_budget_account_detail, budget_account_master where journal_entry_master.organization_serial = 16 and journal_entry_master.date_effective = '2008-01-01' and journal_entry_master.date_effective '2009-03-31' and journal_entry_type.journal_entry_type_serial = journal_entry_master.journal_entry_type_serial and payment_to_fee_link_budget_account_detail_link.journal_entry_master_serial = journal_entry_master.journal_entry_master_serial and payment_to_fee_link_budget_account_detail_link.date_effective = '2008-01-01' and payment_to_fee_link_budget_account_detail_link.date_effective '2009-03-31' and payment_to_fee_link_event.payment_to_fee_link_event_serial = payment_to_fee_link_budget_account_detail_link.payment_to_fee_link_event_serial and payment_to_fee_link.payment_to_fee_link_serial = payment_to_fee_link_event.payment_to_fee_link_serial and transaction_event.transaction_event_serial = payment_to_fee_link_event.transaction_event_serial and fees.fees_serial = payment_to_fee_link.fees_serial and transactions.transactions_serial = fees.transactions_serial and person.person_serial = transactions.person_serial and receipt_master.receipt_serial = transaction_event.receipt_serial and fees_event.fees_serial = payment_to_fee_link.fees_serial and ( fees_event.transaction_event_description_serial = 13 or fees_event.transaction_event_description_serial = 2 ) and fees_budget_account_detail_link.fees_event_serial = fees_event.fees_event_serial and budget_account_detail.budget_account_detail_serial = fees_budget_account_detail_link.budget_account_detail_serial and ptfl_budget_account_detail.budget_account_detail_serial = payment_to_fee_link_budget_account_detail_link.budget_account_detail_serial and budget_account_master.budget_account_serial = budget_account_detail.budget_account_serial and budget_account_master.budget_account_type_serial = 5001 temp.XLS Description: MS-Excel spreadsheet 1 SIMPLE journal_entry_masterref PRIMARY,organization,journal_entry_type_serial,date_effective organization 4 const 56926 Using where 1 SIMPLE journal_entry_type eq_ref PRIMARY PRIMARY 4 PRODUCTION.journal_entry_master.journal_entry_type_serial 1 1 SIMPLE payment_to_fee_link_budget_account_detail_link ref journal_entry,budget_account_detail_serial,event,date_effective journal_entry 4 PRODUCTION.journal_entry_master.journal_entry_master_serial 1 Using where 1 SIMPLE ptfl_budget_account_detail eq_ref PRIMARY PRIMARY 4 PRODUCTION.payment_to_fee_link_budget_account_detail_link.budget_account_detail_serial 1 1 SIMPLE
Re: Select query locks tables in Innodb
2009/3/4 Carl c...@etrak-plus.com: However, when I had all the pieces in the query (copy attached), I could easily see it was locking tables using the Server Monitor in Navicat. I don't know what that is, but I think you'd better look at something closer to the bone, like SHOW INNODB STATUS. Explain (copy as text and copy as Excel attached) seems to indicate that it is fairly good although the first step does get quite a few rows. EXPLAIN isn't really relevant to table locking. InnoDB tables should never let readers block writers for a simple SELECT. Does anyone have any ideas? Did you check that your tables are InnoDB? Are you running some kind of crazy isolation level? - Perrin -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql?unsub=arch...@jab.org
Re: Select query locks tables in Innodb
One more note. Perrin asked if I was using any select... for update. The answer is no, neither in the select query that seems to be locking the tables nor in the queries that are processing transactions. Surprisingly, one of the tables that reports being locked is never accessed in the report query. It is a foreign key on one of the files that is used. TIA, Carl - Original Message - From: Baron Schwartz ba...@xaprb.com To: Brent Baisley brentt...@gmail.com Cc: Carl c...@etrak-plus.com; mysql@lists.mysql.com Sent: Tuesday, March 03, 2009 5:50 PM Subject: Re: Select query locks tables in Innodb On Tue, Mar 3, 2009 at 12:35 PM, Brent Baisley brentt...@gmail.com wrote: A SELECT will/can lock a table. It almost always does in MyISAM (no insert/updates), almost never does in InnoDB. There is an exception to every rule. The problem is most likely in the 107488 rows part of the query. That's too many rows for InnoDB to keep a version history on so it's likely just locking the table. InnoDB does not do lock escalation a la SQL Server etc. I'd look at Perrin's suggestions, I think they are likely to be the problem. More importantly, what is the query? :-) -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql?unsub=c...@etrak-plus.com -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql?unsub=arch...@jab.org
Re: Select query locks tables in Innodb
I did check that all tables are Innodb. I was using the Navicat Server Monitor because I know that when I see the monitor reporting a status of locked during an attempted query, that user comes to a complete halt until the lock is cleared (usually by the bad query finishing.) I will check the isolation level but I believe it is whatever was set out of the box (five years ago.) Thanks, Carl - Original Message - From: Perrin Harkins per...@elem.com To: Carl c...@etrak-plus.com Cc: mysql@lists.mysql.com Sent: Wednesday, March 04, 2009 1:49 PM Subject: Re: Select query locks tables in Innodb 2009/3/4 Carl c...@etrak-plus.com: However, when I had all the pieces in the query (copy attached), I could easily see it was locking tables using the Server Monitor in Navicat. I don't know what that is, but I think you'd better look at something closer to the bone, like SHOW INNODB STATUS. Explain (copy as text and copy as Excel attached) seems to indicate that it is fairly good although the first step does get quite a few rows. EXPLAIN isn't really relevant to table locking. InnoDB tables should never let readers block writers for a simple SELECT. Does anyone have any ideas? Did you check that your tables are InnoDB? Are you running some kind of crazy isolation level? - Perrin -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql?unsub=arch...@jab.org
Re: Select query locks tables in Innodb
Carl, Locked status in SHOW PROCESSLIST and a table being locked are different. There is a bug in MySQL that shows Locked status for queries accessing InnoDB tables in some cases. What version of MySQL are you using? The table is not really locked, you're just seeing that as a side effect of whatever's really happening. Baron On Wed, Mar 4, 2009 at 2:01 PM, Carl c...@etrak-plus.com wrote: I did check that all tables are Innodb. I was using the Navicat Server Monitor because I know that when I see the monitor reporting a status of locked during an attempted query, that user comes to a complete halt until the lock is cleared (usually by the bad query finishing.) -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql?unsub=arch...@jab.org
Re: Select query locks tables in Innodb
Carl, Locked status in SHOW PROCESSLIST and a table being locked are different. There is a bug in MySQL that shows Locked status for queries accessing InnoDB tables in some cases. What version of MySQL are you using? The table is not really locked, you're just seeing that as a side effect of whatever's really happening. Baron On Wed, Mar 4, 2009 at 2:01 PM, Carl c...@etrak-plus.com wrote: I did check that all tables are Innodb. I was using the Navicat Server Monitor because I know that when I see the monitor reporting a status of locked during an attempted query, that user comes to a complete halt until the lock is cleared (usually by the bad query finishing.) -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql?unsub=arch...@jab.org
Re: Select query locks tables in Innodb
Baron,
I am using 5.0.37.
While it may be true that there is a bug that shows tables as being locked
when they really aren't, I do not think that applies here. I do know that
when a table shows a status of 'Locked' in the Navicat Server Monitor that
the transaction which created and is processing the query comes to a
complete stop until the report query (the one I am trying to straighten out
or understand) is finished. For example, the report query is reading from
several files, e.g., receipt_master, if a user tries to check out (which
requires an insert into the receipt_master table), they are stopped until
the report query finishes and query on that table shows in Navicat as
waiting for lock ('Locked'.)
Since the report query is only reading data, I am puzzled why it locks the
tables. Any ideas?
TIA,
Carl
- Original Message -
From: Baron Schwartz ba...@xaprb.com
To: Carl c...@etrak-plus.com
Cc: mysql@lists.mysql.com
Sent: Wednesday, March 04, 2009 2:29 PM
Subject: Re: Select query locks tables in Innodb
Carl,
Locked status in SHOW PROCESSLIST and a table being locked are
different. There is a bug in MySQL that shows Locked status for
queries accessing InnoDB tables in some cases. What version of MySQL
are you using?
The table is not really locked, you're just seeing that as a side
effect of whatever's really happening.
Baron
On Wed, Mar 4, 2009 at 2:01 PM, Carl c...@etrak-plus.com wrote:
I did check that all tables are Innodb.
I was using the Navicat Server Monitor because I know that when I see the
monitor reporting a status of locked during an attempted query, that user
comes to a complete halt until the lock is cleared (usually by the bad
query
finishing.)
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/mysql?unsub=arch...@jab.org
Re: Select query locks tables in Innodb
I don't think it locks the tables. The behavior may be similar, but I
seriously doubt that's what's happening. Take a snapshot of SHOW
INNODB STATUS while this is going on. And use mysqladmin debug and
check the error log. Then put those in some pastebin and send us the
link. And realize that you've only given us bits and snippets of
information about this -- you still haven't given us SHOW CREATE TABLE
or even shown us the query that's running. There's not a lot I can do
to really help you with this other than assume that you are wrong :)
Your version is definitely affected by that bug, which I can't find --
I am using the wrong search terms and can't find the right ones to
find the bug.
5.0.37 is a very buggy version and I would upgrade regardless if I
were you, to the latest 5.0 release. You might be surprised at how
much that changes things.
Baron
On Wed, Mar 4, 2009 at 3:33 PM, Carl c...@etrak-plus.com wrote:
Baron,
I am using 5.0.37.
While it may be true that there is a bug that shows tables as being locked
when they really aren't, I do not think that applies here. I do know that
when a table shows a status of 'Locked' in the Navicat Server Monitor that
the transaction which created and is processing the query comes to a
complete stop until the report query (the one I am trying to straighten out
or understand) is finished. For example, the report query is reading from
several files, e.g., receipt_master, if a user tries to check out (which
requires an insert into the receipt_master table), they are stopped until
the report query finishes and query on that table shows in Navicat as
waiting for lock ('Locked'.)
Since the report query is only reading data, I am puzzled why it locks the
tables. Any ideas?
TIA,
Carl
- Original Message - From: Baron Schwartz ba...@xaprb.com
To: Carl c...@etrak-plus.com
Cc: mysql@lists.mysql.com
Sent: Wednesday, March 04, 2009 2:29 PM
Subject: Re: Select query locks tables in Innodb
Carl,
Locked status in SHOW PROCESSLIST and a table being locked are
different. There is a bug in MySQL that shows Locked status for
queries accessing InnoDB tables in some cases. What version of MySQL
are you using?
The table is not really locked, you're just seeing that as a side
effect of whatever's really happening.
Baron
On Wed, Mar 4, 2009 at 2:01 PM, Carl c...@etrak-plus.com wrote:
I did check that all tables are Innodb.
I was using the Navicat Server Monitor because I know that when I see the
monitor reporting a status of locked during an attempted query, that user
comes to a complete halt until the lock is cleared (usually by the bad
query
finishing.)
--
Baron Schwartz, Director of Consulting, Percona Inc.
Our Blog: http://www.mysqlperformanceblog.com/
Our Services: http://www.percona.com/services.html
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/mysql?unsub=arch...@jab.org
Select query locks tables in Innodb
I have been wrestling with this problem for a couple of weeks and have been unable to find a solution. The MySQL version is 5.0.37 and it is running on a Slackware Linux 11 box. The problem: A query that is selecting data for a report locks the files that it accesses forcing users who are attempting to enter transactions to wait until the select query is finished. The query is sizable so I have not included it here (I can if that would be helpful.) Explain shows (abbreviated): id select_typetabletypepossible keys key_len refrows Extra 1SIMPLE transactions ref PRIMARY,person,organization 4const107448 * 1SIMPLE person eq_ref PRIMARY 4person_serial1 1SIMPLE regs ref transaction 4transactions_serial 1 1SIMPLE transaction_event refPRIMARY, transaction, receipt 4transactions_serial1 1SIMPLE receipt_masterref PRIMARY 4receipt_serial1 The 107448 rows are the transactions for the organization I am reporting. The person is linked directly to the transaction. During the select query, the person table is locked thereby stopping updates to any person in the table. I have always thought a select is only a read and would, therefore, not lock any tables. Anyone have any ideas? TIA, Carl
Re: Select query locks tables in Innodb
A SELECT will/can lock a table. It almost always does in MyISAM (no insert/updates), almost never does in InnoDB. There is an exception to every rule. The problem is most likely in the 107488 rows part of the query. That's too many rows for InnoDB to keep a version history on so it's likely just locking the table. Is that how many records you want to return? That seems like a lot. Maybe reworking your query may help. Heck, post the sizeable query. You've been spending weeks on it. Brent Baisley On Tue, Mar 3, 2009 at 10:53 AM, Carl c...@etrak-plus.com wrote: I have been wrestling with this problem for a couple of weeks and have been unable to find a solution. The MySQL version is 5.0.37 and it is running on a Slackware Linux 11 box. The problem: A query that is selecting data for a report locks the files that it accesses forcing users who are attempting to enter transactions to wait until the select query is finished. The query is sizable so I have not included it here (I can if that would be helpful.) Explain shows (abbreviated): id select_type table type possible keys key_len ref rows Extra 1 SIMPLE transactions ref PRIMARY,person,organization 4 const 107448 * 1 SIMPLE person eq_ref PRIMARY 4 person_serial 1 1 SIMPLE regs ref transaction 4 transactions_serial 1 1 SIMPLE transaction_event ref PRIMARY, transaction, receipt 4 transactions_serial 1 1 SIMPLE receipt_master ref PRIMARY 4 receipt_serial 1 The 107448 rows are the transactions for the organization I am reporting. The person is linked directly to the transaction. During the select query, the person table is locked thereby stopping updates to any person in the table. I have always thought a select is only a read and would, therefore, not lock any tables. Anyone have any ideas? TIA, Carl -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql?unsub=arch...@jab.org
Re: Select query locks tables in Innodb
On Tue, Mar 3, 2009 at 10:53 AM, Carl c...@etrak-plus.com wrote: A query that is selecting data for a report locks the files that it accesses forcing users who are attempting to enter transactions to wait until the select query is finished. Is it an INSERT INTO...SELECT FROM? Those lock. Also, have you verified that each table you think is InnoDB really is? Do a SHOW CREATE TABLE on them. - Perrin -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql?unsub=arch...@jab.org
Re: Select query locks tables in Innodb
On Tue, Mar 3, 2009 at 12:35 PM, Brent Baisley brentt...@gmail.com wrote: A SELECT will/can lock a table. It almost always does in MyISAM (no insert/updates), almost never does in InnoDB. There is an exception to every rule. The problem is most likely in the 107488 rows part of the query. That's too many rows for InnoDB to keep a version history on so it's likely just locking the table. InnoDB does not do lock escalation a la SQL Server etc. I'd look at Perrin's suggestions, I think they are likely to be the problem. More importantly, what is the query? :-) -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql?unsub=arch...@jab.org
