** Also affects: cinder
Importance: Undecided
Status: New
** Changed in: cinder
Status: New => Triaged
** Changed in: cinder
Importance: Undecided => High
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1350466
Title:
deadlock in scheduler expire reservation periodic task
Status in Cinder:
Triaged
Status in OpenStack Compute (Nova):
In Progress
Bug description:
http://logs.openstack.org/54/105554/4/check/gate-tempest-dsvm-neutron-
large-
ops/45501af/logs/screen-n-sch.txt.gz?level=TRACE#_2014-07-30_16_26_20_158
2014-07-30 16:26:20.158 17209 ERROR nova.openstack.common.periodic_task [-]
Error during SchedulerManager._expire_reservations: (OperationalError) (1213,
'Deadlock found when trying to get lock; try restarting transaction') 'UPDATE
reservations SET updated_at=updated_at, deleted_at=%s, deleted=id WHERE
reservations.deleted = %s AND reservations.expire < %s'
(datetime.datetime(2014, 7, 30, 16, 26, 20, 152098), 0, datetime.datetime(2014,
7, 30, 16, 26, 20, 149665))
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
Traceback (most recent call last):
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
File "/opt/stack/new/nova/nova/openstack/common/periodic_task.py", line 198, in
run_periodic_tasks
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
task(self, context)
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
File "/opt/stack/new/nova/nova/scheduler/manager.py", line 157, in
_expire_reservations
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
QUOTAS.expire(context)
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
File "/opt/stack/new/nova/nova/quota.py", line 1401, in expire
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
self._driver.expire(context)
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
File "/opt/stack/new/nova/nova/quota.py", line 651, in expire
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
db.reservation_expire(context)
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
File "/opt/stack/new/nova/nova/db/api.py", line 1173, in reservation_expire
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
return IMPL.reservation_expire(context)
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 149, in wrapper
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
return f(*args, **kwargs)
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
File "/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 3394, in
reservation_expire
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
reservation_query.soft_delete(synchronize_session=False)
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
File "/opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py", line
694, in soft_delete
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
synchronize_session=synchronize_session)
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2690, in
update
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
update_op.exec_()
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line
816, in exec_
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
self._do_exec()
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line
913, in _do_exec
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
update_stmt, params=self.query._params)
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
File "/opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py", line
444, in _wrap
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
_raise_if_deadlock_error(e, self.bind.dialect.name)
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
File "/opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py", line
427, in _raise_if_deadlock_error
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
raise exception.DBDeadlock(operational_error)
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
DBDeadlock: (OperationalError) (1213, 'Deadlock found when trying to get lock;
try restarting transaction') 'UPDATE reservations SET updated_at=updated_at,
deleted_at=%s, deleted=id WHERE reservations.deleted = %s AND
reservations.expire < %s' (datetime.datetime(2014, 7, 30, 16, 26, 20, 152098),
0, datetime.datetime(2014, 7, 30, 16, 26, 20, 149665))
2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task
message:"nova.openstack.common.periodic_task" AND message:"Deadlock found"
AND NOT tags:"console" AND tags:"screen-n-sch.txt"
To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1350466/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
