[Yahoo-eng-team] [Bug 1350466] Re: deadlock in scheduler expire reservation periodic task
** Changed in: cinder/icehouse Status: Fix Committed = Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1350466 Title: deadlock in scheduler expire reservation periodic task Status in Cinder: Fix Released Status in Cinder icehouse series: Fix Released Status in OpenStack Compute (Nova): Fix Released Status in OpenStack Compute (nova) icehouse series: Fix Committed Bug description: http://logs.openstack.org/54/105554/4/check/gate-tempest-dsvm-neutron- large- ops/45501af/logs/screen-n-sch.txt.gz?level=TRACE#_2014-07-30_16_26_20_158 2014-07-30 16:26:20.158 17209 ERROR nova.openstack.common.periodic_task [-] Error during SchedulerManager._expire_reservations: (OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') 'UPDATE reservations SET updated_at=updated_at, deleted_at=%s, deleted=id WHERE reservations.deleted = %s AND reservations.expire %s' (datetime.datetime(2014, 7, 30, 16, 26, 20, 152098), 0, datetime.datetime(2014, 7, 30, 16, 26, 20, 149665)) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task Traceback (most recent call last): 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/periodic_task.py, line 198, in run_periodic_tasks 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task task(self, context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/scheduler/manager.py, line 157, in _expire_reservations 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task QUOTAS.expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/quota.py, line 1401, in expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task self._driver.expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/quota.py, line 651, in expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task db.reservation_expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/api.py, line 1173, in reservation_expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task return IMPL.reservation_expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/sqlalchemy/api.py, line 149, in wrapper 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task return f(*args, **kwargs) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/sqlalchemy/api.py, line 3394, in reservation_expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task reservation_query.soft_delete(synchronize_session=False) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 694, in soft_delete 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task synchronize_session=synchronize_session) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py, line 2690, in update 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task update_op.exec_() 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py, line 816, in exec_ 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task self._do_exec() 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py, line 913, in _do_exec 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task update_stmt, params=self.query._params) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 444, in _wrap 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task _raise_if_deadlock_error(e, self.bind.dialect.name) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 427, in _raise_if_deadlock_error 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task raise exception.DBDeadlock(operational_error) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task DBDeadlock: (OperationalError) (1213, 'Deadlock found when
[Yahoo-eng-team] [Bug 1350466] Re: deadlock in scheduler expire reservation periodic task
** Changed in: nova/icehouse Status: Fix Committed = Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1350466 Title: deadlock in scheduler expire reservation periodic task Status in Cinder: Fix Released Status in Cinder icehouse series: Fix Released Status in OpenStack Compute (Nova): Fix Released Status in OpenStack Compute (nova) icehouse series: Fix Released Bug description: http://logs.openstack.org/54/105554/4/check/gate-tempest-dsvm-neutron- large- ops/45501af/logs/screen-n-sch.txt.gz?level=TRACE#_2014-07-30_16_26_20_158 2014-07-30 16:26:20.158 17209 ERROR nova.openstack.common.periodic_task [-] Error during SchedulerManager._expire_reservations: (OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') 'UPDATE reservations SET updated_at=updated_at, deleted_at=%s, deleted=id WHERE reservations.deleted = %s AND reservations.expire %s' (datetime.datetime(2014, 7, 30, 16, 26, 20, 152098), 0, datetime.datetime(2014, 7, 30, 16, 26, 20, 149665)) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task Traceback (most recent call last): 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/periodic_task.py, line 198, in run_periodic_tasks 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task task(self, context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/scheduler/manager.py, line 157, in _expire_reservations 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task QUOTAS.expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/quota.py, line 1401, in expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task self._driver.expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/quota.py, line 651, in expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task db.reservation_expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/api.py, line 1173, in reservation_expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task return IMPL.reservation_expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/sqlalchemy/api.py, line 149, in wrapper 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task return f(*args, **kwargs) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/sqlalchemy/api.py, line 3394, in reservation_expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task reservation_query.soft_delete(synchronize_session=False) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 694, in soft_delete 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task synchronize_session=synchronize_session) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py, line 2690, in update 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task update_op.exec_() 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py, line 816, in exec_ 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task self._do_exec() 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py, line 913, in _do_exec 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task update_stmt, params=self.query._params) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 444, in _wrap 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task _raise_if_deadlock_error(e, self.bind.dialect.name) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 427, in _raise_if_deadlock_error 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task raise exception.DBDeadlock(operational_error) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task DBDeadlock: (OperationalError) (1213, 'Deadlock found when
[Yahoo-eng-team] [Bug 1350466] Re: deadlock in scheduler expire reservation periodic task
** Changed in: nova Status: Fix Committed = Fix Released ** Changed in: nova Milestone: None = juno-rc1 -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1350466 Title: deadlock in scheduler expire reservation periodic task Status in Cinder: Fix Released Status in Cinder icehouse series: Fix Committed Status in OpenStack Compute (Nova): Fix Released Status in OpenStack Compute (nova) icehouse series: Fix Committed Bug description: http://logs.openstack.org/54/105554/4/check/gate-tempest-dsvm-neutron- large- ops/45501af/logs/screen-n-sch.txt.gz?level=TRACE#_2014-07-30_16_26_20_158 2014-07-30 16:26:20.158 17209 ERROR nova.openstack.common.periodic_task [-] Error during SchedulerManager._expire_reservations: (OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') 'UPDATE reservations SET updated_at=updated_at, deleted_at=%s, deleted=id WHERE reservations.deleted = %s AND reservations.expire %s' (datetime.datetime(2014, 7, 30, 16, 26, 20, 152098), 0, datetime.datetime(2014, 7, 30, 16, 26, 20, 149665)) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task Traceback (most recent call last): 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/periodic_task.py, line 198, in run_periodic_tasks 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task task(self, context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/scheduler/manager.py, line 157, in _expire_reservations 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task QUOTAS.expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/quota.py, line 1401, in expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task self._driver.expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/quota.py, line 651, in expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task db.reservation_expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/api.py, line 1173, in reservation_expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task return IMPL.reservation_expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/sqlalchemy/api.py, line 149, in wrapper 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task return f(*args, **kwargs) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/sqlalchemy/api.py, line 3394, in reservation_expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task reservation_query.soft_delete(synchronize_session=False) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 694, in soft_delete 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task synchronize_session=synchronize_session) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py, line 2690, in update 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task update_op.exec_() 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py, line 816, in exec_ 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task self._do_exec() 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py, line 913, in _do_exec 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task update_stmt, params=self.query._params) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 444, in _wrap 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task _raise_if_deadlock_error(e, self.bind.dialect.name) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 427, in _raise_if_deadlock_error 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task raise exception.DBDeadlock(operational_error) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task DBDeadlock:
[Yahoo-eng-team] [Bug 1350466] Re: deadlock in scheduler expire reservation periodic task
** Also affects: cinder/icehouse Importance: Undecided Status: New ** Changed in: cinder/icehouse Importance: Undecided = High ** Changed in: cinder/icehouse Status: New = Fix Committed ** Changed in: cinder/icehouse Milestone: None = 2014.1.3 ** Changed in: cinder/icehouse Assignee: (unassigned) = Vish Ishaya (vishvananda) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1350466 Title: deadlock in scheduler expire reservation periodic task Status in Cinder: Fix Released Status in Cinder icehouse series: Fix Committed Status in OpenStack Compute (Nova): Fix Committed Bug description: http://logs.openstack.org/54/105554/4/check/gate-tempest-dsvm-neutron- large- ops/45501af/logs/screen-n-sch.txt.gz?level=TRACE#_2014-07-30_16_26_20_158 2014-07-30 16:26:20.158 17209 ERROR nova.openstack.common.periodic_task [-] Error during SchedulerManager._expire_reservations: (OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') 'UPDATE reservations SET updated_at=updated_at, deleted_at=%s, deleted=id WHERE reservations.deleted = %s AND reservations.expire %s' (datetime.datetime(2014, 7, 30, 16, 26, 20, 152098), 0, datetime.datetime(2014, 7, 30, 16, 26, 20, 149665)) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task Traceback (most recent call last): 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/periodic_task.py, line 198, in run_periodic_tasks 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task task(self, context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/scheduler/manager.py, line 157, in _expire_reservations 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task QUOTAS.expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/quota.py, line 1401, in expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task self._driver.expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/quota.py, line 651, in expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task db.reservation_expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/api.py, line 1173, in reservation_expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task return IMPL.reservation_expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/sqlalchemy/api.py, line 149, in wrapper 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task return f(*args, **kwargs) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/sqlalchemy/api.py, line 3394, in reservation_expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task reservation_query.soft_delete(synchronize_session=False) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 694, in soft_delete 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task synchronize_session=synchronize_session) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py, line 2690, in update 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task update_op.exec_() 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py, line 816, in exec_ 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task self._do_exec() 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py, line 913, in _do_exec 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task update_stmt, params=self.query._params) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 444, in _wrap 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task _raise_if_deadlock_error(e, self.bind.dialect.name) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 427, in _raise_if_deadlock_error 2014-07-30 16:26:20.158 17209 TRACE
[Yahoo-eng-team] [Bug 1350466] Re: deadlock in scheduler expire reservation periodic task
** Also affects: nova/icehouse Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1350466 Title: deadlock in scheduler expire reservation periodic task Status in Cinder: Fix Released Status in Cinder icehouse series: Fix Committed Status in OpenStack Compute (Nova): Fix Committed Status in OpenStack Compute (nova) icehouse series: New Bug description: http://logs.openstack.org/54/105554/4/check/gate-tempest-dsvm-neutron- large- ops/45501af/logs/screen-n-sch.txt.gz?level=TRACE#_2014-07-30_16_26_20_158 2014-07-30 16:26:20.158 17209 ERROR nova.openstack.common.periodic_task [-] Error during SchedulerManager._expire_reservations: (OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') 'UPDATE reservations SET updated_at=updated_at, deleted_at=%s, deleted=id WHERE reservations.deleted = %s AND reservations.expire %s' (datetime.datetime(2014, 7, 30, 16, 26, 20, 152098), 0, datetime.datetime(2014, 7, 30, 16, 26, 20, 149665)) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task Traceback (most recent call last): 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/periodic_task.py, line 198, in run_periodic_tasks 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task task(self, context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/scheduler/manager.py, line 157, in _expire_reservations 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task QUOTAS.expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/quota.py, line 1401, in expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task self._driver.expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/quota.py, line 651, in expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task db.reservation_expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/api.py, line 1173, in reservation_expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task return IMPL.reservation_expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/sqlalchemy/api.py, line 149, in wrapper 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task return f(*args, **kwargs) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/sqlalchemy/api.py, line 3394, in reservation_expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task reservation_query.soft_delete(synchronize_session=False) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 694, in soft_delete 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task synchronize_session=synchronize_session) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py, line 2690, in update 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task update_op.exec_() 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py, line 816, in exec_ 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task self._do_exec() 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py, line 913, in _do_exec 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task update_stmt, params=self.query._params) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 444, in _wrap 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task _raise_if_deadlock_error(e, self.bind.dialect.name) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 427, in _raise_if_deadlock_error 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task raise exception.DBDeadlock(operational_error) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task DBDeadlock: (OperationalError) (1213, 'Deadlock found when trying
[Yahoo-eng-team] [Bug 1350466] Re: deadlock in scheduler expire reservation periodic task
** Changed in: cinder Status: Fix Committed = Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1350466 Title: deadlock in scheduler expire reservation periodic task Status in Cinder: Fix Released Status in OpenStack Compute (Nova): In Progress Bug description: http://logs.openstack.org/54/105554/4/check/gate-tempest-dsvm-neutron- large- ops/45501af/logs/screen-n-sch.txt.gz?level=TRACE#_2014-07-30_16_26_20_158 2014-07-30 16:26:20.158 17209 ERROR nova.openstack.common.periodic_task [-] Error during SchedulerManager._expire_reservations: (OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') 'UPDATE reservations SET updated_at=updated_at, deleted_at=%s, deleted=id WHERE reservations.deleted = %s AND reservations.expire %s' (datetime.datetime(2014, 7, 30, 16, 26, 20, 152098), 0, datetime.datetime(2014, 7, 30, 16, 26, 20, 149665)) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task Traceback (most recent call last): 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/periodic_task.py, line 198, in run_periodic_tasks 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task task(self, context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/scheduler/manager.py, line 157, in _expire_reservations 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task QUOTAS.expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/quota.py, line 1401, in expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task self._driver.expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/quota.py, line 651, in expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task db.reservation_expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/api.py, line 1173, in reservation_expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task return IMPL.reservation_expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/sqlalchemy/api.py, line 149, in wrapper 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task return f(*args, **kwargs) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/sqlalchemy/api.py, line 3394, in reservation_expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task reservation_query.soft_delete(synchronize_session=False) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 694, in soft_delete 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task synchronize_session=synchronize_session) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py, line 2690, in update 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task update_op.exec_() 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py, line 816, in exec_ 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task self._do_exec() 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py, line 913, in _do_exec 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task update_stmt, params=self.query._params) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 444, in _wrap 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task _raise_if_deadlock_error(e, self.bind.dialect.name) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 427, in _raise_if_deadlock_error 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task raise exception.DBDeadlock(operational_error) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task DBDeadlock: (OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') 'UPDATE reservations SET updated_at=updated_at, deleted_at=%s, deleted=id
[Yahoo-eng-team] [Bug 1350466] Re: deadlock in scheduler expire reservation periodic task
** Also affects: cinder Importance: Undecided Status: New ** Changed in: cinder Status: New = Triaged ** Changed in: cinder Importance: Undecided = High -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1350466 Title: deadlock in scheduler expire reservation periodic task Status in Cinder: Triaged Status in OpenStack Compute (Nova): In Progress Bug description: http://logs.openstack.org/54/105554/4/check/gate-tempest-dsvm-neutron- large- ops/45501af/logs/screen-n-sch.txt.gz?level=TRACE#_2014-07-30_16_26_20_158 2014-07-30 16:26:20.158 17209 ERROR nova.openstack.common.periodic_task [-] Error during SchedulerManager._expire_reservations: (OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') 'UPDATE reservations SET updated_at=updated_at, deleted_at=%s, deleted=id WHERE reservations.deleted = %s AND reservations.expire %s' (datetime.datetime(2014, 7, 30, 16, 26, 20, 152098), 0, datetime.datetime(2014, 7, 30, 16, 26, 20, 149665)) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task Traceback (most recent call last): 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/periodic_task.py, line 198, in run_periodic_tasks 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task task(self, context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/scheduler/manager.py, line 157, in _expire_reservations 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task QUOTAS.expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/quota.py, line 1401, in expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task self._driver.expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/quota.py, line 651, in expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task db.reservation_expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/api.py, line 1173, in reservation_expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task return IMPL.reservation_expire(context) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/sqlalchemy/api.py, line 149, in wrapper 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task return f(*args, **kwargs) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/db/sqlalchemy/api.py, line 3394, in reservation_expire 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task reservation_query.soft_delete(synchronize_session=False) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 694, in soft_delete 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task synchronize_session=synchronize_session) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/query.py, line 2690, in update 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task update_op.exec_() 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py, line 816, in exec_ 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task self._do_exec() 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py, line 913, in _do_exec 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task update_stmt, params=self.query._params) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 444, in _wrap 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task _raise_if_deadlock_error(e, self.bind.dialect.name) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task File /opt/stack/new/nova/nova/openstack/common/db/sqlalchemy/session.py, line 427, in _raise_if_deadlock_error 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task raise exception.DBDeadlock(operational_error) 2014-07-30 16:26:20.158 17209 TRACE nova.openstack.common.periodic_task DBDeadlock: (OperationalError) (1213, 'Deadlock found when trying to get lock;
