[Yahoo-eng-team] [Bug 1543937] Re: db purge records fails for very large number
Reviewed: https://review.openstack.org/280571 Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=9338e5c046679d665537b4355f0279d14aedd62d Submitter: Jenkins Branch:master commit 9338e5c046679d665537b4355f0279d14aedd62d Author: Dinesh Bhor Date: Tue Feb 16 01:36:15 2016 -0800 Add check to limit maximum value of max_rows Currently 'glance-manage db purge' fails if given a very large number for max_rows. Moved and renamed validate_mysql_int() from glance.common.utils to glance.db.sqlalchemy.api as _validate_db_int() since it is related to database only so that it can be used to validate max_rows for maximum limit. Closes-Bug: #1543937 Change-Id: Id16694807c180632c1785e9b1ebe8d1c79d885ab ** Changed in: glance Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1543937 Title: db purge records fails for very large number Status in Cinder: Fix Released Status in Glance: Fix Released Status in OpenStack Compute (nova): Fix Released Bug description: The command: $ nova-manage db archive_deleted_rows --verbose fails for very large NUMBER value on nova master Nova version: openstack@openstack-136:/opt/stack/nova$ git log -1 commit 29641bd9778b51ac5794dfed9d4b881c5d47dc50 Merge: 21e79d5 9fbe683 Author: Jenkins Date: Wed Feb 10 06:03:00 2016 + Merge "Top 100 slow tests: api.openstack.compute.test_api" Example: openstack@openstack-136:~$ nova-manage db archive_deleted_rows 214748354764774747774747477536654545649 --verbose 2016-02-09 22:17:10.713 ERROR oslo_db.sqlalchemy.exc_filters [-] DBAPIError exception wrapped from (pymysql.err.ProgrammingError) (1064, u"You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '214748354764774747774747477536654545649' at line 4") [SQL: u'INSERT INTO shadow_instance_actions_events (created_at, updated_at, deleted_at, deleted, id, event, action_id, start_time, finish_time, result, traceback, host, details) SELECT instance_actions_events.created_at, instance_actions_events.updated_at, instance_actions_events.deleted_at, instance_actions_events.deleted, instance_actions_events.id, instance_actions_events.event, instance_actions_events.action_id, instance_actions_events.start_time, instance_actions_events.finish_time, instance_actions_events.result, instance_actions_events.traceback, instance_actions_events.host, instance_actions_events.details \nFROM instance_actions_events \nWHERE inst ance_actions_events.deleted != %s ORDER BY instance_actions_events.id \n LIMIT %s'] [parameters: (0, 214748354764774747774747477536654545649L)] 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters Traceback (most recent call last): 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters context) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters cursor.execute(statement, parameters) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 146, in execute 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters result = self._query(query) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 296, in _query 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters conn.query(q) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 781, in query 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters self._affected_rows = self._read_query_result(unbuffered=unbuffered) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 942, in _read_query_result 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters result.read() 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1138, in read 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters first_packet = self.connection._read_packet() 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 906, in _read_packet 2016-02-09 22:17
[Yahoo-eng-team] [Bug 1543937] Re: db purge records fails for very large number
Reviewed: https://review.openstack.org/322757 Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=25f5eed56faa81969a0e0cd6a732f789deaddb0c Submitter: Jenkins Branch:master commit 25f5eed56faa81969a0e0cd6a732f789deaddb0c Author: bhagyashris Date: Wed May 18 18:04:30 2016 +0530 Add check to limit maximum value of age_in_days If you pass age_in_days value greater than unix epoch then it raises OverflowError. Added check to ensure age_in_days is within unix epoch time to fix this problem. Removed the redundant check for age_in_days value from cinder.db.sqlalchemy.api module which is already checked at cinder.cmd.manage module. Closes-Bug: #1543937 Change-Id: Ib418627fb8527a1275c2656d1451f1e1dfeb72ef ** Changed in: cinder Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1543937 Title: db purge records fails for very large number Status in Cinder: Fix Released Status in Glance: In Progress Status in OpenStack Compute (nova): Fix Released Bug description: The command: $ nova-manage db archive_deleted_rows --verbose fails for very large NUMBER value on nova master Nova version: openstack@openstack-136:/opt/stack/nova$ git log -1 commit 29641bd9778b51ac5794dfed9d4b881c5d47dc50 Merge: 21e79d5 9fbe683 Author: Jenkins Date: Wed Feb 10 06:03:00 2016 + Merge "Top 100 slow tests: api.openstack.compute.test_api" Example: openstack@openstack-136:~$ nova-manage db archive_deleted_rows 214748354764774747774747477536654545649 --verbose 2016-02-09 22:17:10.713 ERROR oslo_db.sqlalchemy.exc_filters [-] DBAPIError exception wrapped from (pymysql.err.ProgrammingError) (1064, u"You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '214748354764774747774747477536654545649' at line 4") [SQL: u'INSERT INTO shadow_instance_actions_events (created_at, updated_at, deleted_at, deleted, id, event, action_id, start_time, finish_time, result, traceback, host, details) SELECT instance_actions_events.created_at, instance_actions_events.updated_at, instance_actions_events.deleted_at, instance_actions_events.deleted, instance_actions_events.id, instance_actions_events.event, instance_actions_events.action_id, instance_actions_events.start_time, instance_actions_events.finish_time, instance_actions_events.result, instance_actions_events.traceback, instance_actions_events.host, instance_actions_events.details \nFROM instance_actions_events \nWHERE inst ance_actions_events.deleted != %s ORDER BY instance_actions_events.id \n LIMIT %s'] [parameters: (0, 214748354764774747774747477536654545649L)] 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters Traceback (most recent call last): 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters context) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters cursor.execute(statement, parameters) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 146, in execute 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters result = self._query(query) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 296, in _query 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters conn.query(q) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 781, in query 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters self._affected_rows = self._read_query_result(unbuffered=unbuffered) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 942, in _read_query_result 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters result.read() 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1138, in read 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters first_packet = self.connection._read_packet() 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 906, in _read_packet 2016-02-09
[Yahoo-eng-team] [Bug 1543937] Re: db purge records fails for very large number
** Also affects: cinder Importance: Undecided Status: New ** Changed in: cinder Assignee: (unassigned) => Bhagyashri Shewale (bhagyashri-shewale) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1543937 Title: db purge records fails for very large number Status in Cinder: New Status in Glance: In Progress Status in OpenStack Compute (nova): Fix Released Bug description: The command: $ nova-manage db archive_deleted_rows --verbose fails for very large NUMBER value on nova master Nova version: openstack@openstack-136:/opt/stack/nova$ git log -1 commit 29641bd9778b51ac5794dfed9d4b881c5d47dc50 Merge: 21e79d5 9fbe683 Author: Jenkins Date: Wed Feb 10 06:03:00 2016 + Merge "Top 100 slow tests: api.openstack.compute.test_api" Example: openstack@openstack-136:~$ nova-manage db archive_deleted_rows 214748354764774747774747477536654545649 --verbose 2016-02-09 22:17:10.713 ERROR oslo_db.sqlalchemy.exc_filters [-] DBAPIError exception wrapped from (pymysql.err.ProgrammingError) (1064, u"You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '214748354764774747774747477536654545649' at line 4") [SQL: u'INSERT INTO shadow_instance_actions_events (created_at, updated_at, deleted_at, deleted, id, event, action_id, start_time, finish_time, result, traceback, host, details) SELECT instance_actions_events.created_at, instance_actions_events.updated_at, instance_actions_events.deleted_at, instance_actions_events.deleted, instance_actions_events.id, instance_actions_events.event, instance_actions_events.action_id, instance_actions_events.start_time, instance_actions_events.finish_time, instance_actions_events.result, instance_actions_events.traceback, instance_actions_events.host, instance_actions_events.details \nFROM instance_actions_events \nWHERE inst ance_actions_events.deleted != %s ORDER BY instance_actions_events.id \n LIMIT %s'] [parameters: (0, 214748354764774747774747477536654545649L)] 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters Traceback (most recent call last): 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters context) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters cursor.execute(statement, parameters) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 146, in execute 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters result = self._query(query) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 296, in _query 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters conn.query(q) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 781, in query 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters self._affected_rows = self._read_query_result(unbuffered=unbuffered) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 942, in _read_query_result 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters result.read() 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1138, in read 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters first_packet = self.connection._read_packet() 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 906, in _read_packet 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters packet.check_error() 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 367, in check_error 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters err.raise_mysql_exception(self._data) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 120, in raise_mysql_exception 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters _check_mysql_exception(errinfo) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/pyt
[Yahoo-eng-team] [Bug 1543937] Re: db purge records fails for very large number
Reviewed: https://review.openstack.org/278268 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=02141ae7d7af5a9e04f0c64488dcfc8aeabf82f6 Submitter: Jenkins Branch:master commit 02141ae7d7af5a9e04f0c64488dcfc8aeabf82f6 Author: Ravi Shekhar Jethani Date: Wed Feb 10 00:58:01 2016 -0800 Add check to limit maximum value of max_rows Currently archive_deleted_rows() fails if we provide a very large number as input. This patch adds a check to see that max_rows should not exceed db.MAX_INT and provides a meaningful error message to the user. Change-Id: I66a5d2cc7b7101e987ee89fcc7779bd9d5a4d144 Closes-Bug: #1543937 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1543937 Title: db purge records fails for very large number Status in Glance: In Progress Status in OpenStack Compute (nova): Fix Released Bug description: The command: $ nova-manage db archive_deleted_rows --verbose fails for very large NUMBER value on nova master Nova version: openstack@openstack-136:/opt/stack/nova$ git log -1 commit 29641bd9778b51ac5794dfed9d4b881c5d47dc50 Merge: 21e79d5 9fbe683 Author: Jenkins Date: Wed Feb 10 06:03:00 2016 + Merge "Top 100 slow tests: api.openstack.compute.test_api" Example: openstack@openstack-136:~$ nova-manage db archive_deleted_rows 214748354764774747774747477536654545649 --verbose 2016-02-09 22:17:10.713 ERROR oslo_db.sqlalchemy.exc_filters [-] DBAPIError exception wrapped from (pymysql.err.ProgrammingError) (1064, u"You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '214748354764774747774747477536654545649' at line 4") [SQL: u'INSERT INTO shadow_instance_actions_events (created_at, updated_at, deleted_at, deleted, id, event, action_id, start_time, finish_time, result, traceback, host, details) SELECT instance_actions_events.created_at, instance_actions_events.updated_at, instance_actions_events.deleted_at, instance_actions_events.deleted, instance_actions_events.id, instance_actions_events.event, instance_actions_events.action_id, instance_actions_events.start_time, instance_actions_events.finish_time, instance_actions_events.result, instance_actions_events.traceback, instance_actions_events.host, instance_actions_events.details \nFROM instance_actions_events \nWHERE inst ance_actions_events.deleted != %s ORDER BY instance_actions_events.id \n LIMIT %s'] [parameters: (0, 214748354764774747774747477536654545649L)] 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters Traceback (most recent call last): 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters context) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters cursor.execute(statement, parameters) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 146, in execute 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters result = self._query(query) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 296, in _query 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters conn.query(q) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 781, in query 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters self._affected_rows = self._read_query_result(unbuffered=unbuffered) 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 942, in _read_query_result 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters result.read() 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1138, in read 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters first_packet = self.connection._read_packet() 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 906, in _read_packet 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters packet.check_error() 2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters