[Yahoo-eng-team] [Bug 1543937] Re: db purge records fails for very large number

2016-07-30 Thread OpenStack Infra 
Reviewed:  https://review.openstack.org/280571
Committed: 
https://git.openstack.org/cgit/openstack/glance/commit/?id=9338e5c046679d665537b4355f0279d14aedd62d
Submitter: Jenkins
Branch:master

commit 9338e5c046679d665537b4355f0279d14aedd62d
Author: Dinesh Bhor 
Date:   Tue Feb 16 01:36:15 2016 -0800

Add check to limit maximum value of max_rows

Currently 'glance-manage db purge' fails if given a
very large number for max_rows.

Moved and renamed validate_mysql_int() from
glance.common.utils to glance.db.sqlalchemy.api as
_validate_db_int() since it is related to database
only so that it can be used to validate max_rows
for maximum limit.

Closes-Bug: #1543937
Change-Id: Id16694807c180632c1785e9b1ebe8d1c79d885ab


** Changed in: glance
   Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1543937

Title:
  db purge records fails for very large number

Status in Cinder:
  Fix Released
Status in Glance:
  Fix Released
Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  The command:
  $ nova-manage db archive_deleted_rows  --verbose
  fails for very large NUMBER value on nova master

  Nova version:
  openstack@openstack-136:/opt/stack/nova$ git log -1
  commit 29641bd9778b51ac5794dfed9d4b881c5d47dc50
  Merge: 21e79d5 9fbe683
  Author: Jenkins 
  Date:   Wed Feb 10 06:03:00 2016 +

  Merge "Top 100 slow tests: api.openstack.compute.test_api"

  Example:

  openstack@openstack-136:~$ nova-manage db archive_deleted_rows 
214748354764774747774747477536654545649 --verbose
  2016-02-09 22:17:10.713 ERROR oslo_db.sqlalchemy.exc_filters [-] DBAPIError 
exception wrapped from (pymysql.err.ProgrammingError) (1064, u"You have an 
error in your SQL syntax; check the manual that corresponds to your MySQL 
server version for the right syntax to use near 
'214748354764774747774747477536654545649' at line 4") [SQL: u'INSERT INTO 
shadow_instance_actions_events (created_at, updated_at, deleted_at, deleted, 
id, event, action_id, start_time, finish_time, result, traceback, host, 
details) SELECT instance_actions_events.created_at, 
instance_actions_events.updated_at, instance_actions_events.deleted_at, 
instance_actions_events.deleted, instance_actions_events.id, 
instance_actions_events.event, instance_actions_events.action_id, 
instance_actions_events.start_time, instance_actions_events.finish_time, 
instance_actions_events.result, instance_actions_events.traceback, 
instance_actions_events.host, instance_actions_events.details \nFROM 
instance_actions_events \nWHERE inst
 ance_actions_events.deleted != %s ORDER BY instance_actions_events.id \n LIMIT 
%s'] [parameters: (0, 214748354764774747774747477536654545649L)]
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters Traceback (most 
recent call last):
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, 
in _execute_context
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters context)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 
450, in do_execute
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters 
cursor.execute(statement, parameters)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 146, in 
execute
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters result = 
self._query(query)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 296, in _query
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters conn.query(q)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 781, in 
query
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters 
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 942, in 
_read_query_result
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters result.read()
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1138, in 
read
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters first_packet 
= self.connection._read_packet()
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File

[Yahoo-eng-team] [Bug 1543937] Re: db purge records fails for very large number

2016-07-26 Thread OpenStack Infra 
Reviewed:  https://review.openstack.org/322757
Committed: 
https://git.openstack.org/cgit/openstack/cinder/commit/?id=25f5eed56faa81969a0e0cd6a732f789deaddb0c
Submitter: Jenkins
Branch:master

commit 25f5eed56faa81969a0e0cd6a732f789deaddb0c
Author: bhagyashris 
Date:   Wed May 18 18:04:30 2016 +0530

Add check to limit maximum value of age_in_days

If you pass age_in_days value greater than unix epoch
then it raises OverflowError. Added check to ensure
age_in_days is within unix epoch time to fix this problem.

Removed the redundant check for age_in_days value from
cinder.db.sqlalchemy.api module which is already checked
at cinder.cmd.manage module.

Closes-Bug: #1543937
Change-Id: Ib418627fb8527a1275c2656d1451f1e1dfeb72ef


** Changed in: cinder
   Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1543937

Title:
  db purge records fails for very large number

Status in Cinder:
  Fix Released
Status in Glance:
  In Progress
Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  The command:
  $ nova-manage db archive_deleted_rows  --verbose
  fails for very large NUMBER value on nova master

  Nova version:
  openstack@openstack-136:/opt/stack/nova$ git log -1
  commit 29641bd9778b51ac5794dfed9d4b881c5d47dc50
  Merge: 21e79d5 9fbe683
  Author: Jenkins 
  Date:   Wed Feb 10 06:03:00 2016 +

  Merge "Top 100 slow tests: api.openstack.compute.test_api"

  Example:

  openstack@openstack-136:~$ nova-manage db archive_deleted_rows 
214748354764774747774747477536654545649 --verbose
  2016-02-09 22:17:10.713 ERROR oslo_db.sqlalchemy.exc_filters [-] DBAPIError 
exception wrapped from (pymysql.err.ProgrammingError) (1064, u"You have an 
error in your SQL syntax; check the manual that corresponds to your MySQL 
server version for the right syntax to use near 
'214748354764774747774747477536654545649' at line 4") [SQL: u'INSERT INTO 
shadow_instance_actions_events (created_at, updated_at, deleted_at, deleted, 
id, event, action_id, start_time, finish_time, result, traceback, host, 
details) SELECT instance_actions_events.created_at, 
instance_actions_events.updated_at, instance_actions_events.deleted_at, 
instance_actions_events.deleted, instance_actions_events.id, 
instance_actions_events.event, instance_actions_events.action_id, 
instance_actions_events.start_time, instance_actions_events.finish_time, 
instance_actions_events.result, instance_actions_events.traceback, 
instance_actions_events.host, instance_actions_events.details \nFROM 
instance_actions_events \nWHERE inst
 ance_actions_events.deleted != %s ORDER BY instance_actions_events.id \n LIMIT 
%s'] [parameters: (0, 214748354764774747774747477536654545649L)]
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters Traceback (most 
recent call last):
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, 
in _execute_context
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters context)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 
450, in do_execute
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters 
cursor.execute(statement, parameters)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 146, in 
execute
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters result = 
self._query(query)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 296, in _query
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters conn.query(q)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 781, in 
query
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters 
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 942, in 
_read_query_result
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters result.read()
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1138, in 
read
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters first_packet 
= self.connection._read_packet()
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File

[Yahoo-eng-team] [Bug 1543937] Re: db purge records fails for very large number

2016-05-17 Thread Bhagyashri Shewale 
** Also affects: cinder
   Importance: Undecided
   Status: New

** Changed in: cinder
 Assignee: (unassigned) => Bhagyashri Shewale (bhagyashri-shewale)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1543937

Title:
  db purge records fails for very large number

Status in Cinder:
  New
Status in Glance:
  In Progress
Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  The command:
  $ nova-manage db archive_deleted_rows  --verbose
  fails for very large NUMBER value on nova master

  Nova version:
  openstack@openstack-136:/opt/stack/nova$ git log -1
  commit 29641bd9778b51ac5794dfed9d4b881c5d47dc50
  Merge: 21e79d5 9fbe683
  Author: Jenkins 
  Date:   Wed Feb 10 06:03:00 2016 +

  Merge "Top 100 slow tests: api.openstack.compute.test_api"

  Example:

  openstack@openstack-136:~$ nova-manage db archive_deleted_rows 
214748354764774747774747477536654545649 --verbose
  2016-02-09 22:17:10.713 ERROR oslo_db.sqlalchemy.exc_filters [-] DBAPIError 
exception wrapped from (pymysql.err.ProgrammingError) (1064, u"You have an 
error in your SQL syntax; check the manual that corresponds to your MySQL 
server version for the right syntax to use near 
'214748354764774747774747477536654545649' at line 4") [SQL: u'INSERT INTO 
shadow_instance_actions_events (created_at, updated_at, deleted_at, deleted, 
id, event, action_id, start_time, finish_time, result, traceback, host, 
details) SELECT instance_actions_events.created_at, 
instance_actions_events.updated_at, instance_actions_events.deleted_at, 
instance_actions_events.deleted, instance_actions_events.id, 
instance_actions_events.event, instance_actions_events.action_id, 
instance_actions_events.start_time, instance_actions_events.finish_time, 
instance_actions_events.result, instance_actions_events.traceback, 
instance_actions_events.host, instance_actions_events.details \nFROM 
instance_actions_events \nWHERE inst
 ance_actions_events.deleted != %s ORDER BY instance_actions_events.id \n LIMIT 
%s'] [parameters: (0, 214748354764774747774747477536654545649L)]
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters Traceback (most 
recent call last):
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, 
in _execute_context
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters context)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 
450, in do_execute
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters 
cursor.execute(statement, parameters)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 146, in 
execute
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters result = 
self._query(query)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 296, in _query
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters conn.query(q)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 781, in 
query
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters 
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 942, in 
_read_query_result
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters result.read()
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1138, in 
read
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters first_packet 
= self.connection._read_packet()
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 906, in 
_read_packet
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters 
packet.check_error()
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 367, in 
check_error
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters 
err.raise_mysql_exception(self._data)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 120, in 
raise_mysql_exception
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters 
_check_mysql_exception(errinfo)
  2016-02-09 22:17:10.713 TRACE

[Yahoo-eng-team] [Bug 1543937] Re: db purge records fails for very large number

2016-02-27 Thread OpenStack Infra 
Reviewed:  https://review.openstack.org/278268
Committed: 
https://git.openstack.org/cgit/openstack/nova/commit/?id=02141ae7d7af5a9e04f0c64488dcfc8aeabf82f6
Submitter: Jenkins
Branch:master

commit 02141ae7d7af5a9e04f0c64488dcfc8aeabf82f6
Author: Ravi Shekhar Jethani 
Date:   Wed Feb 10 00:58:01 2016 -0800

Add check to limit maximum value of max_rows

Currently archive_deleted_rows() fails if we provide
a very large number as input. This patch adds a check
to see that max_rows should not exceed db.MAX_INT and
provides a meaningful error message to the user.

Change-Id: I66a5d2cc7b7101e987ee89fcc7779bd9d5a4d144
Closes-Bug: #1543937


** Changed in: nova
   Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1543937

Title:
  db purge records fails for very large number

Status in Glance:
  In Progress
Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  The command:
  $ nova-manage db archive_deleted_rows  --verbose
  fails for very large NUMBER value on nova master

  Nova version:
  openstack@openstack-136:/opt/stack/nova$ git log -1
  commit 29641bd9778b51ac5794dfed9d4b881c5d47dc50
  Merge: 21e79d5 9fbe683
  Author: Jenkins 
  Date:   Wed Feb 10 06:03:00 2016 +

  Merge "Top 100 slow tests: api.openstack.compute.test_api"

  Example:

  openstack@openstack-136:~$ nova-manage db archive_deleted_rows 
214748354764774747774747477536654545649 --verbose
  2016-02-09 22:17:10.713 ERROR oslo_db.sqlalchemy.exc_filters [-] DBAPIError 
exception wrapped from (pymysql.err.ProgrammingError) (1064, u"You have an 
error in your SQL syntax; check the manual that corresponds to your MySQL 
server version for the right syntax to use near 
'214748354764774747774747477536654545649' at line 4") [SQL: u'INSERT INTO 
shadow_instance_actions_events (created_at, updated_at, deleted_at, deleted, 
id, event, action_id, start_time, finish_time, result, traceback, host, 
details) SELECT instance_actions_events.created_at, 
instance_actions_events.updated_at, instance_actions_events.deleted_at, 
instance_actions_events.deleted, instance_actions_events.id, 
instance_actions_events.event, instance_actions_events.action_id, 
instance_actions_events.start_time, instance_actions_events.finish_time, 
instance_actions_events.result, instance_actions_events.traceback, 
instance_actions_events.host, instance_actions_events.details \nFROM 
instance_actions_events \nWHERE inst
 ance_actions_events.deleted != %s ORDER BY instance_actions_events.id \n LIMIT 
%s'] [parameters: (0, 214748354764774747774747477536654545649L)]
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters Traceback (most 
recent call last):
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, 
in _execute_context
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters context)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 
450, in do_execute
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters 
cursor.execute(statement, parameters)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 146, in 
execute
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters result = 
self._query(query)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 296, in _query
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters conn.query(q)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 781, in 
query
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters 
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 942, in 
_read_query_result
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters result.read()
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1138, in 
read
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters first_packet 
= self.connection._read_packet()
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters   File 
"/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 906, in 
_read_packet
  2016-02-09 22:17:10.713 TRACE oslo_db.sqlalchemy.exc_filters 
packet.check_error()