from:"Corey Wright"

[Yahoo-eng-team] [Bug 1619028] [NEW] test_get_free_namespace_port fails on systems without sysctl

2016-08-31 Thread Corey Wright

Public bug reported:

If the unit test test_get_free_namespace_port runs on a system without
sysctl, then the test fails as it tries to execute sysctl, because
sysctl doesn't exist.

==
FAIL: 
neutron.tests.unit.tests.common.test_net_helpers.PortAllocationTestCase.test_get_free_namespace_port
neutron.tests.unit.tests.common.test_net_helpers.PortAllocationTestCase.test_get_free_namespace_port
--
_StringException: Empty attachments:
  pythonlogging:''
  stderr
  stdout

Traceback (most recent call last):
  File "neutron/tests/unit/tests/common/test_net_helpers.py", line 65, in 
test_get_free_namespace_port
n_const.PROTO_NAME_TCP)
  File "neutron/tests/common/net_helpers.py", line 215, in 
get_free_namespace_port
return get_unused_port(used_ports)
  File "neutron/tests/common/net_helpers.py", line 186, in get_unused_port
['sysctl', '-n', 'net.ipv4.ip_local_port_range'])
  File "neutron/agent/linux/utils.py", line 117, in execute
addl_env=addl_env)
  File "neutron/agent/linux/utils.py", line 87, in create_process
stderr=subprocess.PIPE)
  File "neutron/common/utils.py", line 85, in subprocess_popen
close_fds=close_fds, env=env)
  File 
"/var/lib/jenkins/workspace/test-neutron-rax-patches-change/virtualenv/local/lib/python2.7/site-packages/eventlet/green/subprocess.py",
 line 53, in __init__
subprocess_orig.Popen.__init__(self, args, 0, *argss, **kwds)
  File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1259, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

Code under test shouldn't be allowed to execute local commands for multiple 
reasons:
0. a test system is not a deployment system
1. the command might not exist
2. the command's output might be system-dependent
3. the command might unintentionally alter the state of the system

The execution of sysctl should be mocked.

** Affects: neutron
 Importance: Undecided
 Assignee: Corey Wright (coreywright)
 Status: In Progress

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1619028

Title:
  test_get_free_namespace_port fails on systems without sysctl

Status in neutron:
  In Progress

Bug description:
  If the unit test test_get_free_namespace_port runs on a system without
  sysctl, then the test fails as it tries to execute sysctl, because
  sysctl doesn't exist.

  ==
  FAIL: 
neutron.tests.unit.tests.common.test_net_helpers.PortAllocationTestCase.test_get_free_namespace_port
  
neutron.tests.unit.tests.common.test_net_helpers.PortAllocationTestCase.test_get_free_namespace_port
  --
  _StringException: Empty attachments:
pythonlogging:''
stderr
stdout

  Traceback (most recent call last):
File "neutron/tests/unit/tests/common/test_net_helpers.py", line 65, in 
test_get_free_namespace_port
  n_const.PROTO_NAME_TCP)
File "neutron/tests/common/net_helpers.py", line 215, in 
get_free_namespace_port
  return get_unused_port(used_ports)
File "neutron/tests/common/net_helpers.py", line 186, in get_unused_port
  ['sysctl', '-n', 'net.ipv4.ip_local_port_range'])
File "neutron/agent/linux/utils.py", line 117, in execute
  addl_env=addl_env)
File "neutron/agent/linux/utils.py", line 87, in create_process
  stderr=subprocess.PIPE)
File "neutron/common/utils.py", line 85, in subprocess_popen
  close_fds=close_fds, env=env)
File 
"/var/lib/jenkins/workspace/test-neutron-rax-patches-change/virtualenv/local/lib/python2.7/site-packages/eventlet/green/subprocess.py",
 line 53, in __init__
  subprocess_orig.Popen.__init__(self, args, 0, *argss, **kwds)
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
  errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1259, in _execute_child
  raise child_exception
  OSError: [Errno 2] No such file or directory

  Code under test shouldn't be allowed to execute local commands for multiple 
reasons:
  0. a test system is not a deployment system
  1. the command might not exist
  2. the command's output might be system-dependent
  3. the command might unintentionally alter the state of the system

  The execution of sysctl should be mocked.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1619028/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1566622] [NEW] live migration fails with xenapi virt driver and SRs with old-style naming convention

2016-04-05 Thread Corey Wright

Public bug reported:

version: commit ce5a2fb419f999bec0fb2c67413387c8b67a691a

1. create a boot-from-volume instance prior to deploying commit
5bd222e8d854ca7f03ee6936454ee57e0d6e1a78
2. upgrade nova to commit 5bd222e8d854ca7f03ee6936454ee57e0d6e1a78
3. live-migrate instance
4. observe live-migrate action fail

based on my analysis of logs and code:
1. destination uses new-style SR naming convention in sr_uuid_map.
2. source tries to use new-style SR naming convention in talking to XenAPI (in
nova.virt.xenapi.vmops.py:VMOps.live_migrate() -> _call_live_migrate_command())
3. xenapi throws XenAPI.Failure exception because it "Got exception
UUID_INVALID" because it only knows the SR by the old-style naming convention

example destination nova-compute, source nova-compute, and xenapi logs
from a live-migrate request to follow.

** Affects: nova
Importance: Undecided
Status: New

** Tags: live-migrate nova xenapi

** Attachment added: "destination nova-compute, source nova-compute, and xenapi
logs from a live-migrate request"

https://bugs.launchpad.net/bugs/1566622/+attachment/4625560/+files/live-migrate_fails_with_old_and_new-style_sr_naming_convention.log

--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1566622

Title:
live migration fails with xenapi virt driver and SRs with old-style
naming convention

Status in OpenStack Compute (nova):
New

Bug description:
version: commit ce5a2fb419f999bec0fb2c67413387c8b67a691a

based on my analysis of logs and code:
1. destination uses new-style SR naming convention in sr_uuid_map.
2. source tries to use new-style SR naming convention in talking to XenAPI
(in nova.virt.xenapi.vmops.py:VMOps.live_migrate() ->
_call_live_migrate_command())
3. xenapi throws XenAPI.Failure exception because it "Got exception
UUID_INVALID" because it only knows the SR by the old-style naming convention

example destination nova-compute, source nova-compute, and xenapi logs
from a live-migrate request to follow.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1566622/+subscriptions

--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1550795] [NEW] live migration with block migration on Xen produces "cannot marshal" exception

2016-02-27 Thread Corey Wright

Public bug reported:

1. Nova version

commit ce5a2fb419f999bec0fb2c67413387c8b67a691a

2. Deserialized traceback as captured in log as received by messaging

Traceback (most recent call last):
  File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
 line 143, in _dispatch_and_reply
executor_callback))
  File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
 line 189, in _dispatch
executor_callback)
  File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
 line 130, in _do_dispatch
result = func(ctxt, **new_args)
  File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/nova/exception.py",
 line 110, in wrapped
payload)
  File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/oslo_utils/excutils.py",
 line 220, in __exit__
self.force_reraise()
  File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/oslo_utils/excutils.py",
 line 196, in force_reraise
six.reraise(self.type_, self.value, self.tb)
  File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/nova/exception.py",
 line 89, in wrapped
return f(self, context, *args, **kw)
  File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/nova/compute/manager.py",
 line 413, in decorated_function
return function(self, context, *args, **kwargs)
  File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/nova/compute/manager.py",
 line 391, in decorated_function
kwargs['instance'], e, sys.exc_info())
  File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/oslo_utils/excutils.py",
 line 220, in __exit__
self.force_reraise()
  File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/oslo_utils/excutils.py",
 line 196, in force_reraise
six.reraise(self.type_, self.value, self.tb)
  File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/nova/compute/manager.py",
 line 379, in decorated_function
return function(self, context, *args, **kwargs)
  File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/nova/compute/manager.py",
 line 5281, in check_can_live_migrate_source
block_device_info)
  File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/nova/virt/xenapi/driver.py",
 line 530, in check_can_live_migrate_source
dest_check_data)
  File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/nova/virt/xenapi/vmops.py",
 line 2419, in check_can_live_migrate_source
"VM.assert_can_migrate", vm_ref, dest_check_data)
  File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/nova/virt/xenapi/vmops.py",
 line 2524, in _call_live_migrate_command
vdi_map, vif_map, options)
  File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/nova/virt/xenapi/client/session.py",
 line 210, in call_xenapi
return session.xenapi_request(method, args)
  File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/XenAPI.py", 
line 133, in xenapi_request
result = _parse_result(getattr(self, methodname)(*full_params))
  File "/usr/lib/python2.7/xmlrpclib.py", line 1224, in __call__
return self.__send(self.__name, args)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1572, in __request
allow_none=self.__allow_none)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1085, in dumps
data = m.dumps(params)
  File "/usr/lib/python2.7/xmlrpclib.py", line 632, in dumps
dump(v, write)
  File "/usr/lib/python2.7/xmlrpclib.py", line 652, in __dump
raise TypeError, "cannot marshal %s objects" % type(value)
TypeError: cannot marshal  
objects

3. Reproduction steps

* nova boot bug-xmlrpc_marshall_exception
* nova live-migration --block-migrate bug-xmlrpc_marshall_exception

** Affects: nova
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1550795

Title:
  live migration with block migration on Xen produces "cannot marshal"
  exception

Status in OpenStack Compute (nova):
  New

Bug description:
  1. Nova version

  commit ce5a2fb419f999bec0fb2c67413387c8b67a691a

  2. Deserialized traceback as captured in log as received by messaging

  Traceback (most recent call last):
File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
 line 143, in _dispatch_and_reply
  executor_callback))
File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
 line 189, in _dispatch
  executor_callback)
File 
"/opt/rackstack/rackstack.532.2/nova/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
 line 130, in _do_dispatch
  result = func(ctxt, **new_args)
File 
"/opt/

[Yahoo-eng-team] [Bug 1540682] [NEW] InstanceNotFound during instance usage audit

2016-02-01 Thread Corey Wright

Public bug reported:

tl;dr set read_deleted to "yes" on the admin context used by the
periodic _instance_usage_audit() so info can be retrieved on deleted
instances.

Explanation:

nova.compute.manager.Manager._instance_usage_audit() has a bug where,
though get_active_by_window_joined() properly returned a list of
instances including deleted ones (because it's implicit that
read_deleted must be set to "yes" for that function to correctly
operate), notify_usage_exists() indirectly uses the more generic
get_flavor() (which cannot assume read_deleted should be "yes" to
properly operate and therefor defaults to "no") to report on each
instance without setting read_deleted to "yes".  this was masked until
commit 1337890a was merged which removed compatibility code that
retrieved the flavor if InstanceNotFound was encountered and the
instance was deleted.

Nova version: git HEAD since commit 1337890a

Log excerpt:

2016-01-14 00:01:51.112 18707 DEBUG nova.objects.instance 
[req-66e4a380-ed24-4ca1-a971-ee1d80878e2e - - - - -] Lazy-loading `flavor' on 
Instance uuid 85388355-6f83-4bca-9a11-a0914a2e3a3a obj_load_attr 
/opt/rackstack/rackstack.497.5/nova/lib/python2.7/site-packages/nova/objects/instance.py:843
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager 
[req-66e4a380-ed24-4ca1-a971-ee1d80878e2e - - - - -] [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a] Failed to generate usage audit for 
instance on host c-10-13-135-245
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a] Traceback (most recent call last):
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a]   File 
"/opt/rackstack/rackstack.497.5/nova/lib/python2.7/site-packages/nova/compute/manager.py",
 line 6036, in _instance_usage_audit
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a] include_bandwidth=True)
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a]   File 
"/opt/rackstack/rackstack.497.5/nova/lib/python2.7/site-packages/nova/compute/utils.py",
 line 297, in notify_usage_exists
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a] system_metadata=system_metadata, 
extra_usage_info=extra_info)
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a]   File 
"/opt/rackstack/rackstack.497.5/nova/lib/python2.7/site-packages/nova/compute/utils.py",
 line 317, in notify_about_instance_usage
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a] network_info, system_metadata, 
**extra_usage_info)
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a]   File 
"/opt/rackstack/rackstack.497.5/nova/lib/python2.7/site-packages/nova/notifications.py",
 line 386, in info_from_instance
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a] instance_type = instance.get_flavor()
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a]   File 
"/opt/rackstack/rackstack.497.5/nova/lib/python2.7/site-packages/nova/objects/instance.py",
 line 871, in get_flavor
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a] return getattr(self, attr)
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a]   File 
"/opt/rackstack/rackstack.497.5/nova/lib/python2.7/site-packages/oslo_versionedobjects/base.py",
 line 67, in getter
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a] self.obj_load_attr(name)
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a]   File 
"/opt/rackstack/rackstack.497.5/nova/lib/python2.7/site-packages/nova/objects/instance.py",
 line 861, in obj_load_attr
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a] self._load_flavor()
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a]   File 
"/opt/rackstack/rackstack.497.5/nova/lib/python2.7/site-packages/nova/objects/instance.py",
 line 754, in _load_flavor
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a] expected_attrs=['flavor', 
'system_metadata'])
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a11-a0914a2e3a3a]   File 
"/opt/rackstack/rackstack.497.5/nova/lib/python2.7/site-packages/oslo_versionedobjects/base.py",
 line 179, in wrapper
2016-01-14 00:01:51.181 18707 ERROR nova.compute.manager [instance: 
85388355-6f83-4bca-9a1

[Yahoo-eng-team] [Bug 1516174] [NEW] nova-cells in api cells generate redundant instance.update notifications

2015-11-13 Thread Corey Wright

Public bug reported:

Commit eaaa6593 causes instance updates in compute cells to generate
redundant notifications by nova-cells in the api cell.  This can
overwhelm notification consumers as instance.update notifications double
in quantity, especially over short time periods like during multiple
simultaneous instance creations.

That commit used the Instance.skip_cells_sync() contextmanager with
Instance.save() in instance_update_at_top() to prevent update cycles
between api and compute cells, but the notification in Instance.save()
is not guarded by _sync_cells, so notifications generated in compute
cells are also generated in api cells.

** Affects: nova
 Importance: Undecided
 Status: New


** Tags: cells

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1516174

Title:
  nova-cells in api cells generate redundant instance.update
  notifications

Status in OpenStack Compute (nova):
  New

Bug description:
  Commit eaaa6593 causes instance updates in compute cells to generate
  redundant notifications by nova-cells in the api cell.  This can
  overwhelm notification consumers as instance.update notifications
  double in quantity, especially over short time periods like during
  multiple simultaneous instance creations.

  That commit used the Instance.skip_cells_sync() contextmanager with
  Instance.save() in instance_update_at_top() to prevent update cycles
  between api and compute cells, but the notification in Instance.save()
  is not guarded by _sync_cells, so notifications generated in compute
  cells are also generated in api cells.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1516174/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1483132] [NEW] ssh-keygen-to-Paramiko change breaks third-party tools

2015-08-09 Thread Corey Wright

Public bug reported:

Changing ssh key generation from OpenSSH's ssh-keygen to the Paramiko
library [1][2] changed (unintentionally?) the ASN.1 encoding format of
SSH private keys from DER to BER.  (DER is a strict subset of BER, so
anything that can read BER can read DER, but not necessarily the other
way around.)

Some third-party tools only support DER and this has created at least
one issue [3] (specifically because Go's standard library only supports
DER).

I have provided Paramiko with a small change that makes its SSH private
key output equal to OpenSSH's ssh-keygen output (and presumably DER
formatted) [4].

Providing a change to Paramiko is just one method of addressing this
backwards-incompatibility and interoperability issue.  Should the
Paramiko change be accepted the unit test output vectors will need to be
changed, but should it not, is a reversion of or modification to Nova
acceptable to maintain backwards-compatibility and interoperability?

[1] https://review.openstack.org/157931
[2] 
http://git.openstack.org/cgit/openstack/nova/commit/?id=3f3f9bf22efd2fb209d2a2fe0246f4857cd2d21a
[3] https://github.com/mitchellh/packer/issues/2526
[4] https://github.com/paramiko/paramiko/pull/572

** Affects: nova
 Importance: Undecided
 Status: New

** Summary changed:

- ssh-keygen-to-paramiko change breaks third-party tools
+ ssh-keygen-to-Paramiko change breaks third-party tools

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1483132

Title:
  ssh-keygen-to-Paramiko change breaks third-party tools

Status in OpenStack Compute (nova):
  New

Bug description:
  Changing ssh key generation from OpenSSH's ssh-keygen to the Paramiko
  library [1][2] changed (unintentionally?) the ASN.1 encoding format of
  SSH private keys from DER to BER.  (DER is a strict subset of BER, so
  anything that can read BER can read DER, but not necessarily the other
  way around.)

  Some third-party tools only support DER and this has created at least
  one issue [3] (specifically because Go's standard library only
  supports DER).

  I have provided Paramiko with a small change that makes its SSH
  private key output equal to OpenSSH's ssh-keygen output (and
  presumably DER formatted) [4].

  Providing a change to Paramiko is just one method of addressing this
  backwards-incompatibility and interoperability issue.  Should the
  Paramiko change be accepted the unit test output vectors will need to
  be changed, but should it not, is a reversion of or modification to
  Nova acceptable to maintain backwards-compatibility and
  interoperability?

  [1] https://review.openstack.org/157931
  [2] 
http://git.openstack.org/cgit/openstack/nova/commit/?id=3f3f9bf22efd2fb209d2a2fe0246f4857cd2d21a
  [3] https://github.com/mitchellh/packer/issues/2526
  [4] https://github.com/paramiko/paramiko/pull/572

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1483132/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1404390] [NEW] Python 2.7.9 breaks nova.tests.unit.test_wsgi.TestWSGIServerWithSSL.test_app_using_ipv6_and_ssl

2014-12-19 Thread Corey Wright

Public bug reported:

nova git version is 2014.2-1545-gd442187 (ie master as of now).

Python 2.7.9, with its securing the network by default (PEP 466), causes
the test to fail:

URLError: 

if i pass the test ca file through urlopen(), for academic purposes as
it's not backwards compatible, i find that the server cert is ipv4 only:

CertificateError: hostname '::1' doesn't match u'0.0.0.0'

a new certificate is needed, but only the ca's public cert is provided,
so the old ca is useless for signing a new ipv4 & ipv6 certificate.

if i create a new ca and a new certificate and switch to the responses
python package (to enable ssl verification and full 2.7.x
compatibility), then everything works and i have
https://review.openstack.org/143072.

** Affects: nova
 Importance: Undecided
 Status: New


** Tags: 2.7.9 ipv6 python ssl testing wsgi

** Attachment added: "certificate verify failed traceback"
   
https://bugs.launchpad.net/bugs/1404390/+attachment/4284395/+files/certificate_verify_failed.txt

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1404390

Title:
  Python 2.7.9 breaks
  nova.tests.unit.test_wsgi.TestWSGIServerWithSSL.test_app_using_ipv6_and_ssl

Status in OpenStack Compute (Nova):
  New

Bug description:
  nova git version is 2014.2-1545-gd442187 (ie master as of now).

  Python 2.7.9, with its securing the network by default (PEP 466),
  causes the test to fail:

  URLError: 

  if i pass the test ca file through urlopen(), for academic purposes as
  it's not backwards compatible, i find that the server cert is ipv4
  only:

  CertificateError: hostname '::1' doesn't match u'0.0.0.0'

  a new certificate is needed, but only the ca's public cert is
  provided, so the old ca is useless for signing a new ipv4 & ipv6
  certificate.

  if i create a new ca and a new certificate and switch to the responses
  python package (to enable ssl verification and full 2.7.x
  compatibility), then everything works and i have
  https://review.openstack.org/143072.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1404390/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1358881] [NEW] jjsonschema 2.3.0 -> 2.4.0 upgrade breaking nova.tests.test_api_validation tests

2014-08-19 Thread Corey Wright

Public bug reported:

the following two failures appeared after upgrading jsonschema to 2.4.0.
downgrading to 2.3.0 returned the tests to passing.

==
FAIL: 
nova.tests.test_api_validation.TcpUdpPortTestCase.test_validate_tcp_udp_port_fails
--
Traceback (most recent call last):
_StringException: Empty attachments:
  pythonlogging:''
  stderr
  stdout

Traceback (most recent call last):
  File "/home/dev/Desktop/nova-test/nova/tests/test_api_validation.py", line 
602, in test_validate_tcp_udp_port_fails
expected_detail=detail)
  File "/home/dev/Desktop/nova-test/nova/tests/test_api_validation.py", line 
31, in check_validation_error
self.assertEqual(ex.kwargs, expected_kwargs)
  File 
"/home/dev/Desktop/nova-test/.venv/local/lib/python2.7/site-packages/testtools/testcase.py",
 line 321, in assertEqual
self.assertThat(observed, matcher, message)
  File 
"/home/dev/Desktop/nova-test/.venv/local/lib/python2.7/site-packages/testtools/testcase.py",
 line 406, in assertThat
raise mismatch_error
MismatchError: !=:
reference = {'code': 400,
 'detail': u'Invalid input for field/attribute foo. Value: 65536. 65536 is 
greater than the maximum of 65535'}
actual= {'code': 400,
 'detail': 'Invalid input for field/attribute foo. Value: 65536. 65536.0 is 
greater than the maximum of 65535'}


==
FAIL: 
nova.tests.test_api_validation.IntegerRangeTestCase.test_validate_integer_range_fails
--
Traceback (most recent call last):
_StringException: Empty attachments:
  stderr
  stdout

pythonlogging:'': {{{
INFO [migrate.versioning.api] 215 -> 216... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 216 -> 217... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 217 -> 218... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 218 -> 219... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 219 -> 220... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 220 -> 221... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 221 -> 222... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 222 -> 223... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 223 -> 224... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 224 -> 225... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 225 -> 226... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 226 -> 227... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 227 -> 228... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 228 -> 229... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 229 -> 230... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 230 -> 231... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 231 -> 232... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 232 -> 233... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 233 -> 234... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 234 -> 235... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 235 -> 236... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 236 -> 237... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 237 -> 238... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 238 -> 239... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 239 -> 240... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 240 -> 241... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 241 -> 242... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 242 -> 243... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 243 -> 244... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 244 -> 245... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 245 -> 246... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 246 -> 247... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 247 -> 248... 
INFO [248_add_expire_reservations_index] Skipped adding 
reservations_deleted_expire_idx because an equivalent index already exists.
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 248 -> 249... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 249 -> 250... 
INFO [migrate.versioning.api] done
}}}

Traceback (most recent call last):
  File "/home/dev/Desktop/nova-test/nova/tests/test_api_validation.py", line 
361, in test_validate_integer_range_fails
expected_detail=detail)
  File "/home/dev/Desktop/nova-test/nova/tests/test_api_validation.py", line

[Yahoo-eng-team] [Bug 1356687] [NEW] hacking check for jsonutils produces pep8 traceback

2014-08-13 Thread Corey Wright

Public bug reported:

the new jsonutils hacking check produces a pep8 traceback because it
returns a set (column offset and error text) instead of an iterable (as
logical line checks, like this check, should).

commit 243879f5c51fc45f03491bcb78765945ddf76be8
Change-Id: I86ed6cd3316dd4da5e1b10b36a3ddba3739316d3

= 8< = TEST CASE = 8< =
$ echo 'foo = json.dumps(bar)' >nova/foobar.py
$ flake8 -vv nova/foobar.py
local configuration: in /home/dev/Desktop/nova-test
  ignore = E121,E122,E123,E124,E125,E126,E127,E128,E129,E131,E251,H405,H803,H904
  exclude = 
.venv,.git,.tox,dist,doc,*openstack/common*,*lib/python*,*egg,build,tools
checking nova/foobar.py
foo = json.dumps(bar)
Traceback (most recent call last):
  File "/home/dev/Desktop/nova-test/.venv/bin/flake8", line 9, in 
load_entry_point('flake8==2.1.0', 'console_scripts', 'flake8')()
  File 
"/home/dev/Desktop/nova-test/.venv/local/lib/python2.7/site-packages/flake8/main.py",
 line 32, in main
report = flake8_style.check_files()
  File 
"/home/dev/Desktop/nova-test/.venv/local/lib/python2.7/site-packages/pep8.py", 
line 1672, in check_files
runner(path)
  File 
"/home/dev/Desktop/nova-test/.venv/local/lib/python2.7/site-packages/flake8/engine.py",
 line 73, in input_file
return fchecker.check_all(expected=expected, line_offset=line_offset)
  File 
"/home/dev/Desktop/nova-test/.venv/local/lib/python2.7/site-packages/pep8.py", 
line 1436, in check_all
self.check_logical()
  File 
"/home/dev/Desktop/nova-test/.venv/local/lib/python2.7/site-packages/pep8.py", 
line 1338, in check_logical
for offset, text in self.run_check(check, argument_names) or ():
TypeError: 'int' object is not iterable
= 8< = TEST CASE = 8< =


diff --git a/nova/hacking/checks.py b/nova/hacking/checks.py
index a1dd614..7fe7412 100644
--- a/nova/hacking/checks.py
+++ b/nova/hacking/checks.py
@@ -300,7 +300,7 @@ def use_jsonutils(logical_line, filename):
 for f in json_funcs:
 pos = logical_line.find('json.%s' % f)
 if pos != -1:
-return (pos, msg % {'fun': f})
+yield (pos, msg % {'fun': f})
 
 
 def factory(register):
= 8< = PATCH = 8< =

it's late, so tomorrow, if there hasn't been any activity on this, then
i'll submit a patch for review.

** Affects: nova
 Importance: Undecided
 Status: New

** Patch added: "patch to make use_jsonutils() return an iterable by way of a 
yield statement as the pep8 api requires"
   
https://bugs.launchpad.net/bugs/1356687/+attachment/4177110/+files/jsonutils_hacking_check.patch

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1356687

Title:
  hacking check for jsonutils produces pep8 traceback

Status in OpenStack Compute (Nova):
  New

Bug description:
  the new jsonutils hacking check produces a pep8 traceback because it
  returns a set (column offset and error text) instead of an iterable
  (as logical line checks, like this check, should).

  commit 243879f5c51fc45f03491bcb78765945ddf76be8
  Change-Id: I86ed6cd3316dd4da5e1b10b36a3ddba3739316d3

  = 8< = TEST CASE = 8< =
  $ echo 'foo = json.dumps(bar)' >nova/foobar.py
  $ flake8 -vv nova/foobar.py
  local configuration: in /home/dev/Desktop/nova-test
ignore = 
E121,E122,E123,E124,E125,E126,E127,E128,E129,E131,E251,H405,H803,H904
exclude = 
.venv,.git,.tox,dist,doc,*openstack/common*,*lib/python*,*egg,build,tools
  checking nova/foobar.py
  foo = json.dumps(bar)
  Traceback (most recent call last):
File "/home/dev/Desktop/nova-test/.venv/bin/flake8", line 9, in 
  load_entry_point('flake8==2.1.0', 'console_scripts', 'flake8')()
File 
"/home/dev/Desktop/nova-test/.venv/local/lib/python2.7/site-packages/flake8/main.py",
 line 32, in main
  report = flake8_style.check_files()
File 
"/home/dev/Desktop/nova-test/.venv/local/lib/python2.7/site-packages/pep8.py", 
line 1672, in check_files
  runner(path)
File 
"/home/dev/Desktop/nova-test/.venv/local/lib/python2.7/site-packages/flake8/engine.py",
 line 73, in input_file
  return fchecker.check_all(expected=expected, line_offset=line_offset)
File 
"/home/dev/Desktop/nova-test/.venv/local/lib/python2.7/site-packages/pep8.py", 
line 1436, in check_all
  self.check_logical()
File 
"/home/dev/Desktop/nova-test/.venv/local/lib/python2.7/site-packages/pep8.py", 
line 1338, in check_logical
  for offset, text in self.run_check(check, argument_names) or ():
  TypeError: 'int' object is not iterable
  = 8< = TEST CASE = 8< =

  
  diff --git a/nova/hacking/checks.py b/nova/hacking/checks.py
  index a1dd614..7fe7412 100644
  --- a/nova/hacking/checks.py
  +++ b/nova/hacking/checks.py
  @@ -300,7 +300,7 @@ def use_jsonutils(logical_line, filename):
   for f in json_funcs:
   pos = logical_line.fin

[Yahoo-eng-team] [Bug 1342498] [NEW] test_live_migration_* tests fail with fakelibvirt

2014-07-15 Thread Corey Wright

Public bug reported:

With the recent merging of https://review.openstack.org/73428,
test_live_migration_(changes_listen_addresses|raises_exception) requires
fakelibvirt to have the migrateToURI2() method (if libvirt, at least
v0.9.2, is not installed) so that it can be mocked out by those tests,
otherwise the following mox errors occur during unit testing.

UnknownMethodCallError: Method called is not a member of the object:
migrateToURI2

patch attached (which i may submit properly in the next 24 hours if
someone doesn't beat me to it),  with signature taken from upstream
libvirt and mocking stolen from fakelibvirt's migrateToURI().

full error log:

$ ./run_tests.sh 
nova.tests.virt.libvirt.test_driver.LibvirtConnTestCase.test_live_migration_*
Running `tools/with_venv.sh python -m nova.openstack.common.lockutils python 
setup.py testr --testr-args='--subunit --concurrency 0  
nova.tests.virt.libvirt.test_driver.LibvirtConnTestCase.test_live_migration_*'`
nova.tests.virt.libvirt.test_driver.LibvirtConnTestCase
test_live_migration_changes_listen_addresses  FAIL
test_live_migration_uses_migrateToURI_without_dest_listen_addrs   OK  0.27
test_live_migration_raises_exception  FAIL
test_live_migration_fails_without_migratable_flag_or_0_addr   OK  3.32
test_live_migration_uses_migrateToURI_without_migratable_flag OK  0.43

Slowest 5 tests took 8.80 secs:
nova.tests.virt.libvirt.test_driver.LibvirtConnTestCase
test_live_migration_changes_listen_addresses  2.10
test_live_migration_fails_without_migratable_flag_or_0_addr   3.32
test_live_migration_raises_exception  2.68
test_live_migration_uses_migrateToURI_without_dest_listen_addrs   0.27
test_live_migration_uses_migrateToURI_without_migratable_flag 0.43

==
FAIL: 
nova.tests.virt.libvirt.test_driver.LibvirtConnTestCase.test_live_migration_changes_listen_addresses
--
Traceback (most recent call last):
_StringException: Empty attachments:
  stderr
  stdout

pythonlogging:'': {{{
INFO [migrate.versioning.api] 215 -> 216... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 216 -> 217... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 217 -> 218... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 218 -> 219... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 219 -> 220... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 220 -> 221... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 221 -> 222... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 222 -> 223... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 223 -> 224... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 224 -> 225... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 225 -> 226... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 226 -> 227... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 227 -> 228... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 228 -> 229... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 229 -> 230... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 230 -> 231... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 231 -> 232... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 232 -> 233... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 233 -> 234... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 234 -> 235... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 235 -> 236... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 236 -> 237... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 237 -> 238... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 238 -> 239... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 239 -> 240... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 240 -> 241... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 241 -> 242... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 242 -> 243... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 243 -> 244... 
INFO [migrate.versioning.api] done
INFO [migrate.versioning.api] 244 -> 245... 
INFO [migrate.versioning.api] done
INFO [nova.network.driver] Loading network driver 'nova.network.linux_net'
INFO [nova.virt.driver] Loading compute driver 'nova.virt.fake.FakeDriver'
}}}

Traceback (most recent call last):
  File 
"/home/dev/Desktop/nova/.venv/local/lib/python2.7/site-packages/mock.py", line 
1201, in patched
return func(*args, **keywargs)
  File

[Yahoo-eng-team] [Bug 1342464] [NEW] PEP 8 H305 test unpassable on both Python 2.6 & 2.7