[Expired for neutron because there has been no activity for 60 days.]
** Changed in: neutron
Status: Incomplete => Expired
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1315495
Title:
Openvswitch throwing a table missing error?
Status in OpenStack Neutron (virtual network service):
Expired
Bug description:
I am running IceHouse with CentOS 6.4 and I followed the redhat
deployment guide for my install. It is a 3 node setup with
controller, network and compute nodes. My issue is happening on the
compute node.
My /var/log/neutron/openvswitch-agent.log file is continually throwing
this error:
2014-05-02 15:11:44.195 2003 INFO
neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent out of sync with
plugin!
2014-05-02 15:11:44.277 2003 WARNING neutron.agent.linux.ovs_lib [-] Found
failed openvswitch port: [u'qvo93d39ce1-92', [u'map', [[u'attached-mac',
u'fa:16:3e:37:26:f9'], [u'iface-id', u'93d39ce1-9209-49ee-96b5-01ff479b71c5'],
[u'iface-status', u'active'], [u'vm-uuid',
u'74187c71-8fed-4214-abb2-42c301489deb']]], -1]
2014-05-02 15:11:44.278 2003 WARNING neutron.agent.linux.ovs_lib [-] Found
failed openvswitch port: [u'qvo02722581-14', [u'map', [[u'attached-mac',
u'fa:16:3e:1f:59:19'], [u'iface-id', u'02722581-14f4-4e6f-bfd2-8fc6594440da'],
[u'iface-status', u'active'], [u'vm-uuid',
u'3402975f-6a90-45d9-86e4-0f4ed00b2762']]], -1]
2014-05-02 15:11:44.358 2003 INFO neutron.agent.securitygroups_rpc [-]
Preparing filters for devices set([u'831403fc-8d48-4949-8a4f-4c4f3455d81a',
u'fd5bb883-a415-49ec-849a-07442e3c7f1c'])
2014-05-02 15:11:44.796 2003 INFO
neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Port
fd5bb883-a415-49ec-849a-07442e3c7f1c updated. Details: {u'admin_state_up':
True, u'network_id': u'5e377240-4297-44c5-aec3-90e7db89b7a2',
u'segmentation_id': 2, u'physical_network': None, u'device':
u'fd5bb883-a415-49ec-849a-07442e3c7f1c', u'port_id':
u'fd5bb883-a415-49ec-849a-07442e3c7f1c', u'network_type': u'gre'}
2014-05-02 15:11:44.854 2003 ERROR
neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Error while processing
VIF ports
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent Traceback (most recent call
last):
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent File
"/usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py",
line 1226, in rpc_loop
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent sync =
self.process_network_ports(port_info)
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent File
"/usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py",
line 1080, in process_network_ports
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent devices_added_updated)
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent File
"/usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py",
line 985, in treat_devices_added_or_updated
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent self.context, device,
self.agent_id, cfg.CONF.host)
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent File
"/usr/lib/python2.6/site-packages/neutron/agent/rpc.py", line 107, in
update_device_up
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent topic=self.topic)
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent File
"/usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/proxy.py", line
125, in call
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent result =
rpc.call(context, real_topic, msg, timeout)
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent File
"/usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/__init__.py",
line 112, in call
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent return
_get_impl().call(CONF, context, topic, msg, timeout)
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent File
"/usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/impl_qpid.py",
line 784, in call
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent
rpc_amqp.get_connection_pool(conf, Connection))
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent File
"/usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/amqp.py", line
575, in call
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent rv = list(rv)
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent File
"/usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/amqp.py", line
540, in __iter__
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent raise result
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent RemoteError: Remote error:
OperationalError (OperationalError) no such table: ml2_port_bindings u"SELECT
ml2_port_bindings.port_id AS ml2_port_bindings_port_id, ml2_port_bindings.host
AS ml2_port_bindings_host, ml2_port_bindings.vnic_type AS
ml2_port_bindings_vnic_type, ml2_port_bindings.profile AS
ml2_port_bindings_profile, ml2_port_bindings.vif_type AS
ml2_port_bindings_vif_type, ml2_port_bindings.vif_details AS
ml2_port_bindings_vif_details, ml2_port_bindings.driver AS
ml2_port_bindings_driver, ml2_port_bindings.segment AS
ml2_port_bindings_segment \nFROM ml2_port_bindings \nWHERE
ml2_port_bindings.port_id LIKE ? || '%%'"
(u'fd5bb883-a415-49ec-849a-07442e3c7f1c',)
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent [u'Traceback (most recent
call last):\n', u' File
"/usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/amqp.py", line
462, in _process_data\n **args)\n', u' File
"/usr/lib/python2.6/site-packages/neutron/common/rpc.py", line 45, in
dispatch\n neutron_ctxt, version, method, namespace, **kwargs)\n', u' File
"/usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/dispatcher.py",
line 172, in dispatch\n result = getattr(proxyobj, method)(ctxt,
**kwargs)\n', u' File
"/usr/lib/python2.6/site-packages/neutron/plugins/ml2/rpc.py", line 196, in
update_device_up\n if (host and not plugin.port_bound_to_host(port_id,
host)):\n', u' File
"/usr/lib/python2.6/site-packages/neutron/plugins/ml2/plugin.py", line 782, in
port_bound_to_host\n port_host = db.get_port_binding_host(port_id)\n', u'
File "/usr/lib/python2.6/site-packages/neutron/plugins/ml2/db.py", line 130, in
get_p
ort_binding_host\n
filter(models.PortBinding.port_id.startswith(port_id)).\n', u' File
"/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/orm/query.py",
line 2184, in one\n ret = list(self)\n', u' File
"/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/orm/query.py",
line 2227, in __iter__\n return self._execute_and_instances(context)\n', u'
File
"/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/orm/query.py",
line 2242, in _execute_and_instances\n result =
conn.execute(querycontext.statement, self._params)\n', u' File
"/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/engine/base.py",
line 1449, in execute\n params)\n', u' File
"/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/engine/base.py",
line 1584, in _execute_clauseelement\n compiled_sql, distilled_params\n',
u' File "/usr
/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/engine/base.py",
line 1698, in _execute_context\n context)\n', u' File
"/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/engine/base.py",
line 1691, in _execute_context\n context)\n', u' File
"/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/engine/default.py",
line 331, in do_execute\n cursor.execute(statement, parameters)\n',
u'OperationalError: (OperationalError) no such table: ml2_port_bindings
u"SELECT ml2_port_bindings.port_id AS ml2_port_bindings_port_id,
ml2_port_bindings.host AS ml2_port_bindings_host, ml2_port_bindings.vnic_type
AS ml2_port_bindings_vnic_type, ml2_port_bindings.profile AS
ml2_port_bindings_profile, ml2_port_bindings.vif_type AS
ml2_port_bindings_vif_type, ml2_port_bindings.vif_details AS
ml2_port_bindings_vif_details, ml2_port_bindings.driver AS
ml2_port_bindings_driver, ml2_port_bindings.seg
ment AS ml2_port_bindings_segment \\nFROM ml2_port_bindings \\nWHERE
ml2_port_bindings.port_id LIKE ? || \'%%\'"
(u\'fd5bb883-a415-49ec-849a-07442e3c7f1c\',)\n'].
2014-05-02 15:11:44.854 2003 TRACE
neutron.plugins.openvswitch.agent.ovs_neutron_agent
2014-05-02 15:11:46.196 2003 INFO
neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent out of sync with
plugin!
2014-05-02 15:11:46.278 2003 WARNING neutron.agent.linux.ovs_lib [-] Found
failed openvswitch port: [u'qvo93d39ce1-92', [u'map', [[u'attached-mac',
u'fa:16:3e:37:26:f9'], [u'iface-id', u'93d39ce1-9209-49ee-96b5-01ff479b71c5'],
[u'iface-status', u'active'], [u'vm-uuid',
u'74187c71-8fed-4214-abb2-42c301489deb']]], -1]
2014-05-02 15:11:46.278 2003 WARNING neutron.agent.linux.ovs_lib [-] Found
failed openvswitch port: [u'qvo02722581-14', [u'map', [[u'attached-mac',
u'fa:16:3e:1f:59:19'], [u'iface-id', u'02722581-14f4-4e6f-bfd2-8fc6594440da'],
[u'iface-status', u'active'], [u'vm-uuid',
u'3402975f-6a90-45d9-86e4-0f4ed00b2762']]], -1]
2014-05-02 15:11:46.359 2003 INFO neutron.agent.securitygroups_rpc [-]
Preparing filters for devices set([u'831403fc-8d48-4949-8a4f-4c4f3455d81a',
u'fd5bb883-a415-49ec-849a-07442e3c7f1c'])
however my ml2_port_bindings table does exist and has data:
mysql> desc ml2_port_bindings
-> ;
+-------------+---------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------------+------+-----+---------+-------+
| port_id | varchar(36) | NO | PRI | NULL | |
| host | varchar(255) | NO | | NULL | |
| vnic_type | varchar(64) | NO | | NULL | |
| profile | varchar(4095) | NO | | NULL | |
| vif_type | varchar(64) | NO | | NULL | |
| vif_details | varchar(4095) | NO | | NULL | |
| driver | varchar(64) | YES | | NULL | |
| segment | varchar(36) | YES | MUL | NULL | |
+-------------+---------------+------+-----+---------+-------+
8 rows in set (0.00 sec)
mysql> select * from ml2_port_bindings;
+--------------------------------------+-----------------------+-----------+---------+----------+------------------------------------------------+-------------+--------------------------------------+
| port_id | host | vnic_type |
profile | vif_type | vif_details | driver
| segment |
+--------------------------------------+-----------------------+-----------+---------+----------+------------------------------------------------+-------------+--------------------------------------+
| 1c525688-451c-457d-bfc4-b65ebcffcc5a | | normal |
{} | unbound | | NULL
| NULL |
| 27c6fae0-f7a4-4a7d-a39f-c6b8946d2cf2 | nmtg-net001.cisco.com | normal |
{} | ovs | {"port_filter": true, "ovs_hybrid_plug": true} |
openvswitch | a661a94f-0b70-4efe-b072-9dd3b25281fa |
| 4f2d43cb-35f9-48f7-a715-6217aebf49fd | nmtg-cpu001.cisco.com | normal |
| ovs | {"port_filter": true, "ovs_hybrid_plug": true} |
openvswitch | a661a94f-0b70-4efe-b072-9dd3b25281fa |
| 831403fc-8d48-4949-8a4f-4c4f3455d81a | nmtg-cpu001.cisco.com | normal |
| ovs | {"port_filter": true, "ovs_hybrid_plug": true} |
openvswitch | a661a94f-0b70-4efe-b072-9dd3b25281fa |
| 8e24653d-1e5f-4fa1-91a4-c85434c4659b | nmtg-net001.cisco.com | normal |
{} | ovs | {"port_filter": true, "ovs_hybrid_plug": true} |
openvswitch | 9321dddb-75fb-496b-bf9f-ecf85ee202a2 |
| fd5bb883-a415-49ec-849a-07442e3c7f1c | nmtg-cpu001.cisco.com | normal |
| ovs | {"port_filter": true, "ovs_hybrid_plug": true} |
openvswitch | a661a94f-0b70-4efe-b072-9dd3b25281fa |
+--------------------------------------+-----------------------+-----------+---------+----------+------------------------------------------------+-------------+--------------------------------------+
6 rows in set (0.00 sec)
mysql>
The compute node is able to ping and connect to the neutron database
manually on the command line.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1315495/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
