** Also affects: neutron
Importance: Undecided
Status: New
** No longer affects: neutron (Ubuntu)
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1477077
Title:
Exception during message handling: QueuePool limit of size 10 overflow
20 reached, connection timed out, timeout 10 stable/kilo
Status in neutron:
New
Bug description:
OpenStack stable/kilo release, Ubuntu 14.04 devstack setup .while
multiple calls are generated during tempest run(tempest.api.networks)
in HP networking CI setup we see queuePool limit related exception
which results in tempest failure.
tempest tests that are run include
cd /opt/stack/tempest; testr init ; ./tools/pretty_tox_serial.sh \"
tempest.api.network.test_networks tempest.api.network.test_extensions
tempest.api.network.test_allowed_address_pair
tempest.api.network.test_extra_dhcp_options
tempest.api.network.test_floating_ips_negative
tempest.api.network.test_floating_ips
tempest.api.network.test_networks_negative tempest.api.network.test_networks
tempest.api.network.test_ports tempest.api.network.test_routers_negative
tempest.api.network.test_routers
tempest.api.network.test_security_groups_negative
tempest.api.network.test_security_groups
tempest.api.network.test_service_type_management
tempest.scenario.test_network_basic_ops|^test_security_groups_basic_ops|^test_server_basic_ops|^test_network_advanced_server_ops|^test_server_advanced_ops\"
"
detailed logs can be found at
http://15.126.220.115/OVSVAPP-VLAN-KILO/602/203586/9215e7309301fd72961e906a1ae5afe1d5c8f909/
might have been deleted concurrently.
2015-07-21 12:27:55.595 WARNING neutron.db.agentschedulers_db
[req-2585e7c4-8f33-439b-8b64-afc1522a686a None None] Removing network
17d6770a-c478-46d4-9556-ee77bbaf2d6b from agent
04fa72de-c1f9-4b51-8369-3b027aaf42d4 because the agent did not report to the
server in the last 150 seconds.
2015-07-21 12:28:15.822 ERROR oslo_messaging.rpc.dispatcher
[req-99ec4c7c-3f9d-4dd7-b920-c57606a07df7 None None] Exception during message
handling: QueuePool limit of size 10 overflow 20 reached, connection timed out,
timeout 10
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher Traceback (most
recent call last):
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line
142, in _dispatch_and_reply
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher
executor_callback))
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line
186, in _dispatch
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher
executor_callback)
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line
130, in _do_dispatch
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher result =
func(ctxt, **new_args)
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/opt/stack/neutron/neutron/db/agents_db.py", line 281, in report_state
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher
self.plugin.create_or_update_agent(context, agent_state)
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/opt/stack/neutron/neutron/db/agents_db.py", line 238, in
create_or_update_agent
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher return
self._create_or_update_agent(context, agent)
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/opt/stack/neutron/neutron/db/agents_db.py", line 217, in
_create_or_update_agent
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher context,
agent['agent_type'], agent['host'])
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/opt/stack/neutron/neutron/db/agents_db.py", line 193, in
_get_agent_by_type_and_host
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher Agent.host ==
host).one()
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2395, in
one
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher ret =
list(self)
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2438, in
__iter__
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher return
self._execute_and_instances(context)
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2451, in
_execute_and_instances
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher
close_with_result=True)
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2442, in
_connection_from_session
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher **kw)
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 854,
in connection
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher
close_with_result=close_with_result)
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 858,
in _connection_for_bind
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher return
self.transaction._connection_for_bind(engine)
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 322,
in _connection_for_bind
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher conn =
bind.contextual_connect()
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/compat/handle_error.py",
line 208, in contextual_connect
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher
self._wrap_pool_connect(self.pool.connect, None),
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/compat/handle_error.py",
line 215, in _wrap_pool_connect
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher return fn()
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 338, in
connect
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher return
_ConnectionFairy._checkout(self)
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 644, in
_checkout
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher fairy =
_ConnectionRecord.checkout(pool)
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 440, in
checkout
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher rec =
pool._do_get()
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher File
"/usr/local/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 959, in
_do_get
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher (self.size(),
self.overflow(), self._timeout))
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher TimeoutError:
QueuePool limit of size 10 overflow 20 reached, connection timed out, timeout 10
2015-07-21 12:28:15.822 TRACE oslo_messaging.rpc.dispatcher
2015-07-21 12:28:15.885 ERROR oslo_messaging.rpc.dispatcher
[req-6ee592e3-e5aa-4d6d-a422-3e7ca4734a25 None None] Exception during message
handling: QueuePool limit of size 10 overflow 20 reached, connection timed out,
timeout 10
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1477077/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
