Hi colleagues,
Note: errors, described below, don't have a visible impact in my current
configuration. But I draw attention on this since need to understand
whether it's expected behaviour or something wrong with something and
what exactly.
When creating loadbalancer infra (balancer, listener, pool and members)
in active/standby agents configuration, either using Neutron CLI or Heat
orchestration (below), I see the following errors in _neutron-server.log_ :
* Upon listener create (below showed a single block of related messages,
second is very similar to this; both when using either
"lbaas-listener-create" command or Heat orchestration) -
2017-11-29 15:05:29.440 1021 DEBUG neutron.api.v2.base
[req-a78d573d-6cb8-4164-afac-e37bb340640c
2a012afc274341dc81b7ea5140662e8c 413f0da1f66146ed801b1f6ced1cda48 -
default default] Request body: {u'security_group_rule': {u'direction':
u'ingress', u'protocol': 51, u'ethertype': u'IPv4', u'port_range_max':
None, u'security_group_id': u'4669fde0-5d4e-40be-b560-21173ac4561f',
u'port_range_min': None}} prepare_request_body
/usr/lib/python2.7/dist-packages/neutron/api/v2/base.py:695
2017-11-29 15:05:29.443 1021 DEBUG neutron.db.quota.driver
[req-a78d573d-6cb8-4164-afac-e37bb340640c
2a012afc274341dc81b7ea5140662e8c 413f0da1f66146ed801b1f6ced1cda48 -
default default] Resources
member,graph,subnetpool,listener,healthmonitor,l7policy have unlimited
quota limit. It is not required to calculate headroom make_reservation
/usr/lib/python2.7/dist-packages/neutron/db/quota/driver.py:223
2017-11-29 15:05:29.446 1021 DEBUG neutron.quota.resource
[req-a78d573d-6cb8-4164-afac-e37bb340640c
2a012afc274341dc81b7ea5140662e8c 413f0da1f66146ed801b1f6ced1cda48 -
default default] Usage tracker for resource:security_group_rule and
tenant:413f0da1f66146ed801b1f6ced1cda48 is out of sync, need to count
used quota count_used
/usr/lib/python2.7/dist-packages/neutron/quota/resource.py:274
2017-11-29 15:05:29.451 1021 DEBUG neutron.quota.resource
[req-a78d573d-6cb8-4164-afac-e37bb340640c
2a012afc274341dc81b7ea5140662e8c 413f0da1f66146ed801b1f6ced1cda48 -
default default] Quota usage for security_group_rule was recalculated.
Used quota:24. count_used
/usr/lib/python2.7/dist-packages/neutron/quota/resource.py:293
2017-11-29 15:05:29.454 1021 DEBUG neutron.db.quota.driver
[req-a78d573d-6cb8-4164-afac-e37bb340640c
2a012afc274341dc81b7ea5140662e8c 413f0da1f66146ed801b1f6ced1cda48 -
default default] Attempting to reserve 1 items for resource
security_group_rule. Total usage: 24; quota limit: 100; headroom:76
make_reservation
/usr/lib/python2.7/dist-packages/neutron/db/quota/driver.py:255
2017-11-29 15:05:29.613 1021 DEBUG neutron_lib.callbacks.manager
[req-a78d573d-6cb8-4164-afac-e37bb340640c
2a012afc274341dc81b7ea5140662e8c 413f0da1f66146ed801b1f6ced1cda48 -
default default] Notify callbacks [] for security_group_rule,
before_create _notify_loop
/usr/lib/python2.7/dist-packages/neutron_lib/callbacks/manager.py:167
2017-11-29 15:05:29.721 1021 INFO neutron.api.v2.resource
[req-a78d573d-6cb8-4164-afac-e37bb340640c
2a012afc274341dc81b7ea5140662e8c 413f0da1f66146ed801b1f6ced1cda48 -
default default] *create failed (client error): There was a conflict
when trying to complete your request.*
2017-11-29 15:05:29.722 1021 INFO neutron.wsgi
[req-a78d573d-6cb8-4164-afac-e37bb340640c
2a012afc274341dc81b7ea5140662e8c 413f0da1f66146ed801b1f6ced1cda48 -
default default] 10.0.10.10 *"POST /v2.0/security-group-rules HTTP/1.1"*
status: 409 len: 347 time: 0.2877431*
*
* Upon creation of pool members (2 members), these errors appear only
when creating infra using Heat (no errors when using neutron
"lbaas-member-create" CLI command ) :
2017-11-29 12:04:30.229 1018 DEBUG neutron.api.v2.base
[req-e3d66438-da0c-4b68-be6c-d47b3bce55b1
e6406606bd9d48aabc413468f9703cf6 c1114776e144400da17d8e060856be8c -
default default] Request body: {u'member': {u'subnet_id':
u'ecb891c1-b7e5-45e0-8815-8675381d70d2', u'protocol_port': 8080,
u'admin_state_up': True, u'weight': 1, u'address': u'10.1.1.11'}}
prepare_request_body
/usr/lib/python2.7/dist-packages/neutron/api/v2/base.py:695
2017-11-29 12:04:30.232 1018 DEBUG neutron.db.quota.driver
[req-e3d66438-da0c-4b68-be6c-d47b3bce55b1
e6406606bd9d48aabc413468f9703cf6 c1114776e144400da17d8e060856be8c -
default default] Resources
member,graph,subnetpool,listener,healthmonitor,l7policy have unlimited
quota limit. It is not required to calculate headroom make_reservation
/usr/lib/python2.7/dist-packages/neutron/db/quota/driver.py:223
2017-11-29 12:04:30.690 1018 INFO neutron.api.v2.resource
[req-e3d66438-da0c-4b68-be6c-d47b3bce55b1
e6406606bd9d48aabc413468f9703cf6 c1114776e144400da17d8e060856be8c -
default default] *create failed (client error): There was a conflict
when trying to complete your request.*
2017-11-29 12:04:30.691 1018 INFO neutron.wsgi
[req-e3d66438-da0c-4b68-be6c-d47b3bce55b1
e6406606bd9d48aabc413468f9703cf6 c1114776e144400da17d8e060856be8c -
default default] 10.0.10.10 *"POST
/v2.0/lbaas/pools/e106e039-af27-4cfa-baa2-7238acd3078e/members
HTTP/1.1"* status: 409 len: 342 time: 0.4688179*
*
2017-11-29 12:04:31.704 1018 DEBUG neutron.api.v2.base
[req-f8f012ba-885d-4eeb-a3ec-f8a664328926
e6406606bd9d48aabc413468f9703cf6 c1114776e144400da17d8e060856be8c -
default default] Request body: {u'member': {u'subnet_id':
u'ecb891c1-b7e5-45e0-8815-8675381d70d2', u'protocol_port': 8080,
u'admin_state_up': True, u'weight': 1, u'address': u'10.1.1.11'}}
prepare_request_body
/usr/lib/python2.7/dist-packages/neutron/api/v2/base.py:695
2017-11-29 12:04:31.709 1018 DEBUG neutron.db.quota.driver
[req-f8f012ba-885d-4eeb-a3ec-f8a664328926
e6406606bd9d48aabc413468f9703cf6 c1114776e144400da17d8e060856be8c -
default default] Resources
member,graph,subnetpool,listener,healthmonitor,l7policy have unlimited
quota limit. It is not required to calculate headroom make_reservation
/usr/lib/python2.7/dist-packages/neutron/db/quota/driver.py:223
2017-11-29 12:04:32.268 1018 INFO neutron.api.v2.resource
[req-f8f012ba-885d-4eeb-a3ec-f8a664328926
e6406606bd9d48aabc413468f9703cf6 c1114776e144400da17d8e060856be8c -
default default] *create failed (client error): There was a conflict
when trying to complete your request.*
2017-11-29 12:04:32.269 1018 INFO neutron.wsgi
[req-f8f012ba-885d-4eeb-a3ec-f8a664328926
e6406606bd9d48aabc413468f9703cf6 c1114776e144400da17d8e060856be8c -
default default] 10.0.10.10 *"POST
/v2.0/lbaas/pools/e106e039-af27-4cfa-baa2-7238acd3078e/members
HTTP/1.1"* status: 409 len: 342 time: 0.5732501*
*
2017-11-29 12:04:33.283 1018 DEBUG neutron.api.v2.base
[req-48d0074b-07dd-48cc-a7e9-096b5bccb0c4
e6406606bd9d48aabc413468f9703cf6 c1114776e144400da17d8e060856be8c -
default default] Request body: {u'member': {u'subnet_id':
u'ecb891c1-b7e5-45e0-8815-8675381d70d2', u'protocol_port': 8080,
u'admin_state_up': True, u'weight': 1, u'address': u'10.1.1.11'}}
prepare_request_body
/usr/lib/python2.7/dist-packages/neutron/api/v2/base.py:695
2017-11-29 12:04:33.286 1018 DEBUG neutron.db.quota.driver
[req-48d0074b-07dd-48cc-a7e9-096b5bccb0c4
e6406606bd9d48aabc413468f9703cf6 c1114776e144400da17d8e060856be8c -
default default] Resources
member,graph,subnetpool,listener,healthmonitor,l7policy have unlimited
quota limit. It is not required to calculate headroom make_reservation
/usr/lib/python2.7/dist-packages/neutron/db/quota/driver.py:223
2017-11-29 12:04:33.646 1018 INFO neutron.api.v2.resource
[req-48d0074b-07dd-48cc-a7e9-096b5bccb0c4
e6406606bd9d48aabc413468f9703cf6 c1114776e144400da17d8e060856be8c -
default default] *create failed (client error): There was a conflict
when trying to complete your request.*
2017-11-29 12:04:33.647 1018 INFO neutron.wsgi
[req-48d0074b-07dd-48cc-a7e9-096b5bccb0c4
e6406606bd9d48aabc413468f9703cf6 c1114776e144400da17d8e060856be8c -
default default] 10.0.10.10 *"POST
/v2.0/lbaas/pools/e106e039-af27-4cfa-baa2-7238acd3078e/members
HTTP/1.1"* status: 409 len: 342 time: 0.3708489*
*
2017-11-29 12:04:34.659 1018 DEBUG neutron.api.v2.base
[req-ffde5c98-d157-4835-ba87-a01897a5a9b7
e6406606bd9d48aabc413468f9703cf6 c1114776e144400da17d8e060856be8c -
default default] Request body: {u'member': {u'subnet_id':
u'ecb891c1-b7e5-45e0-8815-8675381d70d2', u'protocol_port': 8080,
u'admin_state_up': True, u'weight': 1, u'address': u'10.1.1.11'}}
prepare_request_body
/usr/lib/python2.7/dist-packages/neutron/api/v2/base.py:695
2017-11-29 12:04:34.663 1018 DEBUG neutron.db.quota.driver
[req-ffde5c98-d157-4835-ba87-a01897a5a9b7
e6406606bd9d48aabc413468f9703cf6 c1114776e144400da17d8e060856be8c -
default default] Resources
member,graph,subnetpool,listener,healthmonitor,l7policy have unlimited
quota limit. It is not required to calculate headroom make_reservation
/usr/lib/python2.7/dist-packages/neutron/db/quota/driver.py:223
2017-11-29 12:04:35.015 1018 INFO neutron.api.v2.resource
[req-ffde5c98-d157-4835-ba87-a01897a5a9b7
e6406606bd9d48aabc413468f9703cf6 c1114776e144400da17d8e060856be8c -
default default] *create failed (client error): There was a conflict
when trying to complete your request.*
2017-11-29 12:04:35.016 1018 INFO neutron.wsgi
[req-ffde5c98-d157-4835-ba87-a01897a5a9b7
e6406606bd9d48aabc413468f9703cf6 c1114776e144400da17d8e060856be8c -
default default] 10.0.10.10 *"POST
/v2.0/lbaas/pools/e106e039-af27-4cfa-baa2-7238acd3078e/members
HTTP/1.1"* status: 409 len: 342 time: 0.3637619*
*
Errors above have corresponding errors in _heat-engine.log_ (there are
two members (pm1 and pm2) in pool and no errors regarding second (pm2)
pool member):
2017-11-29 12:04:28.927 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] starting start
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:182
2017-11-29 12:04:28.927 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] running step
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:214
2017-11-29 12:04:28.928 6286 INFO heat.engine.resource
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default]
creating PoolMember "pm1" Stack "nbt" [b8beca77-19c7-49e5-94a7-ec079d841277]
2017-11-29 12:04:30.095 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] sleeping _sleep
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:155
2017-11-29 12:04:30.096 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] running step
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:214
*2017-11-29 12:04:30.692 6286 DEBUG neutronclient.v2_0.client
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default]
Error message: {"NeutronError": {"message": "Invalid state
PENDING_UPDATE of loadbalancer resource
db8ae876-b6eb-4c45-95d1-33e0ca6193de", "type": "StateInvalid", "detail":
""}} _handle_fault_response
/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:258*
2017-11-29 12:04:30.692 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] sleeping _sleep
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:155
2017-11-29 12:04:31.693 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] running step
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:214
*2017-11-29 12:04:32.270 6286 DEBUG neutronclient.v2_0.client
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default]
Error message: {"NeutronError": {"message": "Invalid state
PENDING_UPDATE of loadbalancer resource
db8ae876-b6eb-4c45-95d1-33e0ca6193de", "type": "StateInvalid", "detail":
""}} _handle_fault_response
/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:258*
2017-11-29 12:04:32.271 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] sleeping _sleep
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:155
2017-11-29 12:04:33.272 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] running step
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:214
*2017-11-29 12:04:33.648 6286 DEBUG neutronclient.v2_0.client
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default]
Error message: {"NeutronError": {"message": "Invalid state
PENDING_UPDATE of loadbalancer resource
db8ae876-b6eb-4c45-95d1-33e0ca6193de", "type": "StateInvalid", "detail":
""}} _handle_fault_response
/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:258*
2017-11-29 12:04:33.648 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] sleeping _sleep
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:155
2017-11-29 12:04:34.649 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] running step
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:214
*2017-11-29 12:04:35.016 6286 DEBUG neutronclient.v2_0.client
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default]
Error message: {"NeutronError": {"message": "Invalid state
PENDING_UPDATE of loadbalancer resource
db8ae876-b6eb-4c45-95d1-33e0ca6193de", "type": "StateInvalid", "detail":
""}} _handle_fault_response
/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:258*
2017-11-29 12:04:35.017 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] sleeping _sleep
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:155
2017-11-29 12:04:36.017 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] running step
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:214
2017-11-29 12:04:38.763 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] sleeping _sleep
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:155
2017-11-29 12:04:39.763 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] running step
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:214
2017-11-29 12:04:39.891 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] sleeping _sleep
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:155
2017-11-29 12:04:40.892 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] running step
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:214
2017-11-29 12:04:41.013 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] sleeping _sleep
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:155
2017-11-29 12:04:42.013 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] running step
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:214
2017-11-29 12:04:42.277 6286 DEBUG heat.engine.scheduler
[req-346cc302-ec69-4781-9afb-1dc1474c6ebc - bush - default default] Task
create from PoolMember "pm1" Stack "nbt"
[b8beca77-19c7-49e5-94a7-ec079d841277] complete step
/usr/lib/python2.7/dist-packages/heat/engine/scheduler.py:220
Heat template for loadbalancer is the following:
balancer:
type: OS::Neutron::LBaaS::LoadBalancer
properties:
name: nbt-balancer
vip_subnet: { get_resource: lan-subnet }
listener:
type: OS::Neutron::LBaaS::Listener
properties:
name: nbt-listener
protocol: TCP
protocol_port: { get_param: lb_port }
loadbalancer: { get_resource: balancer }
pool:
type: OS::Neutron::LBaaS::Pool
properties:
name: nbt-pool
protocol: TCP
lb_algorithm: ROUND_ROBIN
listener: { get_resource: listener }
pm1:
type: OS::Neutron::LBaaS::PoolMember
properties:
address: { get_attr: [ n1, first_address ]}
pool: { get_resource: pool }
protocol_port: { get_param: pool_port }
subnet: { get_resource: lan-subnet }
pm2:
type: OS::Neutron::LBaaS::PoolMember
properties:
address: { get_attr: [ n2, first_address ]}
pool: { get_resource: pool }
protocol_port: { get_param: pool_port }
subnet: { get_resource: lan-subnet }
and, of course, servers n1 and n2 are exist and are operational.
I will appreciate if you'll take a look at the issue and give some
feedback on this. I can provide any related information in order to
clarify this issue.
Thank you.
--
Volodymyr Litovka
"Vision without Execution is Hallucination." -- Thomas Edison
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
