Public bug reported: Setup:
1. Loadbalancer with one listener and pool. 2. 4 webservers as members of pool. 3. create health-monitor for the pool Test: 1. Send 4 curl requests, requests are distributed equally among the members as ROUND_ROBIN lb_alorithm is selected. 2. using "nova stop <>" SHUTDOWN one VM and send 4 requests again. 3. health monitor seems to detect member down and it stopped forwarding request to that particular webserver. Problem: 1. CLI "neutron lbaas-member-list <>" says all members are admin_up stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-member-list test-lb-pool-http +--------------------------------------+-----------+---------------+--------+--------------------------------------+----------------+ | id | address | protocol_port | weight | subnet_id | admin_state_up | +--------------------------------------+-----------+---------------+--------+--------------------------------------+----------------+ | 002e68a0-03db-4f46-9c82-e35d395ada6b | 20.0.0.9 | 80 | 1 | 63cbeccd-6887-4dda-b4d2-b7503bce870a | True | | 0a97fecf-30c8-483d-8158-17523a726594 | 20.0.0.28 | 80 | 1 | 63cbeccd-6887-4dda-b4d2-b7503bce870a | True | | 348e0f42-9ab3-417e-a6e3-9857a988b6f4 | 20.0.0.29 | 80 | 1 | 63cbeccd-6887-4dda-b4d2-b7503bce870a | True | | 823d0f22-a442-4450-b7c8-0038e3f142b6 | 20.0.0.8 | 80 | 1 | 63cbeccd-6887-4dda-b4d2-b7503bce870a | True | +--------------------------------------+-----------+---------------+--------+--------------------------------------+----------------+ stack@runner:~/prash/nsbu_cqe_openstack/tools$ 2. CLI "neutron lbaas-loadbalancer-status <>" states all members are "ACTIVE" and "ONLINE". stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-loadbalancer-status test-lb { "loadbalancer": { "listeners": [ { "pools": [ { "name": "test-lb-pool-http", "provisioning_status": "ACTIVE", "healthmonitor": { "type": "HTTP", "id": "96ec79bc-d212-431b-87da-2a053064675a", "provisioning_status": "ACTIVE" }, "members": [ { "provisioning_status": "ACTIVE", "protocol_port": 80, "id": "002e68a0-03db-4f46-9c82-e35d395ada6b", "operating_status": "ONLINE", "address": "20.0.0.9" }, { "provisioning_status": "ACTIVE", "protocol_port": 80, "id": "823d0f22-a442-4450-b7c8-0038e3f142b6", "operating_status": "ONLINE", "address": "20.0.0.8" }, { "provisioning_status": "ACTIVE", "protocol_port": 80, "id": "348e0f42-9ab3-417e-a6e3-9857a988b6f4", "operating_status": "ONLINE", "address": "20.0.0.29" }, { "provisioning_status": "ACTIVE", "protocol_port": 80, "id": "0a97fecf-30c8-483d-8158-17523a726594", "operating_status": "ONLINE", "address": "20.0.0.28" } ], "id": "4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f", "operating_status": "ONLINE" } ], "provisioning_status": "ACTIVE", "name": "test-lb-http", "operating_status": "ONLINE", "id": "dedabd6e-eb51-435f-9fda-e9b90eef4108" } ], "provisioning_status": "ACTIVE", "name": "test-lb", "operating_status": "ONLINE", "id": "57814c66-731b-4dc7-abad-7d582c546873" } } stack@runner:~/prash/nsbu_cqe_openstack/tools$ Nova output: stack@runner:~/prash/nsbu_cqe_openstack/tools$ nova list +--------------------------------------+------------+---------+------------+-------------+---------------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+------------+---------+------------+-------------+---------------------------+ | 10ddd66b-3bdf-4811-97cd-28d6c94b579e | test_esx-1 | SHUTOFF | - | Shutdown | vn=20.0.0.28, 192.168.0.5 | | 93db6815-e209-4c48-bce7-308f769cf745 | test_esx-2 | ACTIVE | - | Running | vn=20.0.0.29, 192.168.0.6 | | 1a9657e6-3037-49a6-b1f3-c79b280772d3 | test_esx-3 | ACTIVE | - | Running | vn=20.0.0.32 | | 63808996-af5c-41b6-aabc-f1a7c8a6abd4 | test_esx-4 | ACTIVE | - | Running | vn=20.0.0.30 | | f5101231-0e34-49b2-81cb-a4e204b3c2d4 | test_esx-5 | ACTIVE | - | Running | vn=20.0.0.31 | | 8e743e4c-b1da-4cfd-ac47-cd2c26504cb6 | test_kvm-1 | ACTIVE | - | Running | vn=20.0.0.8, 192.168.0.2 | | 8ea23ce9-f317-4288-b345-3cb6d4483bce | test_kvm-2 | ACTIVE | - | Running | vn=20.0.0.10 | | 576f6b43-c4c4-4997-b18a-bf9375ad9e47 | test_kvm-3 | ACTIVE | - | Running | vn=20.0.0.9, 192.168.0.4 | | 755de1d3-b05d-42ed-b2ef-506d92e170a9 | test_kvm-4 | ACTIVE | - | Running | vn=20.0.0.11 | | a489472f-923e-44c4-9e25-9015bddeb3e2 | test_kvm-5 | ACTIVE | - | Running | vn=20.0.0.12 | +--------------------------------------+------------+---------+------------+-------------+---------------------------+ stack@runner:~/prash/nsbu_cqe_openstack/tools$ stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-healthmonitor-show 96ec79bc-d212-431b-87da-2a053064675a +----------------+------------------------------------------------+ | Field | Value | +----------------+------------------------------------------------+ | admin_state_up | True | | delay | 5 | | expected_codes | 200 | | http_method | GET | | id | 96ec79bc-d212-431b-87da-2a053064675a | | max_retries | 2 | | pools | {"id": "4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f"} | | tenant_id | ce1d087209c64df4b7e8007dc35def22 | | timeout | 10 | | type | HTTP | | url_path | / | +----------------+------------------------------------------------+ stack@runner:~/prash/nsbu_cqe_openstack/tools$ stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-listener-list +--------------------------------------+--------------------------------------+--------------+----------+---------------+----------------+ | id | default_pool_id | name | protocol | protocol_port | admin_state_up | +--------------------------------------+--------------------------------------+--------------+----------+---------------+----------------+ | dedabd6e-eb51-435f-9fda-e9b90eef4108 | 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f | test-lb-http | HTTP | 80 | True | +--------------------------------------+--------------------------------------+--------------+----------+---------------+----------------+ stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-listener-show dedabd6e-eb51-435f-9fda-e9b90eef4108 +---------------------------+------------------------------------------------+ | Field | Value | +---------------------------+------------------------------------------------+ | admin_state_up | True | | connection_limit | -1 | | default_pool_id | 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f | | default_tls_container_ref | | | description | | | id | dedabd6e-eb51-435f-9fda-e9b90eef4108 | | loadbalancers | {"id": "57814c66-731b-4dc7-abad-7d582c546873"} | | name | test-lb-http | | protocol | HTTP | | protocol_port | 80 | | sni_container_refs | | | tenant_id | ce1d087209c64df4b7e8007dc35def22 | +---------------------------+------------------------------------------------+ stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron pool-list Unknown command [u'pool-list'] stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-pool-list +--------------------------------------+-------------------+----------+----------------+ | id | name | protocol | admin_state_up | +--------------------------------------+-------------------+----------+----------------+ | 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f | test-lb-pool-http | HTTP | True | +--------------------------------------+-------------------+----------+----------------+ stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-pool-show 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f +---------------------+------------------------------------------------+ | Field | Value | +---------------------+------------------------------------------------+ | admin_state_up | True | | description | | | healthmonitor_id | 96ec79bc-d212-431b-87da-2a053064675a | | id | 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f | | lb_algorithm | ROUND_ROBIN | | listeners | {"id": "dedabd6e-eb51-435f-9fda-e9b90eef4108"} | | members | 002e68a0-03db-4f46-9c82-e35d395ada6b | | | 0a97fecf-30c8-483d-8158-17523a726594 | | | 348e0f42-9ab3-417e-a6e3-9857a988b6f4 | | | 823d0f22-a442-4450-b7c8-0038e3f142b6 | | name | test-lb-pool-http | | protocol | HTTP | | session_persistence | | | tenant_id | ce1d087209c64df4b7e8007dc35def22 | +---------------------+------------------------------------------------+ stack@runner:~/prash/nsbu_cqe_openstack/tools$ ** Affects: neutron Importance: Undecided Status: New ** Tags: lbaas -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1603124 Title: [stable/liberty] LBaaS v2 haproxy: pool member status shown wrongly Status in neutron: New Bug description: Setup: 1. Loadbalancer with one listener and pool. 2. 4 webservers as members of pool. 3. create health-monitor for the pool Test: 1. Send 4 curl requests, requests are distributed equally among the members as ROUND_ROBIN lb_alorithm is selected. 2. using "nova stop <>" SHUTDOWN one VM and send 4 requests again. 3. health monitor seems to detect member down and it stopped forwarding request to that particular webserver. Problem: 1. CLI "neutron lbaas-member-list <>" says all members are admin_up stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-member-list test-lb-pool-http +--------------------------------------+-----------+---------------+--------+--------------------------------------+----------------+ | id | address | protocol_port | weight | subnet_id | admin_state_up | +--------------------------------------+-----------+---------------+--------+--------------------------------------+----------------+ | 002e68a0-03db-4f46-9c82-e35d395ada6b | 20.0.0.9 | 80 | 1 | 63cbeccd-6887-4dda-b4d2-b7503bce870a | True | | 0a97fecf-30c8-483d-8158-17523a726594 | 20.0.0.28 | 80 | 1 | 63cbeccd-6887-4dda-b4d2-b7503bce870a | True | | 348e0f42-9ab3-417e-a6e3-9857a988b6f4 | 20.0.0.29 | 80 | 1 | 63cbeccd-6887-4dda-b4d2-b7503bce870a | True | | 823d0f22-a442-4450-b7c8-0038e3f142b6 | 20.0.0.8 | 80 | 1 | 63cbeccd-6887-4dda-b4d2-b7503bce870a | True | +--------------------------------------+-----------+---------------+--------+--------------------------------------+----------------+ stack@runner:~/prash/nsbu_cqe_openstack/tools$ 2. CLI "neutron lbaas-loadbalancer-status <>" states all members are "ACTIVE" and "ONLINE". stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-loadbalancer-status test-lb { "loadbalancer": { "listeners": [ { "pools": [ { "name": "test-lb-pool-http", "provisioning_status": "ACTIVE", "healthmonitor": { "type": "HTTP", "id": "96ec79bc-d212-431b-87da-2a053064675a", "provisioning_status": "ACTIVE" }, "members": [ { "provisioning_status": "ACTIVE", "protocol_port": 80, "id": "002e68a0-03db-4f46-9c82-e35d395ada6b", "operating_status": "ONLINE", "address": "20.0.0.9" }, { "provisioning_status": "ACTIVE", "protocol_port": 80, "id": "823d0f22-a442-4450-b7c8-0038e3f142b6", "operating_status": "ONLINE", "address": "20.0.0.8" }, { "provisioning_status": "ACTIVE", "protocol_port": 80, "id": "348e0f42-9ab3-417e-a6e3-9857a988b6f4", "operating_status": "ONLINE", "address": "20.0.0.29" }, { "provisioning_status": "ACTIVE", "protocol_port": 80, "id": "0a97fecf-30c8-483d-8158-17523a726594", "operating_status": "ONLINE", "address": "20.0.0.28" } ], "id": "4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f", "operating_status": "ONLINE" } ], "provisioning_status": "ACTIVE", "name": "test-lb-http", "operating_status": "ONLINE", "id": "dedabd6e-eb51-435f-9fda-e9b90eef4108" } ], "provisioning_status": "ACTIVE", "name": "test-lb", "operating_status": "ONLINE", "id": "57814c66-731b-4dc7-abad-7d582c546873" } } stack@runner:~/prash/nsbu_cqe_openstack/tools$ Nova output: stack@runner:~/prash/nsbu_cqe_openstack/tools$ nova list +--------------------------------------+------------+---------+------------+-------------+---------------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+------------+---------+------------+-------------+---------------------------+ | 10ddd66b-3bdf-4811-97cd-28d6c94b579e | test_esx-1 | SHUTOFF | - | Shutdown | vn=20.0.0.28, 192.168.0.5 | | 93db6815-e209-4c48-bce7-308f769cf745 | test_esx-2 | ACTIVE | - | Running | vn=20.0.0.29, 192.168.0.6 | | 1a9657e6-3037-49a6-b1f3-c79b280772d3 | test_esx-3 | ACTIVE | - | Running | vn=20.0.0.32 | | 63808996-af5c-41b6-aabc-f1a7c8a6abd4 | test_esx-4 | ACTIVE | - | Running | vn=20.0.0.30 | | f5101231-0e34-49b2-81cb-a4e204b3c2d4 | test_esx-5 | ACTIVE | - | Running | vn=20.0.0.31 | | 8e743e4c-b1da-4cfd-ac47-cd2c26504cb6 | test_kvm-1 | ACTIVE | - | Running | vn=20.0.0.8, 192.168.0.2 | | 8ea23ce9-f317-4288-b345-3cb6d4483bce | test_kvm-2 | ACTIVE | - | Running | vn=20.0.0.10 | | 576f6b43-c4c4-4997-b18a-bf9375ad9e47 | test_kvm-3 | ACTIVE | - | Running | vn=20.0.0.9, 192.168.0.4 | | 755de1d3-b05d-42ed-b2ef-506d92e170a9 | test_kvm-4 | ACTIVE | - | Running | vn=20.0.0.11 | | a489472f-923e-44c4-9e25-9015bddeb3e2 | test_kvm-5 | ACTIVE | - | Running | vn=20.0.0.12 | +--------------------------------------+------------+---------+------------+-------------+---------------------------+ stack@runner:~/prash/nsbu_cqe_openstack/tools$ stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-healthmonitor-show 96ec79bc-d212-431b-87da-2a053064675a +----------------+------------------------------------------------+ | Field | Value | +----------------+------------------------------------------------+ | admin_state_up | True | | delay | 5 | | expected_codes | 200 | | http_method | GET | | id | 96ec79bc-d212-431b-87da-2a053064675a | | max_retries | 2 | | pools | {"id": "4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f"} | | tenant_id | ce1d087209c64df4b7e8007dc35def22 | | timeout | 10 | | type | HTTP | | url_path | / | +----------------+------------------------------------------------+ stack@runner:~/prash/nsbu_cqe_openstack/tools$ stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-listener-list +--------------------------------------+--------------------------------------+--------------+----------+---------------+----------------+ | id | default_pool_id | name | protocol | protocol_port | admin_state_up | +--------------------------------------+--------------------------------------+--------------+----------+---------------+----------------+ | dedabd6e-eb51-435f-9fda-e9b90eef4108 | 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f | test-lb-http | HTTP | 80 | True | +--------------------------------------+--------------------------------------+--------------+----------+---------------+----------------+ stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-listener-show dedabd6e-eb51-435f-9fda-e9b90eef4108 +---------------------------+------------------------------------------------+ | Field | Value | +---------------------------+------------------------------------------------+ | admin_state_up | True | | connection_limit | -1 | | default_pool_id | 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f | | default_tls_container_ref | | | description | | | id | dedabd6e-eb51-435f-9fda-e9b90eef4108 | | loadbalancers | {"id": "57814c66-731b-4dc7-abad-7d582c546873"} | | name | test-lb-http | | protocol | HTTP | | protocol_port | 80 | | sni_container_refs | | | tenant_id | ce1d087209c64df4b7e8007dc35def22 | +---------------------------+------------------------------------------------+ stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron pool-list Unknown command [u'pool-list'] stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-pool-list +--------------------------------------+-------------------+----------+----------------+ | id | name | protocol | admin_state_up | +--------------------------------------+-------------------+----------+----------------+ | 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f | test-lb-pool-http | HTTP | True | +--------------------------------------+-------------------+----------+----------------+ stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-pool-show 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f +---------------------+------------------------------------------------+ | Field | Value | +---------------------+------------------------------------------------+ | admin_state_up | True | | description | | | healthmonitor_id | 96ec79bc-d212-431b-87da-2a053064675a | | id | 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f | | lb_algorithm | ROUND_ROBIN | | listeners | {"id": "dedabd6e-eb51-435f-9fda-e9b90eef4108"} | | members | 002e68a0-03db-4f46-9c82-e35d395ada6b | | | 0a97fecf-30c8-483d-8158-17523a726594 | | | 348e0f42-9ab3-417e-a6e3-9857a988b6f4 | | | 823d0f22-a442-4450-b7c8-0038e3f142b6 | | name | test-lb-pool-http | | protocol | HTTP | | session_persistence | | | tenant_id | ce1d087209c64df4b7e8007dc35def22 | +---------------------+------------------------------------------------+ stack@runner:~/prash/nsbu_cqe_openstack/tools$ To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1603124/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp