[Yahoo-eng-team] [Bug 1729741] Re: os-security-groups api call creates api call explosion to neutron

2017-11-13 Thread Robert van Leeuwen 
Re-opened this bug: I am not sure the fix did anything.
I applied the fix and still see the exact same thing in the neutron logs, e.g.:


2017-11-13 09:55:49,914.914 10778 INFO neutron.wsgi 
[req-4c18bd00-48c2-498f-92ee-824e215b83b9 c70f3fc161e04718a108cf8192d0816e 
d2930aef1c824f048b8e1301b3ded161 - - -] 10.41.0.33 - - [13/Nov/2017 09:55:49] 
"GET /v2.0/security-groups/d2770f7f-8695-48de-806b-e690189e25a8 HTTP/1.1" 200 
41170 0.062934
2017-11-13 09:55:49,982.982 10778 INFO neutron.wsgi 
[req-f69f7d30-6121-456e-8dec-4c142965e91c c70f3fc161e04718a108cf8192d0816e 
d2930aef1c824f048b8e1301b3ded161 - - -] 10.41.0.33 - - [13/Nov/2017 09:55:49] 
"GET /v2.0/security-groups/d2770f7f-8695-48de-806b-e690189e25a8 HTTP/1.1" 200 
41170 0.055435
2017-11-13 09:55:50,054.054 10778 INFO neutron.wsgi 
[req-4a389783-31b1-421e-a0f9-b31e419bc5df c70f3fc161e04718a108cf8192d0816e 
d2930aef1c824f048b8e1301b3ded161 - - -] 10.41.0.33 - - [13/Nov/2017 09:55:50] 
"GET /v2.0/security-groups/d2770f7f-8695-48de-806b-e690189e25a8 HTTP/1.1" 200 
41170 0.058556
2017-11-13 09:55:50,119.119 10778 INFO neutron.wsgi 
[req-3dffc0a6-5f14-400f-8b4c-02555d2e262d c70f3fc161e04718a108cf8192d0816e 
d2930aef1c824f048b8e1301b3ded161 - - -] 10.41.0.33 - - [13/Nov/2017 09:55:50] 
"GET /v2.0/security-groups/d2770f7f-8695-48de-806b-e690189e25a8 HTTP/1.1" 200 
41170 0.052321
2017-11-13 09:55:50,198.198 10778 INFO neutron.wsgi 
[req-007398bc-72e9-4858-8b04-f49face2304d c70f3fc161e04718a108cf8192d0816e 
d2930aef1c824f048b8e1301b3ded161 - - -] 10.41.0.33 - - [13/Nov/2017 09:55:50] 
"GET /v2.0/security-groups/d2770f7f-8695-48de-806b-e690189e25a8 HTTP/1.1" 200 
41170 0.066602
2017-11-13 09:55:50,270.270 10778 INFO neutron.wsgi 
[req-092f3234-926b-4f2c-8626-1b3e7f1203dd c70f3fc161e04718a108cf8192d0816e 
d2930aef1c824f048b8e1301b3ded161 - - -] 10.41.0.33 - - [13/Nov/2017 09:55:50] 
"GET /v2.0/security-groups/d2770f7f-8695-48de-806b-e690189e25a8 HTTP/1.1" 200 
41170 0.059341
2017-11-13 09:55:50,334.334 10778 INFO neutron.wsgi 
[req-20f45327-9d24-482c-b648-176e85d46a81 c70f3fc161e04718a108cf8192d0816e 
d2930aef1c824f048b8e1301b3ded161 - - -] 10.41.0.33 - - [13/Nov/2017 09:55:50] 
"GET /v2.0/security-groups/d2770f7f-8695-48de-806b-e690189e25a8 HTTP/1.1" 200 
41170 0.051137
2017-11-13 09:55:50,397.397 10778 INFO neutron.wsgi 
[req-0fa86b48-59a0-460c-829e-ea008a442ed0 c70f3fc161e04718a108cf8192d0816e 
d2930aef1c824f048b8e1301b3ded161 - - -] 10.41.0.33 - - [13/Nov/2017 09:55:50] 
"GET /v2.0/security-groups/d2770f7f-8695-48de-806b-e690189e25a8 HTTP/1.1" 200 
41170 0.050534
2017-11-13 09:55:50,459.459 10778 INFO neutron.wsgi 
[req-8bb7bfdc-b12a-46b3-a76d-7ba299d45a84 c70f3fc161e04718a108cf8192d0816e 
d2930aef1c824f048b8e1301b3ded161 - - -] 10.41.0.33 - - [13/Nov/2017 09:55:50] 
"GET /v2.0/security-groups/d2770f7f-8695-48de-806b-e690189e25a8 HTTP/1.1" 200 
41170 0.048582
2017-11-13 09:55:50,527.527 10778 INFO neutron.wsgi 
[req-f19d8910-967c-4236-91d3-a59852482383 c70f3fc161e04718a108cf8192d0816e 
d2930aef1c824f048b8e1301b3ded161 - - -] 10.41.0.33 - - [13/Nov/2017 09:55:50] 
"GET /v2.0/security-groups/d2770f7f-8695-48de-806b-e690189e25a8 HTTP/1.1" 200 
41170 0.054999
2017-11-13 09:55:50,596.596 10778 INFO neutron.wsgi 
[req-f21e1335-ddd7-49b0-9f2b-da9d37e48c8b c70f3fc161e04718a108cf8192d0816e 
d2930aef1c824f048b8e1301b3ded161 - - -] 10.41.0.33 - - [13/Nov/2017 09:55:50] 
"GET /v2.0/security-groups/d2770f7f-8695-48de-806b-e690189e25a8 HTTP/1.1" 200 
41170 0.056300

** Changed in: nova
   Status: Fix Released => Confirmed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1729741

Title:
  os-security-groups api call creates api call explosion to neutron

Status in OpenStack Compute (nova):
  Confirmed
Status in OpenStack Compute (nova) pike series:
  Confirmed

Bug description:
  1) create a security group
  2) create a bunch of security group rules which reference a security group 
instead of a CIDR e.g.
  openstack security group rule create --remote-group x-1123--xxx-x

  
  When querying nova api /os-security-groups there will be an API call to 
neutron for each rule that has a remote group attached.

  In the logs you will seee GET /v2.0/security-groups/x-1123--xxx-x
  Creating rules with a CIDR do not have this issue.

  As you can imagine this will quickly get very slow.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1729741/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1729741] Re: os-security-groups api call creates api call explosion to neutron

2017-11-09 Thread OpenStack Infra 
Reviewed:  https://review.openstack.org/517648
Committed: 
https://git.openstack.org/cgit/openstack/nova/commit/?id=dc658dbdcf2325cf6f27d9ae61d85b835f8410f8
Submitter: Zuul
Branch:master

commit dc658dbdcf2325cf6f27d9ae61d85b835f8410f8
Author: Matt Riedemann 
Date:   Fri Nov 3 12:26:03 2017 -0400

Avoid redundant security group queries in GET 
/servers/{id}/os-security-groups

The GET /servers/{server_id}/os-security-groups API code can
perform poorly if the instance has several security groups and
each security group has several rules. This is because when processing
the output, we loop over the groups, and loop over the rules per group,
and then for each rule, if it has a group_id specified, we query
the security group details (from Neutron in most cases).

If more than one rule points at the same group_id, we're doing a redundant
group lookup and sending more traffic to the security group API (aka 
Neutron)
than needed.

This change optimizes that single API to load the rule group details
up front so that we only do at most one lookup per group_id.

This could be extended to GET /os-security-groups but that API is
deprecated so any optimization there is lower priority.

Change-Id: Ia451429f61b15526fade6838386e562c17591d36
Closes-Bug: #1729741


** Changed in: nova
   Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1729741

Title:
  os-security-groups api call creates api call explosion to neutron

Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) pike series:
  Confirmed

Bug description:
  1) create a security group
  2) create a bunch of security group rules which reference a security group 
instead of a CIDR e.g.
  openstack security group rule create --remote-group x-1123--xxx-x

  
  When querying nova api /os-security-groups there will be an API call to 
neutron for each rule that has a remote group attached.

  In the logs you will seee GET /v2.0/security-groups/x-1123--xxx-x
  Creating rules with a CIDR do not have this issue.

  As you can imagine this will quickly get very slow.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1729741/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1729741] Re: os-security-groups api call creates api call explosion to neutron

2017-11-03 Thread Matt Riedemann 
** Also affects: nova/pike
   Importance: Undecided
   Status: New

** Changed in: nova/pike
   Status: New => Confirmed

** Changed in: nova/pike
   Importance: Undecided => Medium

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1729741

Title:
  os-security-groups api call creates api call explosion to neutron

Status in OpenStack Compute (nova):
  In Progress
Status in OpenStack Compute (nova) pike series:
  Confirmed

Bug description:
  1) create a security group
  2) create a bunch of security group rules which reference a security group 
instead of a CIDR e.g.
  openstack security group rule create --remote-group x-1123--xxx-x

  
  When querying nova api /os-security-groups there will be an API call to 
neutron for each rule that has a remote group attached.

  In the logs you will seee GET /v2.0/security-groups/x-1123--xxx-x
  Creating rules with a CIDR do not have this issue.

  As you can imagine this will quickly get very slow.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1729741/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp