Re: [Openstack] [Pike][Neutron] L3 metering with DVR doesn't work

[Brian Haley]

On 10/25/2018 08:06 AM, Alexandru Sorodoc wrote:

Hello,

I'm trying to set up metering for neutron in Pike. I tested it with a
centralized router and it works, but when I try with a distributed router it
doesn't record any usage samples. I have one compute node and one 
network node

and I've created an instance with a floating ip.


The metering agent isn't very well maintained, and I don't see any open 
bugs similar to this issue.  The only thing I can remember is this 
abandoned change regarding traffic counters for DVR routers - 
https://review.openstack.org/#/c/486493/ but there was no follow-on from 
the author.


The best thing to do would be to try and reproduce it on the master 
branch (or Rocky) and file a bug.


> I think this is because the router is running on network1. Why is it
> running on
> network1 and why does it seem that the l3 agent on compute1 does the 
actual

> routing?

The compute node will do all the routing when a floating IP is 
associated, the router on network1 is for default snat traffic when 
there is no floating IP and the instance tries to communicate out the 
external network.


-Brian



openstack router show public-router2
+-++
| Field   | 
Value  |

+-++
| admin_state_up  | 
UP |
| availability_zone_hints 
|    |
| availability_zones  | 
nova   |
| created_at  | 
2018-10-05T12:07:32Z   |

| description |    |
| distributed | 
True   |
| external_gateway_info   | {"network_id": 
"b96473ce-  |
| | 94f6-464f-a703-5285fb8ff3d3", "enable_snat": 
true, |
| | "external_fixed_ips": 
[{"subnet_id":   |
| | 
"6c08c3d9-7df1-4bec-b847-19f80b9d1764",    |
| | "ip_address": 
"192.168.252.102"}]} |
| flavor_id   | 
None   |
| ha  | 
False  |
| id  | 
37c1794b-58d1-4d0d-b34b-944ca411b86b   |
| name    | 
public-router2 |
| project_id  | 
fe203109e67f4e39b066c9529f9fc35d   |
| revision_number | 
5  |

| routes |    |
| status  | 
ACTIVE |

| tags |    |
| updated_at  | 
2018-10-05T12:09:36Z   |

+-++

openstack network agent list
+---++---+---+---+---+--+
| ID    | Agent Type | Host  | Availability Zone | Alive | State 
| Binary   |

+---++---+---+---+---+--+
| 14b9ea75- | L3 agent   | compute1. | nova | :-)   | UP    | neutron-l3-a |
| 1dc1-4e37 |    | localdoma | |   |   | gent |
| -a2b0-508 |    | in    | |   |   |  |
| 3d336916d |    |   | |   |   |  |
| 26139ec1- | Metering   | compute1. | None | :-)   | UP    | neutron- |
| f4f9-4bb3 | agent  | localdoma | |   |   | metering-    |
| -aebb-c35 |    | in    | |   |   | agent    |
| 3a36ed79c |    |   | |   |   |  |
| 2a54971f- | DHCP agent | network1. | nova | :-)   | UP    | neutron- |
| 9849-4ed2 |    | localdoma | |   |   | dhcp-agent   |
| -b009-00e |    | in    | |   |   |  |
| 45eb4d255 |    |   | |   |   |  |
| 443c0b49- | Open   | compute1. | None | :-)   | UP    | neutron- |
| 4484-44d2 | vSwitch    | localdoma | |   |   | openvswitch- |
| -a704-32a | agent  | in    | |   |   | agent    |
| 92ffe6982 |    |   | |   |   |  |
| 5d00a219  | L3 agent   | network1. | nova | :-)   | UP    | neutron-vpn- |
| -abce-    |    | localdoma | |   |   | agent    |
| 48ca- |    | in    | |   |   |  |
| ba1e-d962 |    |   | |   |   |  |
| 01bd7de3  |    |   | |   | 

[openstack-dev] [neutron] Bug deputy report week of October 15th

[Brian Haley]

Hi,

I was Neutron bug deputy last week. Below is a short summary about 
reported bugs.


Note: I will not be at the team meeting this morning, sorry for the late 
notice.


-Brian


Critical bugs
-
None

High bugs
-

* https://bugs.launchpad.net/neutron/+bug/1798472 - Fullstack tests 
fails because process is not killed properly

  - gate failure

* https://bugs.launchpad.net/neutron/+bug/1798475 - Fullstack test 
test_ha_router_restart_agents_no_packet_lost failing

  - gate failure

* https://bugs.launchpad.net/neutron/+bug/1799124 - Path MTU discovery 
fails for VMs with Floating IP behind DVR routers

  - Needs confirmation, I took ownership

Medium bugs
---

* https://bugs.launchpad.net/neutron/+bug/1798577
  - Fix proposed, https://review.openstack.org/#/c/606007/

* A number of port-forwarding bugs were filed by Liu Yulong
  - https://bugs.launchpad.net/neutron/+bug/1799135
  - https://bugs.launchpad.net/neutron/+bug/1799137
  - https://bugs.launchpad.net/neutron/+bug/1799138
  - https://bugs.launchpad.net/neutron/+bug/1799140
  - https://bugs.launchpad.net/neutron/+bug/1799150
  - https://bugs.launchpad.net/neutron/+bug/1799155
  - Will discuss with Liu if he is working on them

Wishlist bugs
-

* https://bugs.launchpad.net/neutron/+bug/146 - When use ‘neutron 
net-update’, we cannot change the 'vlan-transparent' dynamically

  - not a bug as per the API definition, asked if proposing extension
  - perhaps possible to implement in backward-compatible way

* https://bugs.launchpad.net/neutron/+bug/1799178 - l2 pop doesn't 
always provide the whole list of fdb entries on agent restart

  - Need a smarter way to detect agent restarts

Invalid bugs


* https://bugs.launchpad.net/neutron/+bug/1798536 - OpenVswitch: qg-XXX 
goes to br-int instead of br-ext


* https://bugs.launchpad.net/neutron/+bug/1798689 -
Fullstack test test_create_one_default_qos_policy_per_project failed
  - Fixed by https://review.openstack.org/#/c/610280/

Further triage required
---

* https://bugs.launchpad.net/neutron/+bug/1798588 - 
neutron-openvswitch-agent break network connection on second reboot

  - Asked for more information from submitter

* https://bugs.launchpad.net/neutron/+bug/1798688 - iptables_hybrid 
tests 
tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_shelve_shelved_server 
failed

  - tempest.lib.exceptions.NotFound: Object not found
Details: {u'message': u'Instance None could not be found.', 
u'code': 404}

Not sure if issue with shelve/unshelve since the instance is gone

* https://bugs.launchpad.net/bugs/1798713 - [fwaas]wrong judgment in 
_is_supported_by_fw_l2_driver method

  - Fix proposed, https://review.openstack.org/#/c/605988
Need someone from FWaaS team to confirm and set priority

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][stable] Stable Core Team Update

[Brian Haley]

+1 from me :)

-Brian

On 10/02/2018 11:41 AM, Miguel Lavalle wrote:

Hi Stable Team,

I want to nominate Bernard Cafarrelli as a stable core reviewer for 
Neutron and related projects. Bernard has been increasing the number of 
stable reviews he is doing for the project [1]. Besides that, he is a 
stable maintainer downstream for his employer (Red Hat), so he can bring 
that valuable experience to the Neutron stable team.


Thanks and regards

Miguel

[1] 
https://review.openstack.org/#/q/(project:openstack/neutron+OR+openstack/networking-sfc+OR+project:openstack/networking-ovn)++branch:%255Estable/.*+reviewedby:%22Bernard+Cafarelli+%253Cbcafarel%2540redhat.com%253E%22 




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] subnet pool can not delete prefixes

[Brian Haley]

On 09/26/2018 10:11 PM, wenran xiao wrote:

Relation bug: https://bugs.launchpad.net/neutron/+bug/1792901
Any suggestion is welcome!


Removing a prefix from a subnetpool is not supported, there was an 
inadvertent change to the client that made it seem possible.  We are in 
the process of reverting it:


https://review.openstack.org/#/c/599633/

-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Core status

[Brian Haley]

On 09/19/2018 02:19 PM, Gary Kotton wrote:

Hi,

I have recently transitioned to a new role where I will be working on 
other parts of OpenStack. Sadly I do not have the necessary cycles to 
maintain my core responsibilities in the neutron community. Nonetheless 
I will continue to be involved.


Thanks for all your work over the years, especially in keeping the 
reviews moving along on the neutron stable branches.  Good luck in your 
new role!


-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Appointing Slawek Kaplonski to the Neutron Drivers team

[Brian Haley]

Congrats Slawek!

On 08/27/2018 12:42 PM, Miguel Lavalle wrote:

Dear Neutron team,

In order to help the Neutron Drivers team to perform its very important 
job of guiding the community to evolve the OpenStack Networking 
architecture to meet the needs of our current and future users [1], I 
have asked Slawek Kaplonski to join it. Over the past few years, he has 
gained very valuable experience with OpenStack Networking, both as a 
deployer  and more recently working with one of our key packagers. He 
played a paramount role in implementing our QoS (Quality of Service) 
features, currently leading that sub-team. He also leads the CI 
sub-team, making sure the prompt discovery and fixing of bugs in our 
software. On top of that, he is one of our most active reviewers, 
contributor of code to our reference implementation and fixer of bugs. I 
am very confident in Slawek making great contributions to the Neutron 
Drivers team.


Best regards

Miguel

[1] 
https://docs.openstack.org/neutron/latest/contributor/policies/neutron-teams.html#drivers-team



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] Help with ipv6 self-service and ip6tables rule on mangle chain

[Brian Haley]

On 08/23/2018 12:53 PM, Jorge Luiz Correa wrote:

Hi all

I'm deploying a Queens on Ubuntu 18.04 with one controller, one network 
controller e for now one compute node. I'm using ML2 with linuxbridge 
mechanism driver and a self-service type of network. This is is a dual 
stack environment (v4 and v6).


IPv4 is working fine, NATs oks and packets flowing.

With IPv6 I'm having a problem. Packets from external networks to a 
project network are stopping on qrouter namespace firewall. I've a 
project with one network, one v4 subnet and one v6 subnet. Adressing are 
all ok, virtual machines are getting their IPs and can ping the network 
gateway.


However, from external to project network, using ipv6, the packets stop 
in a DROP rule inside de qrouter namespace.


This looks like the address scopes of the subnets are different, so the 
rule to mark packets is not being inserted.  How are you assigning the 
subnet addresses on the external and internal networks?  Typically you 
would define a subnet pool and allocate from that, which should work. 
Perhaps this guide would help with that:


https://docs.openstack.org/neutron/queens/admin/config-address-scopes.html

The last sentence there seems to describe the problem you're having:

"If the address scopes match between networks then pings and other 
traffic route directly through. If the scopes do not match between 
networks, the router either drops the traffic or applies NAT to cross 
scope boundaries."


IPv6 in neutron does not use NAT...

-Brian



The ip6tables path is:

mangle prerouting -> neutron-l3-agent-PREROUTING -> 
neutron-l3-agent-scope -> here we have a MARK rule:


pkts bytes target prot opt in out source   
destination
     3   296 MARK   all  qr-7f2944e7-cc *   
::/0 ::/0 MARK xset 0x400/0x


qr interface is the internal network interface of the project (subnet 
gateway). So, packets from this interface are marked.


But, the returning is the problem. The packets doesn't returns. I've 
rules from the nexthop firewall and packets arrive on the external 
bridge (network node). But, when they arrive on external interface of 
the qrouter namespace, they are filtered.


Inside qrouter namespace this is the rule:

ip netns exec qrouter-5689783d-52c0-4d2f-bef5-99b111f8ef5f ip6tables -t 
mangle -L -n -v


...
Chain neutron-l3-agent-scope (1 references)
  pkts bytes target prot opt in out source   
destination
     0 0 DROP   all  *  qr-7f2944e7-cc  
::/0 ::/0 mark match ! 0x400/0x

...

If I create the following rule everything works great:

ip netns exec qrouter-5689783d-52c0-4d2f-bef5-99b111f8ef5f ip6tables -t 
mangle -I neutron-l3-agent-scope -i qg-b6757bfe-c1 -j MARK --set-xmark 
0x400/0x


where qg is the external interface of virtual router. So, if I mark 
packets from external interface on mangle, they are not filtered.


Is this normal? I've to manually add a rule to do that?

How to use the "external_ingress_mark" option on l3-agent.ini ? Can I 
use it to mark packets using a configuration parameter instead of 
manually inserted ip6tables rule?


Thanks a lot!

- JLC


___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack



___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] [Openstack-operators] Recovering from full outage

[Brian Haley]

On 07/16/2018 08:41 AM, Torin Woltjer wrote:
$ip netns exec qdhcp-87a5200d-057f-475d-953d-17e873a47454 curl 
http://169.254.169.254



  404 Not Found


  404 Not Found
  The resource could not be found. > 



Strange, don't know where the reply came from for that.

$ip netns exec qrouter-80c3bc40-b49c-446a-926f-99811adc0c5e curl 
http://169.254.169.254

curl: (7) Couldn't connect to server


Based on your iptables output below, I would think the metadata proxy is 
running in the qrouter namespace.  However, a curl from there will not 
work since it is restricted to only work for incoming packets from the 
qr- device(s).  You would have to try curl from a running instance.


Is there an haproxy process running?  And is it listening on port 9697 
in the qrouter namespace?


-Brian




*From*: "Torin Woltjer" 
*Sent*: 7/12/18 11:16 AM
*To*: , , 
"jpetr...@coredial.com" 

*Cc*: openstack-operat...@lists.openstack.org, openstack@lists.openstack.org
*Subject*: Re: [Openstack] [Openstack-operators] Recovering from full outage
Checking iptables for the metadata-proxy inside of qrouter provides the 
following:
$ip netns exec qrouter-80c3bc40-b49c-446a-926f-99811adc0c5e 
iptables-save -c | grep 169
[0:0] -A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p 
tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697
[0:0] -A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p 
tcp -m tcp --dport 80 -j MARK --set-xmark 0x1/0x

Packets:Bytes are both 0, so no traffic is touching this rule?

Interestingly the command:
$ip netns exec qrouter-80c3bc40-b49c-446a-926f-99811adc0c5e netstat 
-anep | grep 9697
returns nothing, so there isn't actually anything running on 9697 in the 
network namespace...


This is the output without grep:
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address 
State       User       Inode      PID/Program name
raw        0      0 0.0.0.0:112             0.0.0.0:*               7   
         0          76154      8404/keepalived
raw        0      0 0.0.0.0:112             0.0.0.0:*               7   
         0          76153      8404/keepalived

Active UNIX domain sockets (servers and established)
Proto RefCnt Flags       Type       State         I-Node   PID/Program 
name     Path

unix  2      [ ]         DGRAM                    64501    7567/python2
unix  2      [ ]         DGRAM                    79953    8403/keepalived

Could the reason no traffic touching the rule be that nothing is 
listening on that port, or is there a second issue down the chain?


Curl fails even after restarting the neutron-dhcp-agent & 
neutron-metadata agent.


Thank you for this, and any future help.


___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] [Openstack-operators] Recovering from full outage

[Brian Haley]

On 07/12/2018 08:20 AM, Torin Woltjer wrote:
The neutron-metadata-agent service is running, the the agent is alive, 
and it is listening on port 8775. However, new instances still do not 
get any information like hostname or keypair. If I run `curl 
192.168.116.22:8775` from the compute nodes, I do get a response. The 
metadata agent is running, listening, and accessible from the compute 
nodes; and it worked previously.


I'm stumped.


There is also a metadata proxy that runs in the qrouter namespace, you 
can verify it's running and getting requests by looking at both iptables 
and netstat output.


$ sudo ip netns exec qrouter-$ID iptables-save -c | grep 169
[16:960] -A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p 
tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697
[96:7968] -A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ 
-p tcp -m tcp --dport 80 -j MARK --set-xmark 0x1/0x


The numbers inside [] represent packets:bytes, so non-zero is good.

$ sudo ip netns exec qrouter-$ID netstat -anep | grep 9697
tcp0  0 0.0.0.0:96970.0.0.0:* 
LISTEN  0  294339  4867/haproxy


If you have a running instance you can log into, running curl to the 
metadata IP would be helpful to try and diagnose since it would go 
through this entire path.


-Brian



/*Torin Woltjer*/
*Grand Dial Communications - A ZK Tech Inc. Company*
*616.776.1066 ext. 2006*
/*www.granddial.com */


*From*: அருண் குமார் (Arun Kumar) 
*Sent*: 7/12/18 12:01 AM
*To*: torin.wolt...@granddial.com
*Cc*: "openstack@lists.openstack.org" , 
openstack-operat...@lists.openstack.org

*Subject*: Re: [Openstack-operators] [Openstack] Recovering from full outage
Hi Torin,

If I run `ip netns exec qrouter netstat -lnp` or `ip netns exec
qdhcp netstat -lnp` on the controller, should I see anything
listening on the metadata port (8775)? When I run these commands I
don't see that listening, but I have no example of a working system
to check against. Can anybody verify this?


Either on qrouter/qdhcp namespaces, you won't see port 8775, instead 
check whether meta-data service is running on the neutron controller 
node(s) and listening on port 8775? Aslo, you can verify metadata and 
neturon services using following commands


service neutron-metadata-agent status
neutron agent-list
netstat -ntplua | grep :8775


Thanks & Regards
Arun

ஃஃ
அன்புடன்
அருண்
நுட்பம் நம்மொழியில் தழைக்கச் செய்வோம்
http://thangamaniarun.wordpress.com
ஃஃ



___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack



___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] [neutron] Stable review

[Brian Haley]

On 07/12/2018 02:53 AM, Takashi Yamamoto wrote:

hi,

queens branch of networking-midonet has had no changes merged since
its creation.
the following commit would tell you how many gate blockers have been
accumulated.
https://review.openstack.org/#/c/572242/

it seems the stable team doesn't have a bandwidth to review subprojects
in a timely manner. i'm afraid that we need some policy changes.


In the future I would recommend just adding someone from the neutron 
stable team to the review, as we (I) don't have the bandwidth to go 
through the reviews of every sub-project.  Between Miguel, Armando, Gary 
and myself we can usually get to things pretty quickly. 
https://review.openstack.org/#/admin/groups/539,members


-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron] Bug deputy report

[Brian Haley]

Hi,

I was Neutron bug deputy last week. Below is a short summary about 
reported bugs.



Critical bugs
-
None

High bugs
-

* https://bugs.launchpad.net/neutron/+bug/1777908 -
Ensure _get_changed_synthetic_fields() return updatable fields - breaks 
consumers

  - Boden proposed reverts

* https://bugs.launchpad.net/neutron/+bug/1777922 - neutron is not 
dropping radvd privileges

  - Fix proposed, https://review.openstack.org/#/c/576923/

* https://bugs.launchpad.net/neutron/+bug/1777968 - Too many 
DBDeadlockError and IP address collision during port creating

  - Fix proposed, https://review.openstack.org/#/c/577739/

Medium bugs
---

* https://bugs.launchpad.net/neutron/+bug/1778118 - missing transaction 
in driver_controller.py for l3 flavors

  - Fix proposed, https://review.openstack.org/#/c/577246/

Wishlist bugs
-

* https://bugs.launchpad.net/neutron/+bug/146 - When use ‘neutron 
net-update’, we cannot change the 'vlan-transparent' dynamically

  - not a bug as per the API definition, asked if proposing extension
  - perhaps possible to implement in backward-compatible way

Further triage required
---

* https://bugs.launchpad.net/neutron/+bug/1777866 - QoS CLI – Warning in 
case when provided burst is lower than 80% BW

  - submitter wants CLI warning, not sure it's necessary as it is
already mentioned in the admin guide
  - possible change in OSC could address

* https://bugs.launchpad.net/neutron/+bug/1778407 - Quality of Service 
(QoS) in Neutron - Notes regarding burst (80% BW)

  - Closed as duplicate of 1777866

* https://bugs.launchpad.net/neutron/+bug/1777965 - Create port get 
quota related DBDeadLock


* https://bugs.launchpad.net/neutron/+bug/1778207 - fwaas v2 add port 
into firewall group failed

  - most likely need DVR/HA check in affected code path
  - need to ping SridarK to take a look

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] Is it possible to use OVS inside an Openstack VM ?

[Brian Haley]

On 06/18/2018 06:26 AM, David Fernandes wrote:

Hi guys,

I have just 1 question. Is it possible to use OVS inside an Openstack VM 
? Let me explain my issue.


I have created 2 Ubuntu 16.04 VMs and installed last version of OVS in 
both. Both VMs are connected to the same network and they can ping each 
other.
Then, I have created an OVS bridge (namely "br-test") in the first VM 
and I have added the port of the VM (ens1) to the ovs bridge as usual. 
At this point I cant't ping from one VM to the other.
If I perform tcpdump on the bridge (tcpdump br-test), I see that ARP 
requests are sent but they are not sent by the interface of the VM (ens1).
I tried to add the MAC addreses manually to the ARP table of each VM. 
When I ping, I see the same case :  icmp packets outgoing the bridge 
br-test but never sent by the interface ens1 of the VM.


I can only guess things are being blocked by the MAC anti-spoofing rules 
neutron adds - perhaps the source MAC of the bridge is being used which 
is different from the NIC?


-Brian


I have tried the same scenario using a linux bridge instead of OVS 
swithch and it works.


Do you have any idea of why there are issues binding VM interface to OVS 
bridges ? Any information will be really apreciated !! Thanks in advance 
!!!


David



___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack



___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] [StarlingX] StarlingX code followup discussions

[Brian Haley]

On 05/23/2018 02:00 PM, Jeremy Stanley wrote:

On 2018-05-22 17:41:18 -0400 (-0400), Brian Haley wrote:
[...]

I read this the other way - the goal is to get all the forked code from
StarlingX into upstream repos.  That seems backwards from how this should
have been done (i.e. upstream first), and I don't see how a project would
prioritize that over other work.

[...]

I have yet to see anyone suggest it should be prioritized over other
work. I expect the extracted and proposed changes/specs
corresponding to the divergence would be viewed on their own merits
just like any other change and ignored, reviewed, rejected, et
cetera as appropriate.


Even doing that is work - going through changes, finding nuggets, 
proposing new specs I don't think we can expect a project to even go 
there, it has to be driven by someone already involved in StarlingX, IMHO.


-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [StarlingX] StarlingX code followup discussions

[Brian Haley]

On 05/22/2018 04:57 PM, Jay Pipes wrote:

Warning: strong opinions ahead.

On 05/22/2018 02:54 PM, Dean Troyer wrote:

Developers will need to re-create a repo locally in
order to work or test the code and create reviews (there are more git
challenges here). It would be challenging to do functional testing on
the rest of STX in CI without access to all of the code.


Please don't take this the wrong way, Dean, but you aren't seriously 
suggesting that anyone outside of Windriver/Intel would ever contribute 
to these repos are you?


What motivation would anyone outside of Windriver/Intel -- who must make 
money on this effort otherwise I have no idea why they are doing it -- 
have to commit any code at all to StarlingX?


I read this the other way - the goal is to get all the forked code from 
StarlingX into upstream repos.  That seems backwards from how this 
should have been done (i.e. upstream first), and I don't see how a 
project would prioritize that over other work.


I'm truly wondering why was this even open-sourced to begin with? I'm as 
big a supporter of open source as anyone, but I'm really struggling to 
comprehend the business, technical, or marketing decisions behind this 
action. Please help me understand. What am I missing?


I'm just as confused.

-Brian


My personal opinion is that I don't think that any products, derivatives 
or distributions should be hosted on openstack.org infrastructure.


Are any of the distributions of OpenStack listed at 
https://www.openstack.org/marketplace/distros/ hosted on openstack.org 
infrastructure? No. And I think that is completely appropriate.


Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] private network issue ( kola-ansible pike/stable deployment )

[Brian Haley]

On 04/06/2018 01:28 PM, s serge wrote:

Hello,

I'm evaluating an installation and everything from networking side was looking 
good
until I tried to reach a VM host via private network from another VM via ssh.

In short:
1. Spawn a VM
2. Associate a floating IP
3. Logon to VM via ssh on public network
4. Spawn another VM
5. Try to reach 1st VM via ssh private network IP - FAIL.
6. ICMP to 1st VM IP via private network works well.

Looks pretty weird for me as according to logs everything looks fine,
both VM got assigned a private IP and fetches metadata info.

Some notes about setup:
Separate interfaces for management, private(VXLAN) and external network.
Dozen of similar servers.

I'll continue to debug the issue, but appreciate any relevant feedback.


I would check two things:

1. Security groups are allowing port 22
2. MTU is set correctly, should probably be 1450 if you're using VXLAN, 
which should have been set via the DHCP reply


-Brian

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] [Neutron][DVR] Network issue during migration

[Brian Haley]

On 03/21/2018 05:38 PM, Mārtiņš Jakubovičs wrote:

Hello Brian,

Yes, after migration it works well, but during migration it is not.


Ok, so somehow neutron is configuring things too early, updating the 
floating IP (and your ARP cache), before the instance has finished 
migration.  It would be good if you could look at the nova and neutron 
logs to help piece this together - i.e. when is nova telling neutron to 
configure the migrated port, etc.


Also, without looking at what's changed since Newton, trying a live 
migration on plain Ocata (or Pike) would be a good data point, just in 
case there was a bug somewhere that is now fixed.


Thanks,

-Brian


On 2018.03.21. 23:37, Brian Haley wrote:

On 03/21/2018 04:52 PM, Mārtiņš Jakubovičs wrote:

Dear all,

I faced issue when migrating instance by live-migration + 
block-migration guests floating IP became inaccessible. My setup are 
OVS + DVR. Currently upgrading from Newton to Ocata.


Looks like when migration starts, in destination host neutron 
configures floating IP so traffic start to go to destination host but 
instance are still in source host.


Does live migration work correctly with DVR?


It should.  Does the floating IP work after migration is finished?  Is 
it just temporary?


-Brian






___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] ARP packets not sent during migration

[Brian Haley]

On 03/23/2018 07:05 AM, Ramon Orru wrote:

Hi Brian,

yes, you're right, that's the point, libvirt is sending gARPs too 
early am I misconfiguring something? Seems to me a really strange 
behaviour...


Hi Ramon,

Sorry, don't know much about the libvirt side of things, maybe someone 
more familiar with the Nova side of the migration would know?


-Brian


Il 21/03/18 22:35, Brian Haley ha scritto:

On 03/20/2018 12:40 PM, Ramon Orru wrote:
Hello everybody, I'm running a fresh queens cluster. I'm using 
bridges to support networking. I'm facing an issue when an instance 
is live migrated.
Suppose we have an instance running with an interface on vlan XXX, 
and we want to migrate it to compute host YYY. We'll call that 
instance ZZZ.
If no other instance is already running on YYY using vlan XXX, no 
bridge called 'br-vlan.XXX@br-vlan' exists yet on YYY.
Now, if I migrate ZZZ on YYY host, a new bridge 'br-vlan.XXX@br-vlan' 
will be created.
During the migration process, ZZZ become unreachable while interfaces 
are going up on YYY (from 10 seconds to about 2 minutes).
After some troubleshooting, we spotted the problem: bridge 
'br-vlan.XXX@br-vlan' is being created after gratuitous ARP packets 
are sent from migrating machines to advise other devices about new 
position.
The result is: no other device can reach the fresh migrated machine 
until ARP table becomes stale.
This does not happen when an instance with an interface on same vlan 
is already runnning on destination host ('br-vlan.XXX@br-vlan' is 
already up and running, and ARPs can be sent flawlessly).


Any idea of how to get rid of this? I think it's very unlikely that 
I'm the first to face this problem, but i didn't manage to find 
additional info on this strange behaviour.

Thanks in advance.


Just to clarify - are you talking about connectivity to the floating 
IP or just on the VLAN itself, i.e. the instance is directly connected 
to the VLAN.  I'm thinking it's the latter, which would mean it's 
libvirt(?) sending the gARPs before the bridge is up?


-Brian





___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] ARP packets not sent during migration

[Brian Haley]

On 03/20/2018 12:40 PM, Ramon Orru wrote:
Hello everybody, I'm running a fresh queens cluster. I'm using bridges 
to support networking. I'm facing an issue when an instance is live 
migrated.
Suppose we have an instance running with an interface on vlan XXX, and 
we want to migrate it to compute host YYY. We'll call that instance ZZZ.
If no other instance is already running on YYY using vlan XXX, no bridge 
called 'br-vlan.XXX@br-vlan' exists yet on YYY.
Now, if I migrate ZZZ on YYY host, a new bridge 'br-vlan.XXX@br-vlan' 
will be created.
During the migration process, ZZZ become unreachable while interfaces 
are going up on YYY (from 10 seconds to about 2 minutes).
After some troubleshooting, we spotted the problem: bridge 
'br-vlan.XXX@br-vlan' is being created after gratuitous ARP packets are 
sent from migrating machines to advise other devices about new position.
The result is: no other device can reach the fresh migrated machine 
until ARP table becomes stale.
This does not happen when an instance with an interface on same vlan is 
already runnning on destination host ('br-vlan.XXX@br-vlan' is already 
up and running, and ARPs can be sent flawlessly).


Any idea of how to get rid of this? I think it's very unlikely that I'm 
the first to face this problem, but i didn't manage to find additional 
info on this strange behaviour.

Thanks in advance.


Just to clarify - are you talking about connectivity to the floating IP 
or just on the VLAN itself, i.e. the instance is directly connected to 
the VLAN.  I'm thinking it's the latter, which would mean it's 
libvirt(?) sending the gARPs before the bridge is up?


-Brian

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] [Neutron][DVR] Network issue during migration

[Brian Haley]

On 03/21/2018 04:52 PM, Mārtiņš Jakubovičs wrote:

Dear all,

I faced issue when migrating instance by live-migration + 
block-migration guests floating IP became inaccessible. My setup are OVS 
+ DVR. Currently upgrading from Newton to Ocata.


Looks like when migration starts, in destination host neutron configures 
floating IP so traffic start to go to destination host but instance are 
still in source host.


Does live migration work correctly with DVR?


It should.  Does the floating IP work after migration is finished?  Is 
it just temporary?


-Brian

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

[openstack-dev] [neutron] Bug deputy report

[Brian Haley]

Hi,

I was Neutron bug deputy last week. Below is a short summary about 
reported bugs.



Critical bugs
-
None

High bugs
-

* https://bugs.launchpad.net/bugs/1753504 - Remove mox/mox3 usage from 
testing

Multiple people have taken ownership of fixing this

* https://bugs.launchpad.net/bugs/1754062 - openstack client does not 
pass prefixlen when creating subnet

Looks like both a bug in the api-ref and in the openstacksdk

* https://bugs.launchpad.net/bugs/1754327 - Tempest scenario jobs 
failing due to no FIP connectivity

https://review.openstack.org/#/c/550832/ proposed to try and track
down reason for failure (thanks Slawek!)

* https://bugs.launchpad.net/bugs/1755243 - AttributeError when updating 
DvrEdgeRouter objects running on network nodes

Bug submitter has already posted a patch

* https://bugs.launchpad.net/bugs/1754563 - Arp_responder function has 
failed since Ocata

Looks like there is maybe a missing setup_privsep() call in the L2
agent code?  Yes, and Slawek just fixed it in master, so we'll need
to backport just the change in one file.  haleyb pushed a change.

Medium bugs
---

* https://bugs.launchpad.net/bugs/1753540 - When isolated/force metadata 
is enabled, metadata proxy doesn't get automatically started/stopped 
when needed

Daniel Alvarez sent patch for master and backports (thanks!)

* https://bugs.launchpad.net/bugs/1754770 - Duplicate iptables rule 
detected in Linuxbridge agent logs

Slawek took ownership as messages started after another change

Low bugs


* https://bugs.launchpad.net/bugs/1753384 - The old QoS policy ID is 
returned when updating the QoS policy ID, when the revision plugin is 
enabled

Guoshuai Li took ownership

Bugs that need further triage
-

* https://bugs.launchpad.net/bugs/1753434 - Unbound ports floating ip 
not working with address scopes in DVR HA

Found on Pike, need to determine if already fixed in master

* https://bugs.launchpad.net/bugs/1754695 - Incorrect state of the 
Openflow table

Restarting neutron_openvswitch_agent container fixed issue

* https://bugs.launchpad.net/bugs/1754600 - the detail for openstack 
quota show is not supported
Probably duplicate of 
https://bugs.launchpad.net/neutron/+bug/1716043 since that was opened to 
track CLI quota change


RFE bugs for drivers team
-

* https://bugs.launchpad.net/bugs/1753466 - [RFE] Support stateless 
security groups


* https://bugs.launchpad.net/bugs/1754123 - [RFE] Support filter with 
floating IP address substring



Thanks,
-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] [neutron] Current containerized neutron agents introduce a significant regression in the dataplane

[Brian Haley]

On 02/13/2018 05:08 PM, Armando M. wrote:



On 13 February 2018 at 14:02, Brent Eagles > wrote:


Hi,

The neutron agents are implemented in such a way that key
functionality is implemented in terms of haproxy, dnsmasq,
keepalived and radvd configuration. The agents manage instances of
these services but, by design, the parent is the top-most (pid 1).

On baremetal this has the advantage that, while control plane
changes cannot be made while the agents are not available, the
configuration at the time the agents were stopped will work (for
example, VMs that are restarted can request their IPs, etc). In
short, the dataplane is not affected by shutting down the agents.

In the TripleO containerized version of these agents, the supporting
processes (haproxy, dnsmasq, etc.) are run within the agent's
container so when the container is stopped, the supporting processes
are also stopped. That is, the behavior with the current containers
is significantly different than on baremetal and stopping/restarting
containers effectively breaks the dataplane. At the moment this is
being considered a blocker and unless we can find a resolution, we
may need to recommend running the L3, DHCP and metadata agents on
baremetal.


I didn't think the neutron metadata agent was affected but just the 
ovn-metadata agent?  Or is there a problem with the UNIX domain sockets 
the haproxy instances use to connect to it when the container is restarted?


There's quite a bit to unpack here: are you suggesting that running 
these services in HA configuration doesn't help either with the data 
plane being gone after a stop/restart? Ultimately this boils down to 
where the state is persisted, and while certain agents rely on 
namespaces and processes whose ephemeral nature is hard to persist, 
enough could be done to allow for a non-disruptive bumping of the afore 
mentioned services.


Armando - https://review.openstack.org/#/c/542858/ (if accepted) should 
help with dataplane downtime, as sharing the namespaces lets them 
persist, which eases what the agent has to configure on the restart of a 
container (think of what the l3-agent needs to create for 1000 routers).


But it doesn't address dnsmasq being unavailable when the dhcp-agent 
container is restarted like it is today.  Maybe one way around that is 
to run 2+ agents per network, but that still leaves a regression from 
how it works today.  Even with l3-ha I'm not sure things are perfect, 
might wind-up with two masters sometimes.


I've seen one suggestion of putting all these processes in their own 
container instead of the agent container so they continue to run, it 
just might be invasive to the neutron code.  Maybe there is another option?


-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] DVR Public IP consumption

[Brian Haley]

On 01/31/2018 10:12 AM, Satish Patel wrote:

Brian,

You mean say i can use any private subnet for that service-type?  I am
having hard time to understand could you please give me example or
details would be helpful.


Yes, you should be able to use a private subnet that can be configured 
onto that network, that way the DVR routers would use IPs from it, and 
still be able to communicate with other devices in the datacenter.  I 
believe the docs site I linked previously had an example of this.


-Brian


On Thu, Jan 18, 2018 at 2:12 PM, Brian Haley <haleyb@gmail.com> wrote:

On 01/16/2018 08:37 AM, Satish Patel wrote:


Thanks Brian,

I may having difficulty to understand that example, if I have only /24
public subnet for cloud and have 200 compute node then how does it work?



The intention is to have a second subnet on the external network, but only
have it usable within the datacenter.  If you create it and set the
service-type to only certain types of ports, like DVR, then it won't be used
for floating IP as the other one is.

-Brian



On Jan 15, 2018, at 9:47 PM, Brian Haley <haleyb@gmail.com> wrote:


On 01/15/2018 01:57 PM, Satish Patel wrote:
I am planning to build openstack on production and big question is
network (legacy vs DVR) but in DVR big concern is number of Public IP
used on every compute node, I am planning to add max 100 node in
cluster in that case it will use 100 public IP for compute node, Ouch!



You can reduce this public IP consumption by using multiple subnets on
the external network, with one just for those DVR interfaces. See

https://docs.openstack.org/ocata/networking-guide/config-service-subnets.html
Example #2 for a possible configuration.

As for what type of L3 configuration to run, it seems like you have a
good idea of some of the trade-offs with each.

-Brian


If i use legacy compute node then it could be bottleneck or failure
node if not in HA.
what most of company use for network node?  DVR or legacy?
___
Mailing list:
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe :
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack







___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] DVR Public IP consumption

[Brian Haley]

On 01/16/2018 08:37 AM, Satish Patel wrote:

Thanks Brian,

I may having difficulty to understand that example, if I have only /24 public 
subnet for cloud and have 200 compute node then how does it work?


The intention is to have a second subnet on the external network, but 
only have it usable within the datacenter.  If you create it and set the 
service-type to only certain types of ports, like DVR, then it won't be 
used for floating IP as the other one is.


-Brian


On Jan 15, 2018, at 9:47 PM, Brian Haley <haleyb@gmail.com> wrote:


On 01/15/2018 01:57 PM, Satish Patel wrote:
I am planning to build openstack on production and big question is
network (legacy vs DVR) but in DVR big concern is number of Public IP
used on every compute node, I am planning to add max 100 node in
cluster in that case it will use 100 public IP for compute node, Ouch!


You can reduce this public IP consumption by using multiple subnets on the 
external network, with one just for those DVR interfaces. See
https://docs.openstack.org/ocata/networking-guide/config-service-subnets.html
Example #2 for a possible configuration.

As for what type of L3 configuration to run, it seems like you have a good idea 
of some of the trade-offs with each.

-Brian


If i use legacy compute node then it could be bottleneck or failure
node if not in HA.
what most of company use for network node?  DVR or legacy?
___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack




___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] DVR Public IP consumption

[Brian Haley]

On 01/15/2018 01:57 PM, Satish Patel wrote:

I am planning to build openstack on production and big question is
network (legacy vs DVR) but in DVR big concern is number of Public IP
used on every compute node, I am planning to add max 100 node in
cluster in that case it will use 100 public IP for compute node, Ouch!


You can reduce this public IP consumption by using multiple subnets on 
the external network, with one just for those DVR interfaces. See

https://docs.openstack.org/ocata/networking-guide/config-service-subnets.html
Example #2 for a possible configuration.

As for what type of L3 configuration to run, it seems like you have a 
good idea of some of the trade-offs with each.


-Brian


If i use legacy compute node then it could be bottleneck or failure
node if not in HA.

what most of company use for network node?  DVR or legacy?

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack




___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

[openstack-dev] [neutron][all] nova_metadata_ip removal

[Brian Haley]

Hi,

As part of the normal deprecate/removal process, the 'nova_metadata_ip' 
option is being removed from neutron as it was replaced with 
'nova_metadata_host' in the Pike cycle, 
https://review.openstack.org/#/c/518836/


Codesearch did find various repos still using the old value, so I posted 
a number of cleanups for them since it was pretty painless, 
https://review.openstack.org/#/q/topic:nova_metadata_ip_deprecated+(status:open+OR+status:merged)


This is just an FYI for anyone else that might trip over the old value 
going away once it finally merges, most will probably never notice.


-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Metering can't count traffic for floating ip, or internal ip.

[Brian Haley]

On 01/04/2018 09:50 PM, wenran xiao wrote:

hi all，
neutron metering can only count traffic that we send to 
*remote_ip*(egress), and *remote_ip* send to us(ingress), I think we 
should add method to count the traffic for floating ip or internal ip.

Any suggestions is welcome.


Neutron metering was originally created as a way to get input for 
billing tenants for usage, giving admins numbers for what stayed inside 
or exited a datacenter.  It has languished over the past few cycles 
either because it is working perfectly, or not used very much - my guess 
is the latter.


That said, if you want to propose an enhancement, there is an RFE 
process defined in 
https://docs.openstack.org/neutron/latest/contributor/policies/blueprints.html 
you can use, the neutron drivers team has weekly meetings to discuss 
things for inclusion into releases.


-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] performance issue between virtual networks

[Brian Haley]

On 12/27/2017 08:39 AM, Kim-Norman Sahm wrote:

Hi,

i've detected a performance issue by accessing an floating ip in a
different openstack network (same tenant).

example:
i have one tenant with two internal networks.
each network has its own vrouter which is connectet to the extnet.
the physical network infrastructure is 10Gbit/s.

          networkA
    VM1 --|   extnet
  ||vrouter1||
    VM2 --|  |
 |---ext
  networkB   |
    VM3 --|  |
  ||vrouter2||
    VM4 --|

VM1 -> VM2   ~8,6Gbit/s
VM3 -> VM4   ~8,6GBit/s
VM1 -> vrouter1  ~8.6GBit/s
VM4 -> vrouter2 ~8,6GBit/s
vrouter1 -> vrouter2 ~8,6Gbits
VM1 -> VM4   ~2,5GBit/s
VM1 -> vrouter2 ~2,5Gbit/s


I could only guess that vrouter1 and vrouter2 are on different nodes, so 
you're losing some performance going from virtual to physical and back 
(eg GSO).  Have you tried this for reference:


VM1 -> system on extnet
VM4 -> system on extnet

Also, are you sure when packets from VM1 -> VM4 leave the vrouter1 
interface they are still at the higher MTU?


-Brian




detected with iperf3
it's an openstack newton environment with openvswitch 2.6.1
VXLAN mtu is 8950 and 9000 for physical interfaces

does anybody has an idea what could be the cause of the performance
issue?

Best regards
Kim

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Tis the season...for a cloud reboot

[Brian Haley]

On 12/19/2017 04:00 PM, Ben Nemec wrote:



On 12/19/2017 02:43 PM, Brian Haley wrote:

On 12/19/2017 11:53 AM, Ben Nemec wrote:

The reboot is done (mostly...see below).

On 12/18/2017 05:11 PM, Joe Talerico wrote:

Ben - Can you provide some links to the ovs port exhaustion issue for
some background?


I don't know if we ever had a bug opened, but there's some discussion 
of it in 
http://lists.openstack.org/pipermail/openstack-dev/2016-December/109182.html 
  I've also copied Derek since I believe he was the one who found it 
originally.


The gist is that after about 3 months of tripleo-ci running in this 
cloud we start to hit errors creating instances because of problems 
creating OVS ports on the compute nodes.  Sometimes we see a huge 
number of ports in general, other times we see a lot of ports that 
look like this:


Port "qvod2cade14-7c"
 tag: 4095
 Interface "qvod2cade14-7c"

Notably they all have a tag of 4095, which seems suspicious to me.  I 
don't know whether it's actually an issue though.


Tag 4095 is for "dead" OVS ports, it's an unused VLAN tag in the agent.

The 'qvo' here shows it's part of the VETH pair that os-vif created 
when it plugged in the VM (the other half is 'qvb'), and they're 
created so that iptables rules can be applied by neutron.  It's part 
of the "old" way to do security groups with the 
OVSHybridIptablesFirewallDriver, and can eventually go away once the 
OVSFirewallDriver can be used everywhere (requires newer OVS and agent).


I wonder if you can run the ovs_cleanup utility to clean some of these 
up?


As in neutron-ovs-cleanup?  Doesn't that wipe out everything, including 
any ports that are still in use?  Or is there a different tool I'm not 
aware of that can do more targeted cleanup?


Crap, I thought there was an option to just cleanup these dead devices, 
I should have read the code, it's either neutron ports (default) or all 
ports.  Maybe that should be an option.


-Brian

Oh, also worth noting that I don't think we have os-vif in this cloud 
because it's so old.  There's no os-vif package installed anyway.




-Brian

I've had some offline discussions about getting someone on this cloud 
to debug the problem.  Originally we decided not to pursue it since 
it's not hard to work around and we didn't want to disrupt the 
environment by trying to move to later OpenStack code (we're still 
back on Mitaka), but it was pointed out to me this time around that 
from a downstream perspective we have users on older code as well and 
it may be worth debugging to make sure they don't hit similar problems.


To that end, I've left one compute node un-rebooted for debugging 
purposes.  The downstream discussion is ongoing, but I'll update here 
if we find anything.




Thanks,
Joe

On Mon, Dec 18, 2017 at 10:43 AM, Ben Nemec <openst...@nemebean.com> 
wrote:

Hi,

It's that magical time again.  You know the one, when we reboot rh1 
to avoid

OVS port exhaustion. :-)

If all goes well you won't even notice that this is happening, but 
there is

the possibility that a few jobs will fail while the te-broker host is
rebooted so I wanted to let everyone know.  If you notice anything 
else
hosted in rh1 is down (tripleo.org, zuul-status, etc.) let me know. 
I have

been known to forget to restart services after the reboot.

I'll send a followup when I'm done.

-Ben

__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Tis the season...for a cloud reboot

[Brian Haley]

On 12/19/2017 11:53 AM, Ben Nemec wrote:

The reboot is done (mostly...see below).

On 12/18/2017 05:11 PM, Joe Talerico wrote:

Ben - Can you provide some links to the ovs port exhaustion issue for
some background?


I don't know if we ever had a bug opened, but there's some discussion of 
it in 
http://lists.openstack.org/pipermail/openstack-dev/2016-December/109182.html 
  I've also copied Derek since I believe he was the one who found it 
originally.


The gist is that after about 3 months of tripleo-ci running in this 
cloud we start to hit errors creating instances because of problems 
creating OVS ports on the compute nodes.  Sometimes we see a huge number 
of ports in general, other times we see a lot of ports that look like this:


Port "qvod2cade14-7c"
     tag: 4095
     Interface "qvod2cade14-7c"

Notably they all have a tag of 4095, which seems suspicious to me.  I 
don't know whether it's actually an issue though.


Tag 4095 is for "dead" OVS ports, it's an unused VLAN tag in the agent.

The 'qvo' here shows it's part of the VETH pair that os-vif created when 
it plugged in the VM (the other half is 'qvb'), and they're created so 
that iptables rules can be applied by neutron.  It's part of the "old" 
way to do security groups with the OVSHybridIptablesFirewallDriver, and 
can eventually go away once the OVSFirewallDriver can be used everywhere 
(requires newer OVS and agent).


I wonder if you can run the ovs_cleanup utility to clean some of these up?

-Brian

I've had some offline discussions about getting someone on this cloud to 
debug the problem.  Originally we decided not to pursue it since it's 
not hard to work around and we didn't want to disrupt the environment by 
trying to move to later OpenStack code (we're still back on Mitaka), but 
it was pointed out to me this time around that from a downstream 
perspective we have users on older code as well and it may be worth 
debugging to make sure they don't hit similar problems.


To that end, I've left one compute node un-rebooted for debugging 
purposes.  The downstream discussion is ongoing, but I'll update here if 
we find anything.




Thanks,
Joe

On Mon, Dec 18, 2017 at 10:43 AM, Ben Nemec  
wrote:

Hi,

It's that magical time again.  You know the one, when we reboot rh1 
to avoid

OVS port exhaustion. :-)

If all goes well you won't even notice that this is happening, but 
there is

the possibility that a few jobs will fail while the te-broker host is
rebooted so I wanted to let everyone know.  If you notice anything else
hosted in rh1 is down (tripleo.org, zuul-status, etc.) let me know.  
I have

been known to forget to restart services after the reboot.

I'll send a followup when I'm done.

-Ben

__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Stepping down from core

[Brian Haley]

Armando,

It is sad to see you step down, just wanted to thank you for your 
leadership and hard work, both upstream and downstream, it's definitely 
made neutron a better project for everyone.  Good luck wherever life 
takes you.


-Brian

On 12/15/2017 02:01 PM, Armando M. wrote:

Hi neutrinos,

To some of you this email may not come as a surprise.

During the past few months my upstream community engagements have been 
more and more sporadic. While I tried hard to stay committed and fulfill 
my core responsibilities I feel like I failed to retain the level of 
quality and consistency that I would have liked ever since I stepped 
down from being the Neutron PTL back at the end of Ocata.


I stated many times when talking to other core developers that being 
core is a duty rather than a privilege, and I personally feel like it's 
way overdue for me to recognize on the mailing list that it's the time 
that I state officially my intention to step down due to other commitments.


This does not mean that I will disappear tomorrow. I'll continue to be 
on neutron IRC channels, support the neutron team, being the release 
liasion for Queens, participate at meetings, and be open to providing 
feedback to anyone who thinks my opinion is still valuable, especially 
when dealing with the neutron quirks for which I might be (git) blamed :)


Cheers,
Armando



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Propose Slawek Kaplonski for Neutron core

[Brian Haley]

+1 from me!

On 11/29/2017 02:21 PM, Miguel Lavalle wrote:

Hi Neutron Team,

I want to nominate Slawek Kaplonski (irc: slaweq) to Neutron core. 
Slawek has been an active contributor to the project since the Mitaka 
cycle. He has been instrumental in the development of the QoS 
capabilities in Neutron, becoming the lead of the sub-team focused on 
that set of features. More recently, he has collaborated in the 
implementation of OVO and is an active participant in the CI sub-team. 
His number of code reviews during the Queens cycle is on par with the 
leading core members of the team: http://stackalytics.com/?module=neutron


In my opinion, his efforts are highly valuable to the team and we will 
be very lucky to have him as a fully voting core.


I will keep this nomination open for a week as customary,

Thank you,

Miguel


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] Devstack -Ocata - Failed to launch instance on Host

[Brian Haley]

On 11/27/2017 10:24 AM, Ramu, MohanX wrote:

Hi All,

Not bale to launch an instance on Compute Node. instead of it, instance 
is trying to launch on the same controller.


The added compute node is not there in resource provider table.

Could you please help us what we missed.

Is placement API not configured on compute node side and Is there 
Placement API configurations coming by default along with basic Devstack 
set up?


There is a section on multinode setup in the devstack repo, 
doc/source/guides/multinode-lab.rst, that I think covers the issue 
you're seeing.  Assuming your compute node shows up in  'nova 
service-list' output, you need to run ./tools/discover_hosts.sh to have 
a cell mapping added for the node.


-Brian

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] Deleting one router in a project causes all routers to fail

[Brian Haley]

On 11/15/2017 12:58 PM, Sterdnot Shaken wrote:

Openstack version: Pike
Openvswitch version: 2.7

Let's say I have a OS project that has 2 routers. The routers are HA and 
reside on 2 network nodes. Via VRRP, there are 2 Active and 2 Passive 
routers. As you may know, Neutron creates a custom HA network that's 
project specific and (in our case) assigns a vxlan vni to that HA 
network that allows the routers to send VRRP messages to the backup 
routers basically telling them to stay backup.


If I delete just one of the routers from the project, the assigned vni 
(discovered via openstack network show) gets removed from a HA network. 
When that happens, ALL the other routers go down for the project as they 
can no longer communicate with each other via the HA network per the 
lacking vni assignment.


This problem didn't start happening until we upgraded to Pike.

Anyone have any idea why this is happening?


Looks like this bug perhaps:

https://bugs.launchpad.net/neutron/+bug/1732543

-Brian

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] [puppet][qa][ubuntu][neutron] Xenial Neutron Timeouts

[Brian Haley]

On 10/30/2017 05:46 PM, Matthew Treinish wrote:
 From a quick glance at the logs my guess is that the issue is related 
to this stack trace in the l3 agent logs:


http://logs.openstack.org/47/514347/1/check/puppet-openstack-integration-4-scenario001-tempest-ubuntu-xenial/ed5a657/logs/neutron/neutron-l3-agent.txt.gz?level=TRACE#_2017-10-29_23_11_15_146

I'm not sure what's causing it to complain there. But, I'm on a plane 
right now (which is why this is a top post, sorry) so I can't really dig 
much more than that. I'll try to take a deeper look at things later when 
I'm on solid ground. (hopefully someone will beat me to it by then though)


I don't think that l3-agent trace is it, as the failure is coming from 
the API.  It's actually a trace that's happening due to the async nature 
of how the agent runs arping, fix is 
https://review.openstack.org/#/c/507914/ but it only removes the log noise.


http://logs.openstack.org/47/514347/1/check/puppet-openstack-integration-4-scenario001-tempest-ubuntu-xenial/ed5a657/logs/neutron/neutron-server.txt.gz 
has some tracebacks that look config related, possible missing DB table? 
 But I haven't looked very closely.


-Brian


On October 31, 2017 1:25:55 AM GMT+04:00, Mohammed Naser 
 wrote:


Hi everyone,

I'm looking for some help regarding an issue that we're having with
the Puppet OpenStack modules, we've had very inconsistent failures in
the Xenial with the following error:

 
http://logs.openstack.org/47/514347/1/check/puppet-openstack-integration-4-scenario001-tempest-ubuntu-xenial/ed5a657/
 
http://logs.openstack.org/47/514347/1/check/puppet-openstack-integration-4-scenario001-tempest-ubuntu-xenial/ed5a657/logs/testr_results.html.gz
 Details: {u'message': u'Unable to associate floating IP
172.24.5.17   to fixed IP10.100.0.8  
 for instance
d265626a-77c1-4d2f-8260-46abe548293e. Error: Request to
https://127.0.0.1:9696/v2.0/floatingips/2e3fa334-d6ac-443c-b5ba-eeb521d6324c
timed out', u'code': 400}

At this point, we're at a bit of a loss.  I've tried my best in order
to find the root cause however we have not been able to do this.  It
was persistent enough that we elected to go non-voting for our Xenial
gates, however, with no fix ahead of us, I feel like this is a waste
of resources and we need to either fix this or drop CI for Ubuntu.  We
don't deploy on Ubuntu and most of the developers working on the
project don't either at this point, so we need a bit of resources.

If you're a user of Puppet on Xenial, we need your help!  Without any
resources going to fix this, we'd unfortunately have to drop support
for Ubuntu because of the lack of resources to maintain it (or
assistance).  We (Puppet OpenStack team) would be more than happy to
work together to fix this so pop-in at #puppet-openstack or reply to
this email and let's get this issue fixed.

Thanks,
Mohammed



OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] DHCP for IPv6

[Brian Haley]

BTW, the networking guide does mention this, found after Steve figure 
out what the problem was.


https://docs.openstack.org/ocata/networking-guide/config-ipv6.html#configuring-interfaces-of-the-guest

-Brian

On 09/28/2017 08:49 PM, Jorge Luiz Correa wrote:

Thanks for explain Jeremy! Very clear.

I think systems with cloud-init enabled, like most images, can be easily 
configured to disable this feature.

Thank you!
:)


On 28 Sep 2017, at 21:37, Jeremy Stanley  wrote:

On 2017-09-28 20:29:38 -0300 (-0300), Jorge Luiz Correa wrote:

It would be good if developers could know about that because
privacy extension is becoming the default on every operate
systems. I've tested last version of *ubuntu and some FreeBSD
kernels, all operating with privacy extension by default.

So, this way of creating the iptables rules need to be reviewed.

[...]

To accommodate privacy extensions, we'd basically have to give up on
any assumptions as to what the viable source addresses originating
on a port could be (at least within the netmask). This filtering is
the primary mechanism for preventing address spoofing within a
shared network.

By comparison, RFC 4941 privacy extensions are primarily a
protection for desktop/mobile client systems and do little (if
anything) useful for a statically-addressed server. Disabling it
there makes a lot of sense to me, as a privacy/security-conscious
sysadmin.
--
Jeremy Stanley
___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack



___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack



___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] Disable distributed loadbalancers (LBaaSv2)?

[Brian Haley]

On 09/16/2017 12:25 PM, Turbo Fredriksson wrote:

When I setup my OS cluster over a year ago, I chose to use
distributed LBaaSv2. That sounded like the most sensible
thing - redundancy is the primary goal with me choosing
OS in the first place!

However, it turned out that there’s a very grave bug in
OS - Neutron - (only just recently fixed - a few weeks ago and
only in the latest development code).

https://bugs.launchpad.net/neutron/+bug/1494003
https://bugs.launchpad.net/neutron/+bug/1493809
https://bugs.launchpad.net/neutron/+bug/1583694

I run Newton (and don’t want to risk everything by either
re-installing or upgrading - last time it took me two months
to get things working again!).

Doesn’t seem to be any backport of the fix to Newton :( :(.


Sorry, due to the invasiveness of the changes it won't be backported to 
Newton, only Pike will have this support.  It also might be slightly 
broken until very recent code in stable/pike...



Does anyone have an idea on how I can “hack” the DB
(MySQL) so that it isn’t distributed any more? The OS
command line tools won’t let you de-distribute one :(.

This should be “fairly” straight forward, for anyone that knows
the “inner workings” of Neutron. Simply “undo” whatever

 neutron router-create --distributed True --ha False rname

did. I can’t unfortunately delete the router and then recreate
it without destroying my whole setup, instances, networks,
etc, etc. Everything “hangs” off of that router...


I think you should be able to remove the router interfaces on the 
external and internal networks then remove the router, without removing 
any of the private networks, etc.  Then you can create it again with 
--distributed=False.  VMs might lose connectivity for a bit though.


Ocata supports DVR -> Centralized router migration, so you would only 
have to go forward one release if you choose that path.


-Brian

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] [Neutron] Denver Team Dinner

[Brian Haley]

+1

On 09/12/2017 10:44 PM, Ihar Hrachyshka wrote:

+1

On Tue, Sep 12, 2017 at 9:44 PM, Kevin Benton  wrote:

+1

On Tue, Sep 12, 2017 at 8:50 PM, Sławek Kapłoński 
wrote:


+1

—
Best regards
Slawek Kaplonski
sla...@kaplonski.pl





Wiadomość napisana przez Miguel Lavalle  w dniu
12.09.2017, o godz. 17:23:

Dear Neutrinos,

Our social event will take place on Thursday September 12th at 7:30pm.
The venue is going to be https://www.famousdaves.com/Denver-Stapleton. It is
located 0.4 miles from the Renaissance Hotel, which translates to an easy 9
minutes walk.

I have a reservation for a group of 30 people under my name. Please
respond to this message with your attendance confirmation by Wednesday
night, so I can get a more accurate head count.

Looking forward to see y'all Thursday night

Best regards

Miguel

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [devstack] [ironic] [nova] Trying again on wait_for_compute in devstack

[Brian Haley]

On 08/02/2017 07:17 AM, Sean Dague wrote:

The 3 node scenarios in Neutron (which are still experimental nv) are
typically failing to bring online the 3rd compute. In cells v2 you have
to explicitly add nodes to the cells. There is a nova-manage command
"discover-hosts" that takes all the compute nodes which have checked in,
but aren't yet assigned to a cell, and puts them into a cell of your
choosing. We do this in devstack-gate in the gate.

However... subnodes don't take very long to setup (so few services). And
the nova-compute process takes about 30s before it's done all it's
initialization and actually checks in to the cluster. It's a real
possibility that discover_hosts will run before subnode 3 checks in. We
see it in logs. This also really could come and bite us on any multinode
job, and I'm a bit concerned some of the multinode jobs aren't running
multinode some times because of it.

One way to fix this, without putting more logic in devstack-gate, is
ensure that by the time stack.sh finishes, the compute node is up. This
was tried previously, but it turned out that we totally missed that it
broke Ironic (the check happened too early, ironic was not yet running,
so we always failed), Cells v1 (munges hostnames :(  ), and PowerVM
(their nova-compute was never starting correctly, and they were working
around it with a restart later).

This patch https://review.openstack.org/#/c/488381/ tries again. The
check is moved very late, Ironic seems to be running fine with it. Cells
v1 is just skipped, that's deprecated in Nova now, and we're not going
to use it in multinode scenarios. The PowerVM team fixed their
nova-compute start issues, so they should be good to go as well.


I had also filed https://bugs.launchpad.net/neutron/+bug/1707003 for 
this since it was mainly just affecting that one 3-node neutron job. 
Glad I hadn't started working on a patch, I'll just take a look at yours.


Thanks for working on it!

-Brian


This is an FYI that we're going to land this again soon. If you think
this impacts your CI / jobs, please speak up. The CI runs on both the
main and experimental queue on devstack for this change look pretty
good, so I think we're safe to move forward this time. But we also
thought that the last time, and were wrong.

-Sean




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] Fwd: Unable to reach the vm from outside[from qrouter unable to reach both internal and external network]

[Brian Haley]

On 07/28/2017 05:54 AM, Amit Kumar wrote:

Hi,

Recently i have installed the openstack newton on Ubuntu 16.04. Brought 
up one controller and 2 compute nodes. All services are up and running 
unable to reach the vm from outside



I have logged in to the qrouter tried pinging the internal network as 
well the br-ex  unable to reach either side


Did you update the security groups to allow ICMP?

-Brian


ip netns

qrouter-3dbfca3a-1e95-43db-8901-4ee90d44e3f3
qdhcp-87453284-bf21-4c17-bddf-0742dd394573 (id: 2)
qdhcp-2faf6ae5-21bc-4c2a-8c8f-9bbec5cf2ae6 (id: 1)


ip netns exec qrouter-3dbfca3a-1e95-43db-8901-4ee90d44e3f3 ifconfig

loLink encap:Local Loopback
   inet addr:127.0.0.1  Mask:255.0.0.0
   inet6 addr: ::1/128 Scope:Host
   UP LOOPBACK RUNNING  MTU:65536  Metric:1
   RX packets:200 errors:0 dropped:0 overruns:0 frame:0
   TX packets:200 errors:0 dropped:0 overruns:0 carrier:0
   collisions:0 txqueuelen:1
   RX bytes:21596 (21.5 KB)  TX bytes:21596 (21.5 KB)

qg-3b436f92-71 Link encap:Ethernet  HWaddr fa:16:3e:cd:62:dd
   inet addr:172.24.4.5  Bcast:172.24.4.255  Mask:255.255.255.0
   inet6 addr: fe80::f816:3eff:fecd:62dd/64 Scope:Link
   UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
   RX packets:84 errors:0 dropped:0 overruns:0 frame:0
   TX packets:161 errors:0 dropped:0 overruns:0 carrier:0
   collisions:0 txqueuelen:1
   RX bytes:2972 (2.9 KB)  TX bytes:7558 (7.5 KB)

qr-9aeaa6a2-c2 Link encap:Ethernet  HWaddr fa:16:3e:0c:21:43
   inet addr:10.0.0.1  Bcast:10.0.0.255  Mask:255.255.255.0
   inet6 addr: fe80::f816:3eff:fe0c:2143/64 Scope:Link
   UP BROADCAST RUNNING MULTICAST  MTU:1450  Metric:1
   RX packets:0 errors:0 dropped:0 overruns:0 frame:0
   TX packets:84 errors:0 dropped:0 overruns:0 carrier:0
   collisions:0 txqueuelen:1
   RX bytes:0 (0.0 B)  TX bytes:4148 (4.1 KB)


_ovs- configuration at network node_

e0c37f1f-4c88-419d-a703-9f6b84325c08
 Manager "ptcp:6640:127.0.0.1"
 is_connected: true
 Bridge br-tun
 Controller "tcp:127.0.0.1:6633 "
 is_connected: true
 fail_mode: secure
 Port "gre-0a1f"
 Interface "gre-0a1f"
 type: gre
 options: {df_default="true", in_key=flow, 
local_ip="10.0.0.100", out_key=flow, remote_ip="10.0.0.31"}

 Port br-tun
 Interface br-tun
 type: internal
 Port patch-int
 Interface patch-int
 type: patch
 options: {peer=patch-tun}
 Bridge br-int
 Controller "tcp:127.0.0.1:6633 "
 is_connected: true
 fail_mode: secure
 Port int-br-ex
 Interface int-br-ex
 type: patch
 options: {peer=phy-br-ex}
 Port "qg-3b436f92-71"
 Interface "qg-3b436f92-71"
 type: internal
 Port "qr-9aeaa6a2-c2"
 tag: 1
 Interface "qr-9aeaa6a2-c2"
 type: internal
 Port patch-tun
 Interface patch-tun
 type: patch
 options: {peer=patch-int}
 Port br-int
 Interface br-int
 type: internal
 Bridge br-ex
 Controller "tcp:127.0.0.1:6633 "
 is_connected: true
 fail_mode: secure
 Port br-ex
 Interface br-ex
 type: internal
 Port phy-br-ex
 Interface phy-br-ex
 type: patch
 options: {peer=int-br-ex}
 ovs_version: "2.6.1"

ovs-configuration at compute node









--
Thanks & Regards
Amit Kumar


___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack




___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

[openstack-dev] [all] Proposal to change integrated neutron grenade gate job to multi-node

[Brian Haley]

Hi,

While looking at ways to reduce the number of jobs we run in the Neutron 
gate, I found we ran two very similar jobs for some projects:


gate-grenade-dsvm-neutron-ubuntu-xenial (single-node job)
gate-grenade-dsvm-neutron-multinode-ubuntu-xenial (2-node job)

We talked about this in the Neutron CI meeting this week [1] and felt it 
best to remove the single-node job and just run the multi-node job, 
mostly because it more mimics a "real" Neutron deployment where there 
are separate controller and compute nodes.  Looking at the Neutron 
grafana dashboard [2] the two jobs have about the same failure rate in 
the gate (~0), so I don't think there will be any problems with the switch.


This has an impact on the integrated gate since it currently runs the 
single-node job, so I wanted ot get thoughts on any issues they'd have 
with this change [3].


Thanks,

-Brian

[1] 
http://eavesdrop.openstack.org/meetings/neutron_ci/2017/neutron_ci.2017-07-11-16.00.log.html#l-112

[3] http://grafana.openstack.org/dashboard/db/neutron-failure-rate
[2] https://review.openstack.org/#/c/483600/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] Floating IP issues in multiple cloud installs.

[Brian Haley]

On 06/19/2017 08:51 AM, Ken D'Ambrosio wrote:
Hi, all.  We've got two Canonical Newton installs using VLANs and we're 
having intermittent issues we simply can't figure out.  (Note that a 
third installation using flat networks is not having this issue.) 
Floating IPs set up and work... sporadically.


* Stateful connections (e.g., SSH) often drop after seconds of use to 
both the FIP and when SSH'd in from the

* We see RSTs in our TCP dumps
* Pings work for a while, then don't.
* We see lots of ARP requests -- even one right after another -- to 
resolve hosts on the internal subnets:

05:43:25.859448 ARP, Request who-has 80.0.0.3 tell 80.0.0.1, length 28
05:43:25.859563 ARP, Reply 80.0.0.3 is-at fa:16:3e:28:af:77, length 28
05:43:25.964417 ARP, Request who-has 80.0.0.3 tell 80.0.0.1, length 46
05:43:25.964572 ARP, Reply 80.0.0.3 is-at fa:16:3e:28:af:77, length 28
05:43:26.963989 ARP, Request who-has 80.0.0.3 tell 80.0.0.1, length 46
05:43:26.964156 ARP, Reply 80.0.0.3 is-at fa:16:3e:28:af:77, length 28


Was that run with '-i any' or on a single interface?  I would check the 
ARP cache to make sure things entries are in a complete/reachable state. 
 Or even syslog for any other errors.


80.0.0.1 is the qrouter.  I can't imagine why it asked -- and was ACK'd 
in each case -- three times in just over a second.  In hindsight, I 
should have checked to have seen if the ACK showed up in the qrouter's 
ARP table.  Next time...


I'd also specify -e to tcpdump to see the MACs involved.  Possibly there 
is something else configured with the same IP on the VLAN (shouldn't 
happen, but worth checking).


-Brian

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

[openstack-dev] [neutron][l3][dvr] Pike tasks and meeting consolidation

[Brian Haley]

At the Summit a few of us met (Miguel, Swami and myself) to talk about our
Pike tasks now that we have lost some resources.  We came up with this
short list, which can also be found on
https://etherpad.openstack.org/p/neutron-l3-subteam :

1. Support for configuring Floating IPs on Centralized router in DVR
environment
https://bugs.launchpad.net/neutron/+bug/1667877
2. Spec for Floating IP subnets in routed networks
3. DHCP Agent support for remote IP subnets (via a DHCP relay)
https://bugs.launchpad.net/neutron/+bug/1668145

We also decided that instead of holding two weekly meetings, one for L3 and
another for DVR, we'd just cover any DVR-related issues in the L3 meeting
(Thursdays 1500 UTC) since neither was taking a full hour.  Anyone is
welcome to add items to the agenda and attend to discuss them.

Thanks,

-Brian
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][all] So long, and thanks for all the fish

[Brian Haley]

John,

Thanks so much for all your work on a number of L3 items that wouldn't 
exist without you, IPv6 PD comes to mind :)  It's been great working 
with you, good luck!


-Brian

On 05/02/2017 06:07 AM, John Davidge wrote:

Friends and colleagues,

It is with a heavy heart that I write to say my adventure in the OpenStack
community is coming to an end. It began in 2012 with my first job as an
intern at Cisco, and ends here as the Technical Lead for Neutron in the
OpenStack Innovation Center at Rackspace.

In that time I’ve worked with a great many wonderful people from all
corners of this community, on a variety of projects that I’m proud to
include my name in the commit logs. Thank you all for creating an exciting
place to work, to debate, and occasionally to argue about the incredible
democratizing power that is OpenStack. Your passion and expertise are an
inspiration to the world.

Regretfully, I’m leaving a void in both the Neutron team and the OpenStack
Manuals team. Neutron will need a new Docs liaison, and OpenStack Manuals
will need a new lead for the Networking Guide. The cross-project work
we’ve done together over the last couple of cycles has been engaging and
fulfilling, and I encourage anyone interested in either or both roles to
get in touch with Kevin Benton and Alexandra Settle.

Good luck and best wishes to all of you in the future.

Until we meet again,

John



Rackspace Limited is a company registered in England & Wales (company 
registered number 03897010) whose registered office is at 5 Millington Road, Hyde 
Park Hayes, Middlesex UB3 4AZ. Rackspace Limited privacy policy can be viewed at 
www.rackspace.co.uk/legal/privacy-policy - This e-mail message may contain 
confidential or privileged information intended for the recipient. Any 
dissemination, distribution or copying of the enclosed material is prohibited. If 
you receive this transmission in error, please notify us immediately by e-mail at 
ab...@rackspace.com and delete the original message. Your cooperation is 
appreciated.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] LinuxBridge Agent Error

[Brian Haley]

On 04/11/2017 06:35 PM, Daniel Russell wrote:

This is somewhat unrelated to the original email, but more of a question of an 
issue I have recently come across that seems tangentially related.

When run_one_command is executed and the message is sent to rootwrap via that pipe, if 
rootwrap errors for some reason, I get " Unserializable message: ('#ERROR',  
ValueError('I/O operation on closed file',))" returned with a stack trace.

In this case it appears the problem was that conntrack wasn't installed.  In my 
case it was that the rootwrap filters denied the request to kill the process.  
I only tracked it down because I configured syslog on the rootwrap daemon and 
killed the process (so l3-agent would respawn it, retry the command, and error 
again).  The error then appeared very clearly in /var/log/messages, so rootwrap 
does understand what is going on but it doesn't seem to feed it back to the 
caller.

Is this a known listed bug?  It would make diagnosis a lot easier if an easy to 
understand error message came back instead of the above.


Hi Dan,

https://bugs.launchpad.net/neutron/+bug/1677742 was the bug I opened for 
the original rootwrap error, and the change for that merged upstream. 
That will catch the error and print the command being tried, which is at 
least a bread crumb and finding the failure.  Could be there is more 
info we can gather, if you can try that change and add info to the bug 
it would be great.


-Brian



-Original Message-----
From: Brian Haley [mailto:haleyb@gmail.com]
Sent: Tuesday, 11 April 2017 6:24 AM
To: openstack@lists.openstack.org
Subject: Re: [Openstack] LinuxBridge Agent Error

On 04/10/2017 04:15 PM, Georgios Dimitrakakis wrote:

Installing the "conntrack-tools" package from the
"centos-openstack-ocata" repository seems to have fixed the problem.

My question is if this should have happened automatically during
installation and if yes why it didn't happen?


 From your comment it was a manual install.  Looking at the 
neutron-sanity-check tool it doesn't seem to check for conntrack.  Fix proposed 
at https://review.openstack.org/455450

Thanks!

-Brian



On Fri, 07 Apr 2017 19:26:02 +0300, Georgios Dimitrakakis wrote:

Hello,

I am on CentOS 7 and this is a manual Ocata installation as described
on the deployment guide.

My cloud seems to be operating normally even with these messages. The
thing is that I don't like all these (there are a whole bunch of them
every second) in my log files.

I will try to see if I am missing the "conntrack" and try to install
the patch that will provide more info..


Best,

G.


Is this a packstack installation?

Inviato da iPhone

Il giorno 07 apr 2017, alle ore di 07:19, Kevin Benton  ha scritto:


You may be missing whatever package in your distro provides the
'conntrack' util.

On Apr 7, 2017 03:52, "Georgios Dimitrakakis" wrote:


Hello,

I am having an Ocata installation and on my Compute Node the
"linuxbridge-agent.log" file is constantly filling out with
messages like this:

2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[1]ent._common_agent Traceback (most recent call last):
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[2]ent._common_agent File





"/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/agent/
_common_agent.py",


line 453, in daemon_loop
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[3]ent._common_agent sync =
self.process_network_devices(device_info)
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[4]ent._common_agent File
"/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line
153, in wrapper
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[5]ent._common_agent return f(*args, **kwargs)
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[6]ent._common_agent File





"/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/agent/
_common_agent.py",


line 213, in process_network_devices
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[7]ent._common_agent resync_b =
self.treat_devices_removed(device_info['removed'])
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[8]ent._common_agent File
"/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line
153, in wrapper
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[9]ent._common_agent return f(*args, **kwargs)
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[10]ent._common_agent File





"/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/agent/
_common_agent.py",


line 331, in treat_devices_removed
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[11]ent._common_agent self.sg_agent.remove_devices_filter(devices)
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[12]

Re: [Openstack] LinuxBridge Agent Error

[Brian Haley]

On 04/10/2017 04:15 PM, Georgios Dimitrakakis wrote:

Installing the "conntrack-tools" package from the
"centos-openstack-ocata" repository seems to have fixed the problem.

My question is if this should have happened automatically during
installation and if yes why it didn't happen?


From your comment it was a manual install.  Looking at the 
neutron-sanity-check tool it doesn't seem to check for conntrack.  Fix 
proposed at https://review.openstack.org/455450


Thanks!

-Brian



On Fri, 07 Apr 2017 19:26:02 +0300, Georgios Dimitrakakis wrote:

Hello,

I am on CentOS 7 and this is a manual Ocata installation as described
on the deployment guide.

My cloud seems to be operating normally even with these messages. The
thing is that I don't like all these (there are a whole bunch of them
every second) in my log files.

I will try to see if I am missing the "conntrack" and try to install
the patch that will provide more info..


Best,

G.


Is this a packstack installation?

Inviato da iPhone

Il giorno 07 apr 2017, alle ore di 07:19, Kevin Benton  ha scritto:


You may be missing whatever package in your distro provides the
'conntrack' util.

On Apr 7, 2017 03:52, "Georgios Dimitrakakis" wrote:


Hello,

I am having an Ocata installation and on my Compute Node the
"linuxbridge-agent.log" file is constantly filling out with
messages like this:

2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[1]ent._common_agent Traceback (most recent call last):
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[2]ent._common_agent File





"/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py",


line 453, in daemon_loop
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[3]ent._common_agent sync =
self.process_network_devices(device_info)
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[4]ent._common_agent File
"/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line
153, in wrapper
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[5]ent._common_agent return f(*args, **kwargs)
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[6]ent._common_agent File





"/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py",


line 213, in process_network_devices
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[7]ent._common_agent resync_b =
self.treat_devices_removed(device_info['removed'])
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[8]ent._common_agent File
"/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line
153, in wrapper
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[9]ent._common_agent return f(*args, **kwargs)
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[10]ent._common_agent File





"/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py",


line 331, in treat_devices_removed
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[11]ent._common_agent self.sg_agent.remove_devices_filter(devices)
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[12]ent._common_agent File





"/usr/lib/python2.7/site-packages/neutron/agent/securitygroups_rpc.py",

line 238, in remove_devices_filter
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[13]ent._common_agent self.firewall.remove_port_filter(device)
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[14]ent._common_agent File





"/usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_firewall.py",


line 222, in remove_port_filter
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[15]ent._common_agent
self._remove_conntrack_entries_from_port_deleted(port)
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[16]ent._common_agent File





"/usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_firewall.py",


line 194, in _remove_conntrack_entries_from_port_deleted
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[17]ent._common_agent [device_info], ethertype, set())
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[18]ent._common_agent File





"/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_conntrack.py",

line 121, in delete_conntrack_state_by_remote_ips
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[19]ent._common_agent
self._delete_conntrack_state(device_info_list, rule)
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[20]ent._common_agent File





"/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_conntrack.py",

line 103, in _delete_conntrack_state
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[21]ent._common_agent extra_ok_codes=[1])
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
[22]ent._common_agent File

Re: [Openstack] LinuxBridge Agent Error

[Brian Haley]

On 04/07/2017 07:19 AM, Kevin Benton wrote:

You may be missing whatever package in your distro provides the
'conntrack' util.


Like Kevin said, it's probably missing conntrack.  There is an upstream 
patch that you might be able to apply to show a better message, 
https://review.openstack.org/#/c/453278/ - that will be backported to 
stable/ocata once it lands.


-Brian


On Apr 7, 2017 03:52, "Georgios Dimitrakakis" > wrote:

Hello,

I am having an Ocata installation and on my Compute Node the
"linuxbridge-agent.log" file is constantly filling out with messages
like this:

2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent Traceback
(most recent call last):
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent   File

"/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py",
line 453, in daemon_loop
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent sync =
self.process_network_devices(device_info)
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent   File
"/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 153,
in wrapper
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent return
f(*args, **kwargs)
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent   File

"/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py",
line 213, in process_network_devices
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent
 resync_b = self.treat_devices_removed(device_info['removed'])
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent   File
"/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 153,
in wrapper
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent return
f(*args, **kwargs)
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent   File

"/usr/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py",
line 331, in treat_devices_removed
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent
 self.sg_agent.remove_devices_filter(devices)
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent   File
"/usr/lib/python2.7/site-packages/neutron/agent/securitygroups_rpc.py",
line 238, in remove_devices_filter
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent
 self.firewall.remove_port_filter(device)
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent   File
"/usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_firewall.py",
line 222, in remove_port_filter
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent
 self._remove_conntrack_entries_from_port_deleted(port)
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent   File
"/usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_firewall.py",
line 194, in _remove_conntrack_entries_from_port_deleted
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent
 [device_info], ethertype, set())
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent   File
"/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_conntrack.py",
line 121, in delete_conntrack_state_by_remote_ips
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent
 self._delete_conntrack_state(device_info_list, rule)
2017-04-07 13:37:47.709 1397 ERROR neutron.plugins.ml2.drivers.ag
ent._common_agent   File
"/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_conntrack.py",
line 103, in 

Re: [openstack-dev] [neutron][networking-odl][qa] What's the recommended behavior for SG Rules with ethertype='IPv6' and protocol='icmp' ?

[Brian Haley]

On 04/03/2017 09:23 PM, Ghanshyam Mann wrote:

On Sun, Mar 26, 2017 at 9:31 PM, Sridhar Gaddam <sgad...@redhat.com
<mailto:sgad...@redhat.com>> wrote:

On Fri, Mar 24, 2017 at 7:27 PM, Brian Haley <haleyb@gmail.com
<mailto:haleyb@gmail.com>> wrote:

On 03/24/2017 06:41 AM, Ghanshyam Mann wrote:

Hi All,

Tempest is testing SG rule creation and pinging scenario
tests with
ethertype='IPv6' and protocol='icmp' [0].
In case of ethertype='IPv6', currently neutron accept
protocol type
as 'icmp', 'icmpv6' and 'ipv6-icmp' which again seems like
duplication
of SG rules bug on neutron side but not sure [1]

But it seems like some driver does not work with 'icmp' on
IPv6, at
least ODL as mentioned in bug [2]. Where few others like ML2/OVS
iptables driver convert 'icmp' to 'icmpv6' when
ethertype='IPv6' and had
no issue with 'icmp'.

IMO neutron should keep accepting 'icmp' for IPv6 for backward
compatibility and legacy usage and tempest should test
'icmp' also along
with other protocol type.
But we need more feedback on that what is right way (as per
backward
compatibility pov) and recommended way for having best
behaviour for SG
rules on IPv6. What best can work for all plugins also?


Thanks for raising this issue.  Let me just restate it a little
so it's clear.

1. One can create an IPv6 rule using protocol value "icmp"
today, and the base security group code does the right thing
changing the rule to be correct for the underlying
implementation, for example, "ipv6-icmp" for iptables.  It
doesn't look like all other drivers handle this properly.

​Well, let the Neutron API accept multiple values like
"icmp/icmpv6/ipv6-icmp", but IMO it should store a single Security
Group rule in the DB and raise "Duplicate error when similar rule is
configured once again".
Currently, Neutron treats each of them as a different Security Group
rule and stores them as separate entries in the DB.
However, IPtables Firewall driver in Neutron is converting[1] the
"ethertype=IPv6 and protocol=icmp" as a request to ICMPv6 and
applying the necessary ip-table rule.

https://github.com/openstack/neutron/blob/stable/newton/neutron/agent/linux/iptables_firewall.py#L373

<https://github.com/openstack/neutron/blob/stable/newton/neutron/agent/linux/iptables_firewall.py#L373>

Since this is not a documented behavior, other firewall drivers
(which I guess could be an issue even with OVS firewall driver) may
be missing this info.​

​++ for this, documentation could have helped this better way. ​

2. The neutron API will accept multiple values - "icmp",
"ipv6-icmp" and "icmpv6" for an IPv6 rule, but it will create
unique database entries for each (I just verified that).  While
that shouldn't create multiple entries in the base iptables
code, it will probably generate a warning in the logs about a
duplicate being suppressed.


So there are a few things that could be done:

1. Drivers need to accept "icmp" in order to be
backwards-compatible with the current code.

2. Duplicates should be detected and generate 409 (?) errors.

3. We should add a migration (IMO) where any duplicates are
squashed.

​Agree with your points.


I just created https://review.openstack.org/453346 as a start at #2, 
will try and add code to squash existing duplicates too (#3).


-Brian



​Yes, 409 on same type again make sense. And it can be documented
properly that 'icmp'​ will be treated same as other protocol type for
​
Ethertype=IPv6 and user will get 409 if they try to create and expect 2
different SG rules with those different protocol_type.
This is something change in current behavior where both requests('icmp'
and 'icmpv6') are treated as 2 separate rules. And so this new Tempest
tests will fail [1] which can be modified later.

With those points and current situation, I feel Tempest should tests the
current behavior proposed in [1] which will discover the drivers for
point 1. Later based on bug [2] consensus we can change the tests
accordingly.
I want to merge Tempest changes so that while changing anything on
neutron side, we know exactly what behavior we are going to change and
its impact etc.
Please leave comment on patch if any more feedback.



​

The open question is, do we want to change the DB to have a
different value here, like "icmpv6" ?  We could obviously add a
migration where we

Re: [openstack-dev] [neutron] - Adding Miguel Lavalle to neutron-drivers team

[Brian Haley]

+1 for Miguel!

On 03/30/2017 05:11 PM, Kevin Benton wrote:

Hi,

I would like to add Miguel Lavalle (mlavalle) to the Neutron drivers
team [1].

Miguel has been instrumental in implemented features across Neutron and
Nova (routed networks, DNS, etc) and is now leading the L3 team. He has
a very good high level architectural view of Neutron necessary for
deciding what features are feasible for the platform.



1. 
http://docs.openstack.org/developer/neutron/policies/neutron-teams.html#drivers-team

Cheers,
Kevin Benton


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] Neutron Network DNS Suffix via DHCP

[Brian Haley]

On 03/28/2017 08:39 PM, Oisin O'Malley wrote:


There was 2 separate issues to resolve;

Firstly Nova was appending the default domain name .novalocal to the hostname 
it presents via the meta-data service. This can be resolved by setting 
dhcp_domain to an empty string in nova.conf on the Control node. For instance 
'dhcp_domain='. An instances name can now be set to a FQDN which will then be 
passed cloud-init via the metadata server.

Secondly, The Neutron DHCP service sets the default DNS suffix for a NIC to openstacklocal . 
This causes delays in DNS lookups on external DNS servers, as the wrong domain is used by 
default. Similarly to the above, this can be resolve by setting 'dhcp_domain=' in the Neutron 
DHCP config file dhcp_agent.ini. Once this is set and the DHCP service restarted, the 
"--domain=" parameter no longer gets set


Just an FYI that 'dhcp_domain' had been marked for deprecation and was 
finally removed in Ocata.  The new value is 'dns_domain', which lives in 
neutron.conf - the dhcp agent was changed to pick-up the new value.


-Brian



on the DHCP agents dnsmasq and no default search suffix gets passed via DHCP.

Setting dns_domain Neutron network attribute, appears to do nothing at the 
moment.

Regards,
Oisin


On 03/26/2017 11:49 PM, Matthew Taylor wrote:
Responded off-list.

For the benefit of the community, would one of you care to repeat the answer 
on-list please?

Thanks!
-jay

On 27/3/17 14:22, Oisin O'Malley wrote:

Hi All,

What is the correct way to set an instances DNS search suffix via
DHCP, currently all instances receive the default openstacklocal  DNS
search space.  We are using OpenStack Newton with Neutron Networking.

Setting dhcp_domain in dhcp_agent.ini will set the value globally for
all networks, which is of little use as we host many Windows VMs with
their own domains and DNS servers. Whatever is set as dhcp_domain is
passed to Neutron DHCP Agent dnsmasq subprocess via a
--domain= parameter.

With the Neutron DNS extension enabled, you can set the a networks
dns_domain attribute with "neutron net-update --dns-name", though
this attribute appears to be ignored by the DHCP server. Can this be
used to specify the DNS search space, if so how can it be configured?
I need to be able to configure this on a per network/subnet level.

Regards,
Oisin

Oisin O'Malley
Systems Engineer
Iocane Pty Ltd
763 South Road
Black Forest SA 5035

Office:+61 (8) 8413 1010
Fax:+61 (8) 8231 2050
Email:oisin.omal...@iocane.com.au
Web:www.iocane.com.au


___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack




___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

[openstack-dev] [neutron] No DVR meeting today

[Brian Haley]

Both chairs have conflicts today, will resume the DVR meeting next week. 
 Current work items and bugs can be found on the meeting page as usual, 
https://wiki.openstack.org/wiki/Meetings/Neutron-DVR


-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][networking-odl][qa] What's the recommended behavior for SG Rules with ethertype='IPv6' and protocol='icmp' ?

[Brian Haley]

On 03/24/2017 06:41 AM, Ghanshyam Mann wrote:

Hi All,

Tempest is testing SG rule creation and pinging scenario tests with
ethertype='IPv6' and protocol='icmp' [0].
In case of ethertype='IPv6', currently neutron accept protocol type
as 'icmp', 'icmpv6' and 'ipv6-icmp' which again seems like duplication
of SG rules bug on neutron side but not sure [1]

But it seems like some driver does not work with 'icmp' on IPv6, at
least ODL as mentioned in bug [2]. Where few others like ML2/OVS
iptables driver convert 'icmp' to 'icmpv6' when ethertype='IPv6' and had
no issue with 'icmp'.

IMO neutron should keep accepting 'icmp' for IPv6 for backward
compatibility and legacy usage and tempest should test 'icmp' also along
with other protocol type.
But we need more feedback on that what is right way (as per backward
compatibility pov) and recommended way for having best behaviour for SG
rules on IPv6. What best can work for all plugins also?


Thanks for raising this issue.  Let me just restate it a little so it's 
clear.


1. One can create an IPv6 rule using protocol value "icmp" today, and 
the base security group code does the right thing changing the rule to 
be correct for the underlying implementation, for example, "ipv6-icmp" 
for iptables.  It doesn't look like all other drivers handle this properly.


2. The neutron API will accept multiple values - "icmp", "ipv6-icmp" and 
"icmpv6" for an IPv6 rule, but it will create unique database entries 
for each (I just verified that).  While that shouldn't create multiple 
entries in the base iptables code, it will probably generate a warning 
in the logs about a duplicate being suppressed.



So there are a few things that could be done:

1. Drivers need to accept "icmp" in order to be backwards-compatible 
with the current code.


2. Duplicates should be detected and generate 409 (?) errors.

3. We should add a migration (IMO) where any duplicates are squashed.

The open question is, do we want to change the DB to have a different 
value here, like "icmpv6" ?  We could obviously add a migration where we 
update the value.  The problem is that flag day could pose a problem if 
out-of-tree drivers don't support the new value.  I think we should 
leave it "icmp" for that reason, thoughts from others?


-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] Mitaka Neutron DVR namespace not communicating with DHCP and LbaaS namespace with VLAN isolation

[Brian Haley]

On 02/13/2017 12:26 PM, kevin parrikar wrote:

hello All,
I just installed Mitaka release with DVR and ml2+OVS,everything looks fine .

vms are getting ip address,floating IP address is working,vms can talk to lbaas
namespace

How ever:

Communication from qrouter-namespace to lbass and dhcp namespace are not
 working,because of that floating ip assigned on lbaas is not working.I am not
sure whats is broken,we are using VLAN segmentation,Even tried using GRE but
that didnt make any difference.

i am new to DVR and in my juno environment without DVR, DHCP namespace can talk
to lbaas and router namespace.

on one of the compute node:


ip netns exec qrouter-602e4399-1949-4f7f-9351-0921064a111b ip l

1: lo:  mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
20: qr-0d106417-3b:  mtu 9000 qdisc noqueue
state UNKNOWN mode DEFAULT group default


Are you running into an MTU issue?  I see 9000 here which is not typical, in 
newton this is usually 1450 w/VXLAN overlay to account for overhead.


Something like ping should still work though, so the first things I'd check are:

1) Is there a complete ARP entry in that router namespace for the DHCP IP? 
Maybe you need to enable l2pop?


2) If the packet is making it out of the namespace and onto the bridge, is there 
an iptables rule possibly dropping the packet?


Swami and I (haleyb) are typically on the #openstack-neutron channel if we don't 
respond to email quickly.


-Brian



ovs-vsctl list-ifaces br-int
fg-70b8f7d1-1e
int-br-floating
int-br-prv
qr-0d106417-3b
qr-d1b616f7-45
qvo61d4d7d2-83
qvoc1f28f85-f6

 ovs-vsctl show |grep  -A 2 qr-0d106417-3b
Port "qr-0d106417-3b"
tag: 1
Interface "qr-0d106417-3b"
type: internal
Port "qvo61d4d7d2-83"

any idea why DVR is not able to talk to qdhcp and qlbaas ,but vms can talk to
both namespaces.

Regards,
Kevi


___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack




___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] [all] [goals] proposing a new goal: "Control Plane API endpoints deployment via WSGI"

[Brian Haley]

On 01/12/2017 08:40 PM, Emilien Macchi wrote:

Greetings OpenStack community,

I have been looking for a Community Goal [1] that would directly help
Operators and I found the "run API via WSGI" useful.
So I've decided to propose this one as a goal for Pike but I'll stay
open to postpone it to Queen if our community thinks we already have
too much goals for Pike.

Note that this goal might help to achieve 2 other goals later:
- enable and test SSL everywhere
- enable and test IPv6 everywhere


I know IPv6 is a secondary goal, but since this was previously working in 
devstack 
(https://www.openstack.org/summit/tokyo-2015/videos/presentation/deploying-and-operating-an-ipv6-only-openstack-cloud) 
I'll try and help out getting that working again 
(https://bugs.launchpad.net/devstack/+bug/1656329).  Added a line to the 
etherpad as well.  I know there's more to do than just that, but if we had a job 
in place to catch regressions at least we'd have a baseline that *something* works.


-Brian



Here's the draft:
https://review.openstack.org/419706

Any feedback is very welcome, thanks for reading so far.

[1] https://etherpad.openstack.org/p/community-goals




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] [Neutron][DHCP] vm can not get ip by dnsmasq

[Brian Haley]

On 01/04/2017 03:22 AM, zhaolihuisky wrote:

hi, all

I have a problem about that vm can not get ip address by dnsmasq server.
I have got the DHCP packets by tcpdump.
With the DHCP protocol, client send DHCP Discover, and then server reply DHCP
Offer, next client send DHCP Request and server reply DHCP ACK, at last vm get
ip address.
But in my openstack env, only one vm can not get ip address, and others could
get a ip address.
By analysing the DHCP protocol packets (see attachment), I found that vm send
DHCP Discover, but dnsmasq didn't reply DHCP Offer.


Did dnsmasq log anything on receipt of the packet?  Does the host file in 
/opt/stack/data/neutron/dhcp/* have information for this VM?  Do you have any 
iptables rules that might be causing it?  I'm assuming the trace was taken 
inside the dhcp namespace so you know the packet got that far?


Since you can reproduce it with this one VM you can hopefully track it down, but 
there's nothing in your explanation that points at a particular issue in neutron.


-Brian

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] [NEUTRON][DHCP] iptables chain rate limit dhcp connection

[Brian Haley]

On 12/31/2016 03:58 AM, Davide Panarese wrote:

Sorry Gary,
i don’t use this driver and however i can’t find in the code.


The upstream neutron code does not add these rules, so they are either in a 
driver or vendor-specific distro code.


-Brian



On 30 Dec 2016, at 20:00, Gary Kotton  wrote:

https://github.com/openstack/vmware-nsx

On 12/30/16, 6:36 PM, "Davide Panarese"  wrote:

   Hello everyone,
   anyone know where is the source code of neutron that create seguent CHAIN in 
iptables into dhcp namespaces?

   -A INPUT -p udp -m udp --dport 67 -m hashlimit --hashlimit-above 5/min 
--hashlimit-burst 5 --hashlimit-mode srcip --hashlimit-name LIMIT_DHCP_UDP -j 
DROP
   -A INPUT -p tcp -m tcp --dport 67 -m hashlimit --hashlimit-above 5/min 
--hashlimit-burst 5 --hashlimit-mode srcip --hashlimit-name LIMIT_DHCP_TCP -j 
DROP

   Thanks a lot

   Davide
   ___
   Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
   Post to : openstack@lists.openstack.org
   Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack


___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

--
Questo messaggio e' stato analizzato con Libra ESVA ed e' risultato non infetto.
Seguire il link qui sotto per segnalarlo come spam:
http://mx01.enter.it/cgi-bin/learn-msg.cgi?id=086555DF5A.A970F





___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack




___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] [neutron] proposing Miguel Lavalle as neutron core and Brian Haley as L3 Lt

[Brian Haley]

+1 for Miguel, he's been doing a great job :)

On 12/15/2016 06:14 PM, Armando M. wrote:

Hi neutrinos,

Miguel Lavalle has been driving the project forward consistently and reliably. I
would like to propose him to be entrusted with +2/+A rights in the areas he's
been most prolific, which are L3 and DHCP.

At the same time, I'd like to propose Brian Haley as our next Chief L3 Officer.
Both of them have worked with Carl Baldwin extensively and that can only be a
guarantee of quality.

Cheers,
Armando

[1] https://review.openstack.org/#/c/411531/


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] proposing Ryan Tidwell and Nate Johnston as service LTs

[Brian Haley]

+1

On 12/15/2016 06:58 PM, Armando M. wrote:

Hi neutrinos,

I would like to propose Ryan and Nate as the go-to fellows for service-related
patches.

Both are core in their repos of focus, namely neutron-dynamic-routing and
neutron-fwaas, and have a good understanding of the service framework, the agent
framework and other bits and pieces. At this point, entrusting them with the
responsibility is almost a formality.

Cheers,
Armando

[1] https://review.openstack.org/#/c/411536/


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] DVR ARP cache update loop delaying launch of metadata proxy

[Brian Haley]

On 12/13/2016 02:45 PM, Gustavo Randich wrote:

Hi Openstackers,

We have the folowing issue (using Mitaka / DVR / Xenial), perhaps someone can
help ;)

When our hosts boots up, the ARP cache population loop of L3 Agent is delaying
the start of neutron-ns-metadata-proxy for around a minute -- see logs below;
then, when nova-compute launches VMs, all of cloud-init runs fail with timeout
when reading metadata


Hi Gustavo,

We had seen a similar slowdown with DVR and the ARP cache, see:

https://bugs.launchpad.net/neutron/+bug/1511134
https://review.openstack.org/#/c/239543/

We decided against that approach in favor of using privsep and the pyroute2 
library.  That adoption has not been as fast as we hoped, so it is probably time 
to re-visit this decision and possibly resurrect that change.


-Brian



To workaround this, we've made a systemd unit on which nova-compute is
dependent; this unit waits for ns-metadata-proxy process to appear, and only
then nova-compute starts

Curiously, in dvr_local_router.py, in _update_arp_entry function, there is a
comment saying "# TODO(mrsmith): optimize the calls below for bulk calls"...

By now we have a single virtual router with 170 VMs, but the number of VMs will
grow, so my questions are

Should this be issue of concern?

Is there a better / faster / bulk way to execute those "ip neigh" commands?

Or simply, metadata proxy should launch before ARP cache population?




PD: I've also seen (obviously) this ARP cache population in the L3 agent of
Neutron Nodes, and I hope it does not affect / delay the HA failover
mechanism... (didn't test yet)




# journalctl -u neutron-l3-agent | grep "COMMAND=/usr/bin/neutron-rootwrap
/etc/neutron/rootwrap.conf" | sed 's,neutron : TTY=unknown ;
PWD=/var/lib/neutron ; USER=root ; COMMAND=/usr/bin/neutron-rootwrap
/etc/neutron/rootwrap.conf,,g' | head -25

Dec 13 13:33:43 e71-host15 sudo[20157]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 neutron-ns-metadata-proxy
--pid_file=/var/lib/neutron/external/pids/6149559f-fa54-493c-bf37-7d1827181228.pid
--metadata_proxy_socket=/var/
Dec 13 13:33:55 e71-host15 sudo[20309]:   ip -o netns list
Dec 13 13:33:55 e71-host15 sudo[20315]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 sysctl -w net.ipv4.ip_forward=1
Dec 13 13:33:55 e71-host15 sudo[20322]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 sysctl -w
net.ipv6.conf.all.forwarding=1
Dec 13 13:33:56 e71-host15 sudo[20331]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 ip -o link show rfp-6149559f-f
Dec 13 13:33:56 e71-host15 sudo[20336]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 ip -o link show qr-24f3070a-d4
Dec 13 13:33:56 e71-host15 sudo[20342]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 ip -o link show qr-24f3070a-d4
Dec 13 13:33:56 e71-host15 sudo[20345]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 ip addr show qr-24f3070a-d4 
permanent
Dec 13 13:33:56 e71-host15 sudo[20348]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 ip -4 route list dev qr-24f3070a-d4
scope link
Dec 13 13:33:56 e71-host15 sudo[20354]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 ip -6 route list dev qr-24f3070a-d4
scope link
Dec 13 13:33:56 e71-host15 sudo[20357]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 arping -A -I qr-24f3070a-d4 -c 3 -w
4.5 10.96.0.1
Dec 13 13:33:57 e71-host15 sudo[20368]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 ip -o link show qr-24f3070a-d4
Dec 13 13:33:57 e71-host15 sudo[20372]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 ip -4 neigh replace 10.96.0.100
lladdr fa:16:3e:1b:d6:cd nud permanent dev qr-24f3070a-d4
Dec 13 13:33:57 e71-host15 sudo[20375]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 ip -o link show qr-24f3070a-d4
Dec 13 13:33:57 e71-host15 sudo[20378]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 ip -4 neigh replace 10.96.0.101
lladdr fa:16:3e:b4:12:28 nud permanent dev qr-24f3070a-d4
Dec 13 13:33:58 e71-host15 sudo[20384]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 ip -o link show qr-24f3070a-d4
Dec 13 13:33:58 e71-host15 sudo[20387]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 ip -4 neigh replace 10.96.0.102
lladdr fa:16:3e:3f:bb:58 nud permanent dev qr-24f3070a-d4
Dec 13 13:33:58 e71-host15 sudo[20390]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 ip -o link show qr-24f3070a-d4
Dec 13 13:33:58 e71-host15 sudo[20393]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 ip -4 neigh replace 10.96.0.103
lladdr fa:16:3e:5a:90:67 nud permanent dev qr-24f3070a-d4
Dec 13 13:33:58 e71-host15 sudo[20399]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 ip -o link show qr-24f3070a-d4
Dec 13 13:33:58 e71-host15 sudo[20402]:   ip netns exec
qrouter-6149559f-fa54-493c-bf37-7d1827181228 ip -4 neigh replace 10.96.0.104

[openstack-dev] [Neutron][DVR] - No IRC Meeting until 2017

[Brian Haley]

Hi Folks,

We will not have another DVR sub-team meeting until 2017, resuming on January 
4th, to accommodate those that will be away.


Enjoy the break!

-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Proposing Abhishek Raut as neutronclient core

[Brian Haley]

+1

On 12/13/16 8:22 PM, Armando M. wrote:

Hi folks,

Abhishek Raut's recent involvement in OSC and python-neutronclient has
helped moving a few efforts along in the right direction. I would like
to suggest we double down on core firepower for the neutronclient repo
alongside Akihiro [1].

This not only will help speed up our transition to OSC CLI, but also
improve the number of folks who can effectively liasion with the OSC
team, and look after the needs of neutron's client repo.

Many thanks,
Armando

[1] https://review.openstack.org/#/c/410485/


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][tricircle]DVR issue in cross Neutron networking

[Brian Haley]

On 12/05/2016 10:38 PM, joehuang wrote:

Hello, Brian,

Thank you for your comment, see inline comments marked with [joehuang].

The ASCII figure is not good in the plain text mail, you can check it at the 
browser: 
http://lists.openstack.org/pipermail/openstack-dev/2016-December/108447.html

Best Regards
Chaoyi Huang (joehuang)


From: Brian Haley [brian.ha...@hpe.com]
Sent: 06 December 2016 10:29
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron][tricircle]DVR issue in cross Neutron 
networking

On 12/5/16 3:03 AM, joehuang wrote:

Hello,


Hi Chaoyi,

Comments inline below.


 Tricircle plans to provide L2 network across Neutron to ease supporting
high
 availability of application:

 For example, in the following figure, the application is consisted of
 instance1 and instance2, these two instances will be deployed into two
 OpenStack. Intance1 will provide service through "ext net1"(i.e, external
 network in OpenStack1), and Instance2 will provide service through
 "ext net2". Instance1 and Instance2 will be plugged into same L2 network
 net3 for data replication( for example database replication ).

  +-+   +-+
  |OpenStack1   |   |OpenStack2   |
  | |   | |
  | ext net1|   | ext net2|
  |   +-+-+ |   |   +-+-+ |
  | |   |   | |   |
  | |   |   | |   |
  |  +--+--+|   |  +--+--+|
  |  | ||   |  | ||
  |  | R1  ||   |  | R2  ||
  |  | ||   |  | ||
  |  +--+--+|   |  +--+--+|
  | |   |   | |   |
  | |   |   | |   |
  | +---+-+-+   |   | +---+-+-+   |
  | net1  | |   | net2  | |
  |   | |   |   | |
  |  ++--+  |   |  ++--+  |
  |  | Instance1 |  |   |  | Instance2 |  |
  |  +---+  |   |  +---+  |
  | |   |   | |   |
  | |   | net3  | |   |
  |  +--+-++  |
  | |   | |
  +-+   +-+


Are Openstack1 and 2 simply different compute nodes?

[joehuang] No, OpenStack 1 and 2 are two OpenStack clouds, each OpenStack cloud
 includes its own services like Nova, Cinder, Neutron. That 
means
 R1, net1 are network objects in OpenStack cloud1 ( Neutron1)
 R2, net2 are network objects in OpenStack cloud2 ( Neutron2),
 but net3 will be a shared L2 network existing in both 
OpenStack cloud1 and
 OpenStack cloud2, i.e in Neutron 1 and Neutron 2.


So net3 is a shared private network, perhaps just a shared VLAN.  And something 
makes sure IP address allocations don't overlap.  Ok.



 When we deploy the application in such a way, no matter which part of the
 application stops providing service, the other part can still provide
 service, and take the workload from the failure one. It'll bring the
failure
 tolerance no matter the failure is due to OpenStack crush or upgrade, or
 part of the application crush or upgrade.

 This mode can work very well and helpful, and router R1 R2 can run in DVR
 or legacy mode.

 While during the discussion and review of the spec:
 https://review.openstack.org/#/c/396564/, in this deployment, the end user
 has to add two NICs for each instance, one for the net3(a L2 network across
 OpenStack). And the net3 (a L2 network across OpenStack) can not be allowed
 to add_router_interface to router R1 R2, this is not good in networking.

 If the end user wants to do so, there is DVR MAC issues if more than one L2
 network across OpenStack are performed add_router_interface to router
R1 R2.

 Let's look at the following deployment scenario:
 +-+   +---+
 |OpenStack1   |   |OpenStack2 |
 | |   |   |
 | ext net1|   | ext net2  |
 |   +-+-+ |   |   +-+-+   |
 | |   |   | | |
 | |   |   | | |
 | +---+--+|   |  +--+---+ |
 | |  ||   |  |  | |
 | |R1||   |  |   R2 | |
 | |  ||   |  |  | |
 | ++--+--+|   |  +--+-+-+ |
 |  |  |   |   | | |   |
 |  |  |   | net3  | | |   |
 |  |   -+-+---+-+--+  |   |
 |  || |   |   |   |   |
 |  | +--+---+ |   | +-+-+ |   |
 |  | | Instance1| |

[openstack-dev] [Neutron][DVR] - No IRC Meeting this week

[Brian Haley]

Hi Folks,

We will not have a DVR sub-team meeting this week since neither Swami nor myself 
will be there to chair it.


We will resume our meetings next week on December 14th.

If you have any questions please ping us on IRC or send an email to the list.

https://wiki.openstack.org/wiki/Meetings/Neutron-DVR

Thanks,

-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][tricircle]DVR issue in cross Neutron networking

[Brian Haley]

On 12/5/16 3:03 AM, joehuang wrote:

Hello,


Hi Chaoyi,

Comments inline below.


 Tricircle plans to provide L2 network across Neutron to ease supporting
high
 availability of application:

 For example, in the following figure, the application is consisted of
 instance1 and instance2, these two instances will be deployed into two
 OpenStack. Intance1 will provide service through "ext net1"(i.e, external
 network in OpenStack1), and Instance2 will provide service through
 "ext net2". Instance1 and Instance2 will be plugged into same L2 network
 net3 for data replication( for example database replication ).

  +-+   +-+
  |OpenStack1   |   |OpenStack2   |
  | |   | |
  | ext net1|   | ext net2|
  |   +-+-+ |   |   +-+-+ |
  | |   |   | |   |
  | |   |   | |   |
  |  +--+--+|   |  +--+--+|
  |  | ||   |  | ||
  |  | R1  ||   |  | R2  ||
  |  | ||   |  | ||
  |  +--+--+|   |  +--+--+|
  | |   |   | |   |
  | |   |   | |   |
  | +---+-+-+   |   | +---+-+-+   |
  | net1  | |   | net2  | |
  |   | |   |   | |
  |  ++--+  |   |  ++--+  |
  |  | Instance1 |  |   |  | Instance2 |  |
  |  +---+  |   |  +---+  |
  | |   |   | |   |
  | |   | net3  | |   |
  |  +--+-++  |
  | |   | |
  +-+   +-+


Are Openstack1 and 2 simply different compute nodes?


 When we deploy the application in such a way, no matter which part of the
 application stops providing service, the other part can still provide
 service, and take the workload from the failure one. It'll bring the
failure
 tolerance no matter the failure is due to OpenStack crush or upgrade, or
 part of the application crush or upgrade.

 This mode can work very well and helpful, and router R1 R2 can run in DVR
 or legacy mode.

 While during the discussion and review of the spec:
 https://review.openstack.org/#/c/396564/, in this deployment, the end user
 has to add two NICs for each instance, one for the net3(a L2 network across
 OpenStack). And the net3 (a L2 network across OpenStack) can not be allowed
 to add_router_interface to router R1 R2, this is not good in networking.

 If the end user wants to do so, there is DVR MAC issues if more than one L2
 network across OpenStack are performed add_router_interface to router
R1 R2.

 Let's look at the following deployment scenario:
 +-+   +---+
 |OpenStack1   |   |OpenStack2 |
 | |   |   |
 | ext net1|   | ext net2  |
 |   +-+-+ |   |   +-+-+   |
 | |   |   | | |
 | |   |   | | |
 | +---+--+|   |  +--+---+ |
 | |  ||   |  |  | |
 | |R1||   |  |   R2 | |
 | |  ||   |  |  | |
 | ++--+--+|   |  +--+-+-+ |
 |  |  |   |   | | |   |
 |  |  |   | net3  | | |   |
 |  |   -+-+---+-+--+  |   |
 |  || |   |   |   |   |
 |  | +--+---+ |   | +-+-+ |   |
 |  | | Instance1| |   | | Instance2 | |   |
 |  | +--+ |   | +---+ |   |
 |  |  | net4  |   |   |
 | ++---+--+---+-+ |
 |  |  |   |   |   |
 |  +---+---+  |   |  ++---+   |
 |  | Instance3 |  |   |  | Instance4  |   |
 |  +---+  |   |  ++   |
 | |   |   |
 +-+   +---+

 net3 and net4 are two L2 network across OpenStacks. These two networks will
 be added router interface to R1 R2. Tricircle can help this, and addressed
 the DHCP and gateway challenges: different gateway port for the same
network
 in different OpenStack, so there is no problem for north-south traffic, the
 north-south traffic will goes to local external network directly, for
example,
 Instance1->R1->ext net1, instance2->R2->ext net2.


Can you describe the subnet configuration here?  Is there just one per 
network and was is the IP range?



 The issue is in east-west traffic if R1 R2 are running in DVR mode:
 when instance1 tries to ping instance4, DVR MAC replacement will happen
before
 the packet leaves the host where the instance1 is running, when the packet
 arrives at the host where 

Re: [openstack-dev] [neutron][octavia] Neutron LBaaS governance change and Octavia to the big tent

[Brian Haley]

On 12/01/2016 08:54 AM, Ihar Hrachyshka wrote:

Armando M.  wrote:


Hi folks,

A few hours ago a governance change [1] has been approved by TC members. This
means that from now on, the efforts for Load Balancing as a Service efforts
rest officially in the hands of the Octavia PTL and the Octavia core team.

I will work with the help of the respective core teams to implement a smooth
transition. My suggestion at this point is for any ML communication that
pertain LBaaS issues to include [octavia] tag on the email subject.

Please do not hesitate to reach out for any questions and/or clarifications.

Cheers,
Armando

[1] https://review.openstack.org/#/c/313056/


Should we also move all neutron lbaas-tagged bugs to octavia in LP? And kill the
‘lbaas’ tag from doc/source/policies/bugs.rst?


Yes.  I added the lbaas bugs.rst tag removal to
https://review.openstack.org/#/c/404872/ already.  Can someone from Octavia 
close and/or move-over any remaining lbaas bugs?  I only see three at

https://bugs.launchpad.net/ubuntu/+source/neutron-lbaas

Thanks,

-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Stepping down from core

[Brian Haley]

Hi Carl,

Thanks for all the hard work, Neutron is definitely better because of it.  Hope 
to work with you again some day.


Good luck with your new endeavor!

-Brian

On 11/17/2016 01:42 PM, Carl Baldwin wrote:

Neutron (and Openstack),

It is with regret that I report that my work situation has changed such that I'm
not able to keep up with my duties as a Neutron core reviewer, L3 lieutenant,
and drivers team member. My participation has dropped off considerably since
Newton was released and I think it is fair to step down and leave an opening for
others to fill. There is no shortage of talent in Neutron and Openstack and I
know I'm leaving it in good hands.

I will be more than happy to come back to full participation in Openstack and
Neutron in the future if things change again in that direction. This is a great
community and I've had a great time participating and learning with you all.

Well, I don't want to drag this out. I will still be around on IRC and will be
happy to help out where I am able. Feel free to ping me.

Carl


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] OVS agent reports error "ovs-ofctl: br-int is not a bridge or a socket"

[Brian Haley]

On 11/17/2016 03:14 AM, zhi wrote:

Hi, all

I build a devstack which code from master branch. By default the OVS version is
2.0.2 when install devstack. So I reinstall the OVS 2.6.0.

I meet an exception in OVS agent when all OVS services runs okay. Exception
shows below:

2016-11-17 15:47:29.175 ERROR neutron.agent.linux.utils
[req-2393c604-1938-4841-9ea9-8f691e6c2262 None None] Exit code: 1; Stdin:
hard_timeout=0,idle_timeout=0,priority=0,table=71,cookie=10671737546270118050,actions=drop;
Stdout: ; Stderr: ovs-ofctl: br-int is not a bridge or a socket


The error is saying br-int doesn't exist, and from your output below it doesn't. 
 Try running:


# ovs-vsctl add-br br-int

-Brian



I want to know why OVS agent tells me these info. All the OVS services run okay.
I use " ovs-vsctl show " to ensure OVS services and the output is right.

The detailed output when excute " ovs-vsctl show ":

root@devstack:~/openvswitch-2.6.0# ovs-vsctl show
6e695086-6663-4fd0-b713-5a12605c92cb
Manager "ptcp:6640:127.0.0.1"
is_connected: true
Bridge br-ex
Controller "tcp:127.0.0.1:6633 "
is_connected: true
fail_mode: secure
Port br-ex
Interface br-ex
type: internal
Port phy-br-ex
Interface phy-br-ex
type: patch
options: {peer=int-br-ex}
... ...

I think that OVS runs okay.

Did I miss some configuration or something about neutron or OVS?


Hope for your reply. :-)


Thanks
Zhi Chang


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][trunk-port] OVS tbr bridge wasn't be created by OVS agent

[Brian Haley]

On 11/15/16 5:12 AM, zhi wrote:

Sorry, I forgot to say my local environment is Liberty. :)


According to the blueprint and reviews this didn't land until Newton, 
maybe some in Mitaka, so I wouldn't expect it to work in Liberty.


-Brian



2016-11-15 18:07 GMT+08:00 zhi >:

Hi, all

I followed this guide[1] to create trunk ports and created a vm
by using trunk port. But I met a weird problem. OVS agent didn't
generate " tbr " bridge. All the OVS bridges shows below:
"
[root@server-64 ~]# ovs-vsctl list-br
br-int
br-physnet4
br-tun
"
Why did the OVS agent doesn't create " tbr " bridge ? I think I must
miss something but I don't know.

I enabled " trunk " in service_plugins configuration in neutron
server. And I did not add anything in OVS agent. Did I miss any
configuration in OVS agent ?


Thanks
Zhi Chang

[1]: https://wiki.openstack.org/wiki/Neutron/TrunkPort#CLI_usage_example





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] HostNotCompatibleWithFixedIps exception happens when setting router's gateway.

[Brian Haley]

Hi Zhi,

Thanks for the report, comment below.

On 10/27/2016 05:04 AM, zhi wrote:

Hi, all.

I installed a devstack in my local environment. All the code from master
branch. After the installation, I have to show you some problems which I met.

First of all, I create an external network by this command " neutron
net-create public --router:external=True --provider:network_type=flat
--provider:physical_network=public ".

Secondly, I create a subnet with " subnet_type " by this command " neutron
subnet-create [net-id] 20.20.20.0/24  --service-types
list=true network:router_gateway ".

At last, I create a router and setting this router's gateway by this command
" neutron router-gateway-set [router-id] [net-id]".

Exception happens in Neutron Server, it says "
HostNotCompatibleWithFixedIps: Host devstack is not connected to a segment where
the existing fixed_ips on port 0f38ba01-8dd0-43de-92e3-b294bd4ebed8 will
function given the routed network topology. ".


Subnet service types is new in Newton, and it seems you've found a bug - can you 
file a bug on launchpad for it?


The one thing you might try to get past this is to disable DHCP on these 
subnets, but the error you linked seems different from [1].


-Brian

[1] https://bugs.launchpad.net/neutron/+bug/1636963


After I did some research about the exception,  I found this patch[1] was
adding this exception into neutron repo. I am confused about that. Why setting
router's gateway will trigger this exception? I don't execute any commands about
" routed_network ".

What's wrong ?

Could someone give some advice about that ? I upload all the network and
subnets info at here [2]. Detail exception at here [3].

BTW, what's the meaning of " tags " in network?

Hope for your reply. :)


Thanks
Zhi Chang


[1]. https://review.openstack.org/#/c/346217/3
[2]. http://paste.openstack.org/show/587157/
[3]. http://paste.openstack.org/show/587158/


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] running non-devstack jobs in Python projects aka "it works outside devstack"

[Brian Haley]

On 10/14/2016 05:53 PM, Clark Boylan wrote:

On Thu, Oct 13, 2016, at 05:47 PM, Emilien Macchi wrote:

Greetings OpenStack,

== Background

Since the beginning of OpenStack (or almost), devstack has been used
as a common tool to deploy OpenStack in CI environment. Most of
OpenStack projects (if not all) that are written in Python use it to
deploy the different components.
While devstack became popular and the reference in term of deployment
tool for continuous integration, devstack doesn't deploy OpenStack in
production (versus some tools like Kolla, Fuel, TripleO, Juju, etc).
It means things might (and did) break when deploying OpenStack outside
devstack, for different reasons. Some examples:

* until recently, SSL was not tested, and I believe some projects
still don't test with SSL enabled.
* IPv6 is not tested everywhere.




IPv6 testing likely means two different things to two different groups
of people. First is whether or not the cloud endpoints are hosted over
IPv6 and the other is whether or not instances launched by openstack are
assigned IPv6 addresses. The second thing has been tested for quite a
while now (tempest has had tests for it for almost 2 years though it
hasn't been enabled in the gate for that long). We should definitely
ensure that we are testing with openstack servers listening on IPv6
addresses as well.


The first item - IPv6 service endpoints, is actually covered by an experimental 
job (tempest-dsvm-neutron-serviceipv6), and devstack supports it in local.conf:


SERVICE_IP_VERSION=6

Last year it was working great, bug I've noticed there are failures now, I'll 
take a crack at getting it all working again.  Maybe it's something we could 
then promote to voting?


-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] floating IP is DOWN

[Brian Haley]

On 09/22/2016 10:19 AM, Barber, Ofer wrote:

when i assign a floating IP to a server, i see that the status of the floating
IP is "down"

why is that so ?

*_code:_*

LOG.info("\n<== float IP address: %s and status: %s  ==>" %
(float_ip['floating_ip_address'],float_ip['status']))

*_Output:_*

<== float IP address: 10.63.101.225 and status: DOWN  ==>


I couldn't find that code anywhere, what release was this on?

From a Newton-based system created yesterday, this is the message in the 
l3-agent log when I associate a floating IP:


Floating ip 4c1b4571-a003-43f2-96a1-f7073cd1319d added, status ACTIVE

-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Adding ihrachys to the neutron-drivers team

[Brian Haley]

Congrats Ihar!

-Brian

On 09/17/2016 12:40 PM, Armando M. wrote:

Hi folks,

I would like to propose Ihar to become a member of the Neutron drivers team [1].

Ihar wide knowledge of the Neutron codebase, and his longstanding duties as
stable core, downstream package whisperer, release and oslo liaison (I am sure I
am forgetting some other capacity he is in) is going to put him at great comfort
in the newly appointed role, and help him grow and become wise even further.

Even though we have not been meeting regularly lately we will resume our
Thursday meetings soon [2], and having Ihar onboard by then will be highly
beneficial.

Please, join me in welcome Ihar to the team.

Cheers,
Armando

[1] 
http://docs.openstack.org/developer/neutron/policies/neutron-teams.html#drivers-team

[2] https://wiki.openstack.org/wiki/Meetings/NeutronDrivers



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] Help with ipv6 route configuration and problem to traverse virtual router.

[Brian Haley]

On 08/31/2016 07:00 AM, Jorge Luiz Correa wrote:


*Chain neutron-l3-agent-scope (1 references)*
 pkts bytes target prot opt in out source
destination
   78  4368 *DROP*   all  *  qr-1ee33f03-23  ::/0
::/0 mark match ! 0x400/0x

Packets pass in chain FORWARD -> neutron-filter-top ->
neutron-l3-agent-local ->
back to FORWARD -> neutron-l3-agent-FORWARD -> neutron-l3-agent-scope ->
DROP.


This looks similar to https://bugs.launchpad.net/neutron/+bug/1570122



Thank you Brian, this is the problem.

IPv4 rules is very similar but works. Ipv6 is blocking for some reason.

Do you have the same mark/match rules with IPv4, they're just not getting 
hit?

Yes, IPv4 have this rule and works fine. Adding a similar rule manually with
ip6tables the traffic traverses the virtual router.


So is the ip6tables rule just wrong?  Feel free to add any info to the bug that 
might help fix this.


Thanks,

-Brian


___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] Help: Openstack DVR floating ip No Port available

[Brian Haley]

On 08/30/2016 03:08 PM, Satish Patel wrote:

Interesting, when i create V-Router then it suppose to appear on
Compute node right but it doesn't. What could be wrong, Does mikata
support DVR or it's something i am doing wrong

[root@controller ~(keystone_admin)]# ip netns
snat-99960b5a-b475-436a-a284-7239546fb91b
qrouter-99960b5a-b475-436a-a284-7239546fb91b

[root@compute-1]# ip netns


The router will only get created when a VM in that network is placed on the 
compute node.  Does a neutron router-show of the router have distributed=True? 
You'll need to have admin privs to view it.


-Brian

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] Help with ipv6 route configuration and problem to traverse virtual router.

[Brian Haley]

On 08/30/2016 02:53 PM, Jorge Luiz Correa wrote:

Thank you Tomas and Brian!

Here they are (just replace my ipv6 prefix with 2001:DB8). But, I think the
problem is with firewall rules (see bellow).

 


root@dataexp-network:/# ip netns exec
qrouter-eb42f197-8969-4744-b226-49653ed2bf48 ip -6 route show
*2001:DB8:1400:c539::/64 dev qr-1ee33f03-23*  proto kernel  metric 256  pref 
medium
fe80::/64 dev qg-69fbbe1a-ee  proto kernel  metric 256  pref medium
fe80::/64 dev qr-9f742219-78  proto kernel  metric 256  pref medium
fe80::/64 dev qr-1ee33f03-23  proto kernel  metric 256  pref medium
*default via fe80::215:17ff:fea0:211d* dev qg-69fbbe1a-ee  metric 1024  pref 
medium

fe80::215:17ff:fea0:211d is my firewall/router and this route was learned via 
RA.

At this moment my firewall/router has one route to 2001:DB8:1400::1/52 via
fe80::f816:3eff:fed5:c5f8 (the path is firewall/router -> br-ex -> br-int ->
qg-69fbbe1a-ee). The packets go up to qg-69fbbe1a-ee.

I think these setting are ok!


Yes, those look good.


Now, I found something with iptables. See the rules in qrouter namespace:




*Chain neutron-l3-agent-scope (1 references)*
 pkts bytes target prot opt in out source
destination
   78  4368 *DROP*   all  *  qr-1ee33f03-23  ::/0
::/0 mark match ! 0x400/0x

Packets pass in chain FORWARD -> neutron-filter-top -> neutron-l3-agent-local ->
back to FORWARD -> neutron-l3-agent-FORWARD -> neutron-l3-agent-scope -> DROP.


This looks similar to https://bugs.launchpad.net/neutron/+bug/1570122


IPv4 rules is very similar but works. Ipv6 is blocking for some reason.


Do you have the same mark/match rules with IPv4, they're just not getting hit?

-Brian


___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] Help with ipv6 route configuration and problem to traverse virtual router.

[Brian Haley]

On 08/30/2016 09:54 AM, Jorge Luiz Correa wrote:

Hi! I need some help to understand and configure my network node to provide
network access using a dual stack configuration. I've a scenario with one
controller, one network node and a lot of compute nodes. The version is Mitaka
on Ubuntu 16.04 LTS, Kernel 4.4.0-36.


Hi,

Thanks for giving so much information, comments below.


The IPv4 is working fine. Instances can get IPv4 inside tenant networks, I can
configure floating IPs, access external hosts etc.

The IPv6 has some features working, but I still didn't got the traffic pass
between  internal and the external networks.

I'm using prefix delegation with dibbler as described here:

http://docs.openstack.org/mitaka/networking-guide/config-ipv6.html

I can create IPv6 tenant subnets, they can get a prefix from dibbler and
instances on this subnets can configure IPv6 normally.

I've a default security group with rules passing any IPv4 and IPv6 traffic and
any ICMP.

The problem is that the packages from and to instances don't pass through
virtual router. The virtual router has one external interface named qg-
(connected to br-int -> br-ex) and one internal interface named qr- connected to
tenant network (br-int -> int-br-vlan). When testing connectivity I can see
packages (with tcpdump) on my external router/firewall and on qg- interface. For
example, when I try to ping my external router/firewall from an instance, echo
requests pass to the external network (through the virtual router) but echo
reply die on virtual router (last seen on qg-).

## echo request:

Instance A
|
|
v
br-int
|
|
v
qr- interface
VIRTUAL ROUTER
qg- interface
|
|
v
br-int
|
|
v
br-ex
|
|
v
Router/Firewall (I can see here with tcpdump)


## echo reply:

Instance A
x
x
x
qr- interface (I CAN'T SEE HERE, LOST)
VIRTUAL ROUTER
qg- interface (I can see here with tcpdump)
^
|
|
br-int (ovs bridge, can't do tcpdump, but ok)
^
|
|
br-ex (I can see here with tcpdump)
^
|
|
Router/Firewall

Question 1) Where can I start to debug this problem?

I'm thinking that can be something with ipv6 packet forwarding (configurable
with sysctl). Using 'ip6tables -v' I can't see droppings.

Chain neutron-openvswi-sg-fallback (0 references)
 pkts bytes target prot opt in out source
destination
0 0 DROP   all  *  *   ::/0
::/0 /* Default drop rule for unmatched traffic. */


Can you verify there is a default IPv6 route in the qrouter namespace? 
Something like 'ip -6 r' should show it.  In general, seeing what is configured 
in that namespace and seeing if you can ping things from there is a good start.



Another thing I would like to understand is about how I should configure my
router/firewall to send IPv6 packets to Openstack network node. For example, if
I have the network 2001:DB8::/52 to use on Openstack. Each project will get a
2001:DB8::/64 range from prefix delegation. When one project get its prefix, the
virtual router knows how to send traffic to external world because my
router/firewall sends RA. But, on my router/firewall I need to configure a route
to 2001:DB8::/52. To do this, I need to inform one next-hop. I'm using de LLA
(fe80::...) of br-ex as next-hop. So, all traffic destinated to any network
inside 2001:DB8::/52 will be send to br-ex (that is on network node). This
configuration seems to work because packets arrive on virtual router as
described above.

Question 2) Is this the right way?


That external router is giving you the prefix via PD, right?  I would have 
thought it would have added a route for your /64 when it did that.


-Brian

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] [infra][neutron] IPv6 gate issue in OSIC cloud

[Brian Haley]

On 08/24/2016 04:52 PM, Brian Haley wrote:

Hi,

Starting sometime earlier in the week, gate jobs started failing that were
running in the OSIC cloud, due to a loss of connectivity to VMs running there
when Neutron L3 was configured.  I wanted to send some information out on what
we found in case other deployment tools trip over the same issue.


And I didn't mean to leave out all the people that helped track this down and 
expedite the patches - clarkb, dougwig, sc68cal, mordred, armax, sdague... many 
thanks everyone!


-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [infra][neutron] IPv6 gate issue in OSIC cloud

[Brian Haley]

Hi,

Starting sometime earlier in the week, gate jobs started failing that were 
running in the OSIC cloud, due to a loss of connectivity to VMs running there 
when Neutron L3 was configured.  I wanted to send some information out on what 
we found in case other deployment tools trip over the same issue.


The problem was the VMs in OSIC are IPv6-only by default, so must be reachable 
using a global IPv6 address.  At boot, everything was fine - IPv6 address and 
default route were configured, but when IPv6 forwarding was enabled, poof!  The 
problem is that the default behavior in Linux is to remove any IPv6 default 
routes when forwarding is enabled, and that action caused connectivity loss to 
systems outside the OSIC datacenter using IPv6.  Luckily there's a sysctl to 
control this, and there is a patch out [1] that is working it's way through the 
check queue now.


So if you're creating puppet, chef, etc, scripts and deploying in an IPv6-only 
environment, you might need a few tweaks to not hit the same issue.


-Brian

[1] https://review.openstack.org/#/c/359490/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] [neutron][devstack]subnet created with wrong gateway

[Brian Haley]

On 08/22/2016 04:30 PM, Brent Troge wrote:

Thanks Brian for your reply and your contributions.


No problem.

Devstack patch is at https://review.openstack.org/#/c/358848/

-Brian



On Aug 22, 2016 2:09 PM, "Brian Haley" <brian.ha...@hpe.com
<mailto:brian.ha...@hpe.com>> wrote:

Yes, it uses the lowest for the gateway, I was using ".1" to denote this but
realize it might not have been the best example.  I'm testing a patch now to
remove the '--gateway a.b.c.d' if it's not explicitly set since it isn't
necessary.

-Brian

On 08/22/2016 01:57 PM, Brent Troge wrote:

Doesn't neutron use the first available host address in the subnet as 
the
gateway? So if your cidr is 128/25 then neutron would default to 129 as 
the
gateway.


    On Aug 22, 2016 9:54 AM, "Brian Haley" <brian.ha...@hpe.com
<mailto:brian.ha...@hpe.com>
<mailto:brian.ha...@hpe.com <mailto:brian.ha...@hpe.com>>> wrote:

On 08/21/2016 08:40 PM, Tony Breeds wrote:

On Sun, Aug 21, 2016 at 05:28:44PM +0530, Akilesh K wrote:

Hi,
I am using devstack for the first time and I see that it has
created a
subnet like below. The cidr and gateway do not match.
Because of this
devstack fails to run completely and fails while attaching a
router
to this
subnet. Why is devstack doing this??


You also need to specify NETWORK_GATEWAY in your local.conf:
NETWORK_GATEWAY=10.4.128.1
should match:
FIXED_RANGE=10.4.128.0/20 <http://10.4.128.0/20>
<http://10.4.128.0/20>

Our docs need updating as we recently switched from nova to
neutron as the
default network provider.


What devstack should really be doing is defaulting the gateway to 
"",
neutron will automatically use the ".1" of the subnet for the
gateway if not
told otherwise.

I'll try and get a quick patch out today.

-Brian

___
Mailing list:
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack>
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack>>
Post to : openstack@lists.openstack.org
<mailto:openstack@lists.openstack.org>
<mailto:openstack@lists.openstack.org
<mailto:openstack@lists.openstack.org>>
Unsubscribe :
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack>
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack>>





___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] [neutron][devstack]subnet created with wrong gateway

[Brian Haley]

Yes, it uses the lowest for the gateway, I was using ".1" to denote this but 
realize it might not have been the best example.  I'm testing a patch now to 
remove the '--gateway a.b.c.d' if it's not explicitly set since it isn't necessary.


-Brian

On 08/22/2016 01:57 PM, Brent Troge wrote:

Doesn't neutron use the first available host address in the subnet as the
gateway? So if your cidr is 128/25 then neutron would default to 129 as the
gateway.


On Aug 22, 2016 9:54 AM, "Brian Haley" <brian.ha...@hpe.com
<mailto:brian.ha...@hpe.com>> wrote:

On 08/21/2016 08:40 PM, Tony Breeds wrote:

On Sun, Aug 21, 2016 at 05:28:44PM +0530, Akilesh K wrote:

Hi,
I am using devstack for the first time and I see that it has 
created a
subnet like below. The cidr and gateway do not match. Because of 
this
devstack fails to run completely and fails while attaching a router
to this
subnet. Why is devstack doing this??


You also need to specify NETWORK_GATEWAY in your local.conf:
NETWORK_GATEWAY=10.4.128.1
should match:
FIXED_RANGE=10.4.128.0/20 <http://10.4.128.0/20>

Our docs need updating as we recently switched from nova to neutron as 
the
default network provider.


What devstack should really be doing is defaulting the gateway to "",
neutron will automatically use the ".1" of the subnet for the gateway if not
told otherwise.

I'll try and get a quick patch out today.

-Brian

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack>
Post to : openstack@lists.openstack.org
<mailto:openstack@lists.openstack.org>
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack>




___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] [neutron][devstack]subnet created with wrong gateway

[Brian Haley]

On 08/21/2016 08:40 PM, Tony Breeds wrote:

On Sun, Aug 21, 2016 at 05:28:44PM +0530, Akilesh K wrote:

Hi,
I am using devstack for the first time and I see that it has created a
subnet like below. The cidr and gateway do not match. Because of this
devstack fails to run completely and fails while attaching a router to this
subnet. Why is devstack doing this??


You also need to specify NETWORK_GATEWAY in your local.conf:
NETWORK_GATEWAY=10.4.128.1
should match:
FIXED_RANGE=10.4.128.0/20

Our docs need updating as we recently switched from nova to neutron as the
default network provider.


What devstack should really be doing is defaulting the gateway to "", neutron 
will automatically use the ".1" of the subnet for the gateway if not told otherwise.


I'll try and get a quick patch out today.

-Brian

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] [OpenStack] Second Public IP for VM on Another Public Network does not work properly

[Brian Haley]

On 08/17/2016 07:40 PM, Neil Jerram wrote:

On Wed, Aug 17, 2016 at 7:35 PM Brian Haley <brian.ha...@hpe.com
<mailto:brian.ha...@hpe.com>> wrote:


Is a cloud datacenter going to have multiple link types?  I doubt it.  And
having two VMs would be easier anyways as there are probably other things
you'reha
going to tune.


Yes, I think so.  I'm currently working on an OpenStack customer deployment
where they want each instance to have an additional NIC for 'management' access
into and out from that instance; and there is a correspondingly separate
provider network for 'management' purposes between the compute hosts.  In
general, when an instance has multiple NICs, it needs somehow to arrange that it
sets up only one default route, through one of those NICs, and that non-default
routes are used to direct traffic to specific destinations through the other 
NICs.


So people are re-creating their legacy setups, right down to the "iLO management 
network", sigh :)  They do know there is a noVNC answer to get a console:


http://docs.openstack.org/admin-guide/compute-remote-console-access.html


(FWIW, I also submitted a proposal to talk about this in Barcelona: "Routed
(Calico) networking with ECMP, dual fabric planes and data/management 
separation".)


I'll definitely come and see it if it's accepted Neil.  I guess speed 
designations will be the next thing added to the Network model...


-Brian

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] [OpenStack] Second Public IP for VM on Another Public Network does not work properly

[Brian Haley]

On 08/17/2016 01:54 PM, Piotr Baranowski wrote:

- Oryginalna wiadomość -

Od: "Brian Haley" <brian.ha...@hpe.com>
Do: openstack@lists.openstack.org
Wysłane: środa, 17 sierpnia, 2016 18:53:47
Temat: Re: [Openstack] [OpenStack] Second Public IP for VM on Another Public 
Network does not work properly



On 08/17/2016 05:08 AM, Ludwig Tirazona wrote:

Hello,


Has anybody had this experience/problem as well?


I'm not sure I'd expect this to work, here is just one reason.


Brian,

Ever heard of policy based routing?

>

You can have multiple outbound links and use them when your traffic meets some 
particular criteria.


Yes, and that was what I was alluding to in order to make packets go out the 
correct interface based on the source IP in the packets.



One simple example:

You have an expensive low latency link and a high latency cheap one.
You may use policy routing to automatically select which route to use.


Is a cloud datacenter going to have multiple link types?  I doubt it.  And 
having two VMs would be easier anyways as there are probably other things you're 
going to tune.


> TL;DR It makes sense to have multiple uplinks.

A cloud provider is probably going to have multiple links to the Internet, and 
they'll be able to deal with one being down without any action on the part of a 
tenant.


I can only guess the OP wants to do this because their existing architecture is 
doing something like it, but I don't know.


-Brian

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] [OpenStack] Second Public IP for VM on Another Public Network does not work properly

[Brian Haley]

On 08/17/2016 05:08 AM, Ludwig Tirazona wrote:

Hello,


Has anybody had this experience/problem as well?


I'm not sure I'd expect this to work, here is just one reason.

The VM really doesn't know when to use which router, because it doesn't know 
which target IP was used pre-NAT.  For example, given some Internet IP source 
address, say 8.8.8.8, where will the VM send a response?  It will use the 
default route going through Router1 in most cases.  The way you've had to go 
setup a static route on the VM to get to PubNet2 will only affect packets going 
to that subnet, but being a Public IP means it will be communicating with other 
systems not on that subnet.  You would have to create route entries based on the 
source IP being used, but there still might be edge cases that cause problems.


I think a better question to ask is, why do you need two Public IPs?  Don't make 
things more complicated than they need to be.


BTW, the best way to figure out why this isn't working is looking at tcpdump 
traces on all the interfaces and bridges, and possibly even flow rules if you're 
using OVS, as well as iptables rules for security groups.  That will at least 
tell you where the packet is getting dropped.


-Brian


---
OVERVIEW

I have two separate public networks, each with their entirely separate
IP block.

I need a VM to have Floating IPs on both of these networks.

I am on OpenStack Liberty.
--

ACTIONS

I create two routers and two private subnets in my Project, one for each
public network.

I create an instance attached to subnet1, and give it a floating IP on
PubNet1.

Everything is working fine.

I attach a second interface for subnet2 to the VM. I give it the static
address that Neutron-DHCP would have given it, were it using DHCP.

Everything is working fine.

From the "Access & Security" > "Floating IPs" interface on Horizon, I
assign a Floating IP from PubNet2 to the VM's interface on subnet2.


--

PROBLEM

Here's where things get wonky:

Although the Floating IP assignment request completes successfully,
connections to the VM on the PubNet2 floating IP do not reach the VM.
---


DETAILS

I have a wide-open Security Group for the VM, allowing everything in and
out.

On the VM, I have configured a static route to PubNet2 through the
subnet2 gateway.

From the VM, I can ping my PubNet2 router's PubNet2 IP, and the PubNet2
gateway as well. I can't ping the VM's PubNet2 Floating IP.


I see the VM's 2nd Floating IP on the qrouter's network namespace on my
Network Node.

I do the following:

ip netns exec qrouter- ping 

that pings successfully.

ip netns exec qrouter- ping 

that fails to ping. Even through I see it's on the same network
namespace interface ast the PubNet2 Router Public IP.


---

I hope I was able to describe the problem accurately, but concisely as well.

Does anybody have an idea as to what the problem might be?

Is what I'm even attempting supposedly possible with Neutron-Liberty?

What can I try?




Thanks in advance!

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack




___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] [all] devstack changing to neutron by default RSN

[Brian Haley]

On 08/05/2016 02:32 PM, Armando M. wrote:


>
> Looking at the health trend for DVR [1], the test hasn't failed in a
> while, so I wonder if this is induced by the proposed switch, even
> though I can't correlate it just yet (still waiting for caffeine to 
kick
> in). Perhaps we can give ourselves today to look into it and pull the
> trigger for 351450 > on Monday?
>
> [1]

http://status.openstack.org/openstack-health/#/job/gate-tempest-dsvm-neutron-dvr



The only functional difference in the new code that happens in the gate
is the iptables rule:

local default_dev=""
default_dev=$(ip route | grep ^default | awk '{print $5}')
sudo iptables -t nat -A POSTROUTING -o $default_dev -s
$FLOATING_RANGE -j MASQUERADE


I skipped this in [0], to give us further data pointsclasping at straws 
still.

[0] https://review.openstack.org/#/c/351876/


I haven't been able to reproduce it either, but it's unclear how packets would 
get into a VM on an island since there is no router interface, and the VM can't 
respond even if it did get it.


I do see outbound pings from the connected VM get to eth0, hit the masquerade 
rule, and continue on their way.  But those packets get dropped at my ISP since 
they're in the 10/8 range, so perhaps something in the datacenter where this is 
running is responding?  Grasping at straws is right until we see the results of 
Armando's test patch.


-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] devstack changing to neutron by default RSN

[Brian Haley]

On 08/05/2016 08:59 AM, Sean Dague wrote:

On 08/04/2016 09:15 PM, Armando M. wrote:

So glad we are finally within the grasp of this!

I posted [1], just to err on the side of caution and get the opportunity
to see how other gate jobs for Neutron might be affected by this change.

Are there any devstack-gate changes lined up too that we should be aware of?

Cheers,
Armando

[1] https://review.openstack.org/#/c/351450/


Nothing at this point. devstack-gate bypasses the service defaults in
devstack, so it doesn't impact that at all. Over time we'll want to make
neutron the default choice for all devstack-gate setups, and nova-net to
be the exception. But that actually can all be fully orthoginal to this
change.

The experimental results don't quite look in yet, it looks like one test
is failing on dvr (which is the one that tests for cross tenant
connectivity) -
http://logs.openstack.org/50/350750/5/experimental/gate-tempest-dsvm-neutron-dvr/4958140/

That test has been pretty twitchy during this patch series, and it's
quite complex, so figuring out exactly why it's impacted here is a bit
beyond me atm. I think we need to decide if that is going to get deeper
inspection, we live with the fails, or we disable the test for now so we
can move forward and get this out to everyone.


I took a quick look at this and can't reproduce it yet, here's what the test 
seems to do:


1a. Create a network/subnet (10.100.0.0/28)
 b. attach a router interface to the subnet
 c. boot VM1 on the network

2a. Create a network/subnet (10.100.0.16/28)
 b. do NOT attach a router interface to the subnet
 c. boot VM2 on the network

3. Ssh to VM1 and ping VM2 - it should fail since there's no route to the 
network, but it succeeds


The only place you should be able to ping that VM2 IP from is the dhcp 
namespace, which does work for me.


So if you are seeing it be flaky it could the VM placement (same host vs 
different host) is impacting it?  In the logs it showed the same hostId, but so 
did my test, so I don't have a good answer.


-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] stable/liberty busted

[Brian Haley]

On 08/04/2016 03:25 PM, Brian Haley wrote:

On 08/04/2016 03:16 PM, Rick Jones wrote:

On 08/04/2016 12:04 PM, Kevin Benton wrote:

Yeah, I wasn't thinking when I +2'ed that. There are two use cases for
the pinger, one for ensuring continuous connectivity and one for
eventual connectivity.

I think the revert is okay for a quick fix, but we really need a new
argument to the pinger for strictness to decide which behavior the test
is looking for.


What situation requires continuous connectivity?


Maybe the test names can answer that:

test_assert_pings_during_br_int_setup_not_lost()
_test_assert_pings_during_br_phys_setup_not_lost()

In other cases we want the previous behavior - is that IP alive?  It's probably
just best to put the old code back and make a new assert_async_ping() based on
this code.


https://review.openstack.org/351356

^^ that makes a new assert_async_ping() and restores assert_ping() to previous 
behavior.


It will need a few rechecks to verify it helps, although this error would be 
hard to trigger.


-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] stable/liberty busted

[Brian Haley]

On 08/04/2016 03:16 PM, Rick Jones wrote:

On 08/04/2016 12:04 PM, Kevin Benton wrote:

Yeah, I wasn't thinking when I +2'ed that. There are two use cases for
the pinger, one for ensuring continuous connectivity and one for
eventual connectivity.

I think the revert is okay for a quick fix, but we really need a new
argument to the pinger for strictness to decide which behavior the test
is looking for.


What situation requires continuous connectivity?


Maybe the test names can answer that:

test_assert_pings_during_br_int_setup_not_lost()
_test_assert_pings_during_br_phys_setup_not_lost()

In other cases we want the previous behavior - is that IP alive?  It's probably 
just best to put the old code back and make a new assert_async_ping() based on 
this code.


-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] stable/liberty busted

[Brian Haley]

On 08/04/2016 01:36 PM, Armando M. wrote:

Hi Neutrinos,

I have noticed that Liberty seems to be belly up [1]. I wonder if anyone knows
anything or has the time to look into it.

Many thanks,
Armando

[1] https://review.openstack.org/#/c/349039/


This could be due to this backport;

https://review.openstack.org/#/c/347062/

Before we were doing 'ping -c 3 -W 1 $IP', which will succeed as long as one 
packet is returned.


Now there is an outer loop that runs 'ping -c 1 -W 1 $IP', so a single dropped 
packet could cause an error.  Since sometimes that first packet causes ARP to 
happen, any delay near the 1-second mark looks like a lost packet, but is really 
just transient and packets 2 and 3 are fine.


I've started a revert and will recheck, but if I'm right an async issue like 
this is hard to reliably reproduce - I had to use iptables directly to test my 
theory about the return code from ping when 1/3 packets were lost.


-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][dvr][fip] fg device allocated private ip address

[Brian Haley]

On 08/02/2016 08:15 AM, huangdenghui wrote:

hi john and brain
   thanks for your information, if we get patch[1],patch[2] merged,then fg can
allocate private ip address. after that, we need consider floating ip dataplane,
in current dvr implementation, fg is used to reachment testing for floating ip,
now,with subnet types bp,fg has different subnet than floating ip address, from
fg'subnet gateway point view, to reach floating ip, it need a routes entry,
destination is some floating ip address, fg'ip address is the nexthop, and this
routes entry need be populated at the event of floating ip creating, deleting
when floating ip is dissociated. any comments?


Yes, there could be a small change necessary to the l3-agent in order to route 
packets with DVR enabled, but I don't see it being a blocker.


-Brian



On 2016-08-01 23:38 , John Davidge <mailto:john.davi...@rackspace.com> Wrote:

Yes, as Brian says this will be covered by the follow-up patch to [2]
which I¹m currently working on. Thanks for the question.

John


On 8/1/16, 3:17 PM, "Brian Haley" <brian.ha...@hpe.com> wrote:

>On 07/31/2016 06:27 AM, huangdenghui wrote:
>> Hi
>>Now we have spec named subnet service types, which provides a
>>capability of
>> allowing different port of a network to allocate ip address from
>>different
>> subnet. In current implementation of DVR, fip also is distributed on
>>every
>> compute node, floating ip and fg's ip are both allocated from external
>>network's
>> subnets. In large public cloud deployment, current implementation will
>>consume
>> lots of public ip address. Do we need a RFE to apply subnet service
>>types spec
>> to resolve this problem. Any thoughts?
>
>Hi,
>
>This is going to be covered in the existing RFE for subnet service types
>[1].
>We currently have two reviews in progress for CRUD [2] and CLI [3], the
>IPAM
>changes are next.
>
>-Brian
>
>[1] https://review.openstack.org/#/c/300207/
>[2] https://review.openstack.org/#/c/337851/
>[3] https://review.openstack.org/#/c/342976/
>
>__
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



Rackspace Limited is a company registered in England & Wales (company
registered number 03897010) whose registered office is at 5 Millington Road,
Hyde Park Hayes, Middlesex UB3 4AZ. Rackspace Limited privacy policy can be
viewed at www.rackspace.co.uk/legal/privacy-policy - This e-mail message may
contain confidential or privileged information intended for the recipient.
Any dissemination, distribution or copying of the enclosed material is
prohibited. If you receive this transmission in error, please notify us
immediately by e-mail at ab...@rackspace.com and delete the original
message. Your cooperation is appreciated.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Belated nova newton midcycle recap (part 2)

[Brian Haley]

On 08/01/2016 10:15 PM, Matt Riedemann wrote:

Starting from where I accidentally left off:





We also talked a bit about live migration with Neutron. There has been a fix up
for live migration + DVR since Mitaka:

https://review.openstack.org/#/c/275073

It's a bit of a hacky workaround but the longer term solution that we all want (
https://review.openstack.org/#/c/309416 ) is not going to be in Newton and will
need discussion at the Ocata summit in Barcelona (John Garbutt was going to work
with the Neutron team on preparing for the summit on that one). So we agreed to
go with Swami's DVR fix but it needs to be rebased (which still hasn't happened
since the midcycle).


I just pushed an update to the DVR live-migration patch re-based to master, so 
feel free to review again.  Swami or myself will answer any other comments as 
quickly as possible.


Thanks,

-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][dvr][fip] fg device allocated private ip address

[Brian Haley]

On 07/31/2016 06:27 AM, huangdenghui wrote:

Hi
   Now we have spec named subnet service types, which provides a capability of
allowing different port of a network to allocate ip address from different
subnet. In current implementation of DVR, fip also is distributed on every
compute node, floating ip and fg's ip are both allocated from external network's
subnets. In large public cloud deployment, current implementation will consume
lots of public ip address. Do we need a RFE to apply subnet service types spec
to resolve this problem. Any thoughts?


Hi,

This is going to be covered in the existing RFE for subnet service types [1]. 
We currently have two reviews in progress for CRUD [2] and CLI [3], the IPAM 
changes are next.


-Brian

[1] https://review.openstack.org/#/c/300207/
[2] https://review.openstack.org/#/c/337851/
[3] https://review.openstack.org/#/c/342976/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Proposing Jakub Libosvar for testing core

[Brian Haley]

+1

On 07/22/2016 04:12 AM, Oleg Bondarev wrote:

+1

On Fri, Jul 22, 2016 at 2:36 AM, Doug Wiegley > wrote:

+1


On Jul 21, 2016, at 5:13 PM, Kevin Benton > wrote:

+1

On Thu, Jul 21, 2016 at 2:41 PM, Carl Baldwin > wrote:

+1 from me

On Thu, Jul 21, 2016 at 1:35 PM, Assaf Muller > wrote:

As Neutron's so called testing lieutenant I would like to propose
Jakub Libosvar to be a core in the testing area.

Jakub has demonstrated his inherent interest in the testing area 
over
the last few years, his reviews are consistently insightful and his
numbers [1] are in line with others and I know will improve if given
the responsibilities of a core reviewer. Jakub is deeply involved 
with
the project's testing infrastructures and CI systems.

As a reminder the expectation from cores is found here [2], and
specifically for cores interesting in helping out shaping Neutron's
testing story:

* Guide community members to craft a testing strategy for features 
[3]
* Ensure Neutron's testing infrastructures are sufficiently
sophisticated to achieve the above.
* Provide leadership when determining testing Do's & Don'ts [4]. 
What
makes for an effective test?
* Ensure the gate stays consistently green

And more tactically we're looking at finishing the Tempest/Neutron
tests dedup [5] and to provide visual graphing for historical 
control
and data plane performance results similar to [6].

[1] http://stackalytics.com/report/contribution/neutron/90
[2]

http://docs.openstack.org/developer/neutron/policies/neutron-teams.html
[3]

http://docs.openstack.org/developer/neutron/devref/development.environment.html#testing-neutron
[4] https://assafmuller.com/2015/05/17/testing-lightning-talk/
[5] https://etherpad.openstack.org/p/neutron-tempest-defork
[6]

https://www.youtube.com/watch?v=a0qlsH1hoKs=youtu.be=24m22s


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe


http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org
?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] [openstack][neutron] dvr_snat + l3_ha on compute nodes

[Brian Haley]

On 07/14/2016 07:12 AM, Vladislav Belogrudov wrote:

Hello,

is it possible to run dvr_snat on computes only (in ha mode as well)? Docs [1]
tell that dvr_snat+l3_ha agents should be on controllers / network nodes while
dvr agents on computes. Does dvr part of dvr_snat only work in active/passive if
we do ha?

I wonder if compute nodes can be network ones as well without much overhead.
This way network nodes would go away from the setup.


SNAT is always a centralized component of Neutron, since there is a single 
default SNAT IP address that must be shared by all VMs for a tenant - that IP 
can't be in two places at once.  Running in dvr_snat mode on computes will just 
make them all network nodes, which should be avoided since losing a compute then 
means you've lost more of your control plane.  They are typically separate due 
to the differences in how they are managed - network nodes are more like pets, 
computes more like cattle.


-Brian

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

[openstack-dev] [Neutron][DVR] - No IRC Meeting today

[Brian Haley]

Hi Folks,

Sorry for the late notice, but we will not have a DVR sub-team meeting today 
since neither Swami nor myself will be there to chair it.


We will resume our meetings next week on July 20th.

If you have any questions please ping us on IRC or send an email to the list.

https://wiki.openstack.org/wiki/Meetings/Neutron-DVR

Thanks,

-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] Can't get data from the neutron metadata proxy

[Brian Haley]

On 07/01/2016 05:49 AM, Turbo Fredriksson wrote:

On Jun 30, 2016, at 4:07 PM, Turbo Fredriksson wrote:

Stracing the running process, I noticed this:

connect(8, {sa_family=AF_LOCAL, sun_path="/var/lib/neutron/metadata_proxy"}, 
33) = -1 ENOENT (No such file or directory)

Creating the socket manually, I instead get "Connection refused".
So whatever is/was supposed to create that socket, isn't running.

What else than the neutron-ns-metadata-proxy are supposed to create
this?

bladeA01:~# ps | grep meta
  4946 ?S  0:00 /usr/bin/python2.7 
/usr/bin/neutron-ns-metadata-proxy 
--pid_file=/var/lib/neutron/external/pids/39c01b3c-7709-4e4b-9455-60daf31f2b1e.pid
 --metadata_proxy_socket=/var/lib/neutron/metadata_proxy 
--router_id=39c01b3c-7709-4e4b-9455-60daf31f2b1e --state_path=/var/lib/neutron 
--metadata_port=80 --metadata_proxy_user=120 --metadata_proxy_group=125 
--verbose 
--log-file=neutron-ns-metadata-proxy-39c01b3c-7709-4e4b-9455-60daf31f2b1e.log 
--log-dir=/var/log/neutron


You also need to have the neutron-metadata-agent running anywhere you run the 
proxy (well, anywhere the l3-agent is running since it starts them typically). 
Each namespace proxy communicates with it over a Unix Domain Socket, and it 
communicates with the Nova metadata server.


-Brian

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

[openstack-dev] [Neutron][DVR] - No IRC Meeting this week

[Brian Haley]

Hi Folks,

We will not have a DVR sub-team meeting this week since neither Swami nor myself 
will be there to chair it.


We will resume our meetings next week on July 6th.

If you have any questions please ping us on IRC or send an email to the list.

https://wiki.openstack.org/wiki/Meetings/Neutron-DVR

Thanks,

-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] packets not reaching VM

[Brian Haley]

On 06/23/2016 03:23 AM, Priyanka wrote:

Hi,

We want direct routing LB and LVS supports it. So we were trying that option.
Can we add some rule in neutron-openvswi chain of the LB VM on compute node to
prevent the drop of these packets? If yes please guide us on how can we
configure such a rule. As i can see a drop rule in the chain which drops
anything other than packet with IP and MAC of the LB VM. But our packet has a
different IP. The rule addition would be required at the backend server VM
neutron-openvswi chain as well?


Doing it on your own is fine, but do know that there's only so much support we 
can give when you're not using the built-in tools that already exist.  Luckily 
you have all the Neutron source code, especially since you're running an 
unsupported release like Juno.


Here are two things you can look at:

1) Allowed address pairs

2) Remove the port security feature on certain ports:

2a) remove the security group from the port
2b) neutron port-update $port --port-security-enabled=False

Typically you'd create the port first, then pass it along during the nova boot 
phase, but you should be able to update it afterwards.


-Brian


On Wednesday 22 June 2016 08:16 PM, Brian Haley wrote:

On 06/22/2016 03:42 AM, Priyanka wrote:

Hi,

We have a Openstack Juno setup with 1 controller+neutron node and 3 compute
nodes. 1 VM (LB) has ipvsadm installed and two VMs act as back end server.

On the server with ipvsadm I have eth0:0 IP as 192.168.1.21 which acts as
application IP. The ipvsadm uses round robin scheme. This is done using commands
as below:

sudo ipvsadm -A -t 192.168.1.21:6000 -s rr
sudo ipvsadm -a -t 192.168.1.21:6000 -r 192.168.1.77:6000 -g
sudo ipvsadm -a -t 192.168.1.21:6000 -r 192.168.1.79:6000 -g

where 192.168.1.77 and 192.168.1.79 are back end server VM IP.

The problem is that the packets go out of the LB VM but never reach the back end
server.


You had asked a similar question last week, and I had asked why you just
weren't using Neutron LBaaS to do this?  Seems you are trying to implement
your own load-balancer inside a tenant VM.

Also, Juno is very old, using a newer release would give you access to Octavia
(LBaaS v2) that has more advanced features.


In the tcpdumps on various interfaces show that the packet reach till qbr of the
LB VM but donot reach the qvo interface of LB VM. Are there any rules that get
applied here which block these packets. The packets from the client VM are sent
to back end server by the LB VM by changing the destination MAC of the packets.
  The packets that leave LB VM to reach back end VM have source as the client VM
IP and destination IP as 192.168.1.21 (application IP) and the src MAC of LB VM
and dst MAC of backend server VM. Is this the reason for the packets to be
blocked. Is there any way to allow these packets to flow to the back end server?


There are anti-spoofing rules installed that are most likely causing the
packets to get dropped.

-Brian





___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] packets not reaching VM

[Brian Haley]

On 06/22/2016 03:42 AM, Priyanka wrote:

Hi,

We have a Openstack Juno setup with 1 controller+neutron node and 3 compute
nodes. 1 VM (LB) has ipvsadm installed and two VMs act as back end server.

On the server with ipvsadm I have eth0:0 IP as 192.168.1.21 which acts as
application IP. The ipvsadm uses round robin scheme. This is done using commands
as below:

sudo ipvsadm -A -t 192.168.1.21:6000 -s rr
sudo ipvsadm -a -t 192.168.1.21:6000 -r 192.168.1.77:6000 -g
sudo ipvsadm -a -t 192.168.1.21:6000 -r 192.168.1.79:6000 -g

where 192.168.1.77 and 192.168.1.79 are back end server VM IP.

The problem is that the packets go out of the LB VM but never reach the back end
server.


You had asked a similar question last week, and I had asked why you just weren't 
using Neutron LBaaS to do this?  Seems you are trying to implement your own 
load-balancer inside a tenant VM.


Also, Juno is very old, using a newer release would give you access to Octavia 
(LBaaS v2) that has more advanced features.



In the tcpdumps on various interfaces show that the packet reach till qbr of the
LB VM but donot reach the qvo interface of LB VM. Are there any rules that get
applied here which block these packets. The packets from the client VM are sent
to back end server by the LB VM by changing the destination MAC of the packets.
  The packets that leave LB VM to reach back end VM have source as the client VM
IP and destination IP as 192.168.1.21 (application IP) and the src MAC of LB VM
and dst MAC of backend server VM. Is this the reason for the packets to be
blocked. Is there any way to allow these packets to flow to the back end server?


There are anti-spoofing rules installed that are most likely causing the packets 
to get dropped.


-Brian

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [Openstack] load balancer using iptables rules in neutron

[Brian Haley]

On 06/15/2016 12:18 AM, Priyanka wrote:

Hi,

I want to create a basic round-robin layer 3 load balancer. The request would be
coming from same tenant network and the backend servers would also be in the
same network. For this the steps I think should be done are: create a neutron
port (for the load balancer) and attach it to ovs on the neutron node and apply
ip tables to load balance traffic coming to this port (using the nth patch of
iptable).

Are the steps I am thinking correct? Also, to which chain should these rules be
put? Please guide me on this.


Why don't you just use LBaaS?

https://wiki.openstack.org/wiki/Neutron/LBaaS/HowToRun

Manipulating iptables rules manually will just lead to problems, see a response 
I sent yesterday to this list for an example.


-Brian

___
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] [Neutron] Question about service subnets spec

[Brian Haley]

Thanks Carl for bringing this up, comments below.

On 05/26/2016 02:04 PM, Carl Baldwin wrote:

Hi folks,

Some (but not all) of you will remember a discussion we had about
service subnets at the last mid-cycle.  We've been iterating a little
bit on a spec [1] and we have just one issue that we'd like to get a
little bit more feedback on.

As a summary:  To me, the idea of this spec is to reserve certain
subnets for certain kinds of ports.  For example, DVR FIP gateway
ports, and router ports.  The goal of this is to be able to use
subnets with private addresses for these kinds of ports instead of
wasting public IP addresses.

The remaining question is how to expose this through the API.  I had
thought about just attaching a list of device_owners to the subnet
resource.  If a list is attached, then only ports with the right
device_owner will be allocated IP addresses from that subnet.  I
thought this would be an easy way to implement it and I thought since
device owner is already exposed through the API, maybe it would be
acceptable.  However, there is some concern that this exposes too much
of the internal implementation.  I understand this concern.

At the mid-cycle we had discussed some enumeration values that
combined several types to avoid having to allow a list of types on a
subnet.  They were going to look like this:

   dvr_gateway -> ["network:floatingip_agent_gateway"]
   router_gateway -> ["network:floatingip_agent_gateway",
"network:router_gateway"]

The idea was that we'd only allow one value for a subnet and the
difference between the two would be whether you wanted router ports to
use private IPs.  I think it would be clearer if we just have simpler
definitions of types and allow a list of them.


Yes, this was the original plan - two values (well, three since None was 
default), each mapping to a set of owners.



At this point the enumeration values map simply to device owners.  For example:

   router_ports -> "network:router_gateway"
   dvr_fip_ports -> "network:floatingip_agent_gateway"

It was at this point that I questioned the need for the abstraction at
all.  Hence the proposal to use the device owners directly.


I would agree, think having another name to refer to a device_owner makes it 
more confusing.  Using it directly let's us be flexible for deployers, and 
allows for using additional owners values if/when they are added.



Armando expressed some concern about using the device owner as a
security issue.  We have the following policy on device_owner:

   "not rule:network_device or rule:context_is_advsvc or
rule:admin_or_network_owner"

At the moment, I don't see this as much of an issue.  Do you?


I don't, since only admins should be able to set device_owner to these values 
(that's the policy we're talking about here, right?).


To be honest, I think Armando's other comment - "Do we want to expose 
device_owner via tha API or leave it an implementation detail?" is important as 
well.  Even though I think an admin should know this level of neutron detail, 
will they really?  It's hard to answer that question being so close to the code.


-Brian


[1] 
https://review.openstack.org/#/c/300207/3/specs/newton/subnet-service-types.rst




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev