Re: [openstack-dev] [Neutron][networking-ovn][networking-odl] Syncing neutron DB and OVN DB

[Amitabha Biswas]

Hi Numan,

Thanks for the proposal. We have also been thinking about this use-case.

If I’m reading this accurately (and I may not be), it seems that the proposal 
is to not have any OVN NB (CUD) operations (R operations outside the scope) 
done by the api_worker threads but rather by a new journal thread.

If this is indeed the case, I’d like to consider the scenario when there any N 
neutron nodes, each node with M worker threads. The journal thread at the each 
node contain list of pending operations. Could there be (sequence) dependency 
in the pending operations amongst each the journal threads in the nodes that 
prevents them from getting applied (for e.g. Logical_Router_Port and 
Logical_Switch_Port inter-dependency), because we are returning success on 
neutron operations that have still not been committed to the NB DB.

Couple of clarifications and thoughts below.

Thanks
Amitabha 

> On Jul 13, 2016, at 1:20 AM, Numan Siddique  wrote:
> 
> Adding the proper tags in subject
> 
> On Wed, Jul 13, 2016 at 1:22 PM, Numan Siddique  > wrote:
> Hi Neutrinos,
> 
> Presently, In the OVN ML2 driver we have 2 ways to sync neutron DB and OVN DB
>  - At neutron-server startup, OVN ML2 driver syncs the neutron DB and OVN DB 
> if sync mode is set to repair.
>  - Admin can run the "neutron-ovn-db-sync-util" to sync the DBs.
> 
> Recently, in the v2 of networking-odl ML2 driver (Please see (1) below which 
> has more details). (ODL folks please correct me if I am wrong here)
> 
>   - a journal thread is created which does the CRUD operations of neutron 
> resources asynchronously (i.e it sends the REST APIs to the ODL controller).

Would this be the equivalent of making OVSDB transactions to the OVN NB DB?

>   - a maintenance thread is created which does some cleanup periodically and 
> at startup does full sync if it detects ODL controller cold reboot.
> 
> 
> Few question I have
>  - can OVN ML2 driver take same or similar approach. Are there any advantages 
> in taking this approach ? One advantage is neutron resources can be 
> created/updated/deleted even if the OVN ML2 driver has lost connection to the 
> ovsdb-server. The journal thread would eventually sync these resources in the 
> OVN DB. I would like to know the communities thoughts on this.

If we can make it work, it would indeed be a huge plus for system wide upgrades 
and some corner cases in the code (ACL specifically), where the post_commit 
relies on all transactions to be successful and doesn’t revert the neutron db 
if something fails.

> 
>  - Are there are other ML2 drivers which might have to handle the DB sync's 
> (cases where the other controllers also maintain their own DBs) and how they 
> are handling it ?
> 
>  - Can a common approach be taken to sync the neutron DB and controller DBs ?
> 
> 
> ---
> 
> (1)
> Sync threads created by networking-odl ML2 driver
> --
> ODL ML2 driver creates 2 threads (threading.Thread module) at init
>  - Journal thread
>  - Maintenance thread
> 
> Journal thread
> 
> The journal module creates a new journal table by name “opendaylightjournal”  
> - 
> https://github.com/openstack/networking-odl/blob/master/networking_odl/db/models.py#L23
>  
> 
> 
> Journal thread will be in loop waiting for the sync event from the ODL ML2 
> driver.
> 
>  - ODL ML2 driver resource (network, subnet, port) precommit functions when 
> called by the ML2 plugin adds an entry in the “opendaylightjournal” table 
> with the resource data and sets the journal operation state for this entry to 
> “PENDING”.
>  - The corresponding resource postcommit function of the ODL ML2 plugin when 
> called, sets the sync event flag.
>  - A timer is also created which sets the sync event flag when it expires 
> (the default value is 10 seconds).
>  - Journal thread wakes up, looks into the “opendaylightjournal” table with 
> the entries with state “pending” and runs the CRUD operation on those 
> resources in the ODL DB. Once done, it sets the state to “completed”.
> 
> Maintenance thread
> --
> Maintenance thread does 3 operations
>  - JournalCleanup - Delete completed rows from journal table 
> “opendaylightjournal”.
>  - CleanupProcessing - Mark orphaned processing rows to pending.
>  - Full sync - Re-sync when detecting an ODL "cold reboot”.
> 
> 
> 
> Thanks
> Numan
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Dev

Re: [openstack-dev] [ovs-discuss] [OVN] [networking-ovn] [networking-sfc] SFC andOVN

[Amitabha Biswas]

I didn’t see that idea earlier, +2 to it :)

Regards
Amitabha

> On Jun 8, 2016, at 9:52 AM, John McDowall  
> wrote:
> 
> Amitabha,
> 
> Thanks for looking at it . I took the suggestion from Juno and implemented 
> it. I think it is a good solution as it minimizes impact on both 
> networking-ovn and networking-sfc. I have updated my repos, if you have 
> suggestions for improvements let me know.
> 
> I agree that there needs to be some refactoring of the networking-sfc driver 
> code. I think the team did a good job with it as it was easy for me to create 
> the OVN driver ( copy and paste). As more drivers are created I think the 
> model will get polished and refactored.
> 
> Regards
> 
> John
> 
> From: Amitabha Biswas mailto:azbis...@gmail.com>>
> Date: Tuesday, June 7, 2016 at 11:36 PM
> To: John McDowall  <mailto:jmcdow...@paloaltonetworks.com>>
> Cc: Na Zhu mailto:na...@cn.ibm.com>>, Srilatha Tangirala 
> mailto:srila...@us.ibm.com>>, "OpenStack Development 
> Mailing List (not for usage questions)"  <mailto:openstack-dev@lists.openstack.org>>, discuss  <mailto:disc...@openvswitch.org>>
> Subject: Re: [ovs-discuss] [openstack-dev] [OVN] [networking-ovn] 
> [networking-sfc] SFC andOVN
> 
> Hi John,
> 
> Looking at the code with Srilatha, it seems like the 
> https://github.com/doonhammer/networking 
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_doonhammer_networking&d=CwMF-g&c=V9IgWpI5PvzTw83UyHGVSoW3Uc1MFWe5J8PTfkrzVSo&r=vZ6VUDaavDpfOdPQrz1ED54jEjvAE36A8TVJroVlrOQ&m=HkxBkhm69SxTBQTw2-s5MvEDiFsrOjXero1F3DeFO88&s=0qATypbGGShogPxjkRq8oKnjHMUvVwiWLAY17gBr6iM&e=>-ovn
>  repo has gone down the path of having a sfc_ovn.py file in the 
> networking-ovn/ovsdb directory. This file deals with the SFC specific OVSDB 
> transactions in OVN. So to answer your question of invoking OVS-IDL, we can 
> import the src_ovn.py file from 
> networking_sfc/services/src/drivers/ovn/driver.py and invoke calls into IDL.
> 
> Another aspect from a networking-sfc point of view is the duplication of code 
> between networking_sfc/services/src/drivers/ovn/driver.py and 
> networking_sfc/services/src/drivers/ovs/driver.py in the 
> https://github.com/doonhammer/networking-sfc 
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_doonhammer_networking-2Dsfc&d=CwMF-g&c=V9IgWpI5PvzTw83UyHGVSoW3Uc1MFWe5J8PTfkrzVSo&r=vZ6VUDaavDpfOdPQrz1ED54jEjvAE36A8TVJroVlrOQ&m=HkxBkhm69SxTBQTw2-s5MvEDiFsrOjXero1F3DeFO88&s=8ftwwsdvSwYTc3WSmDDad9L9dwV6CwaIuOwtoN5FwqA&e=>
>  repo. There should be a mechanism to coalesce the common code and invoke the 
> OVS and OVN specific parts separately.
> 
> Regards
> Amitabha
> 
>> On Jun 7, 2016, at 9:54 PM, John McDowall > <mailto:jmcdow...@paloaltonetworks.com>> wrote:
>> 
>> Juno, Srilatha,
>> 
>> I need some help – I have fixed most of the obvious typo’s in the three 
>> repos and merged them with mainline. There is still a problem with the build 
>> I think in mech_driver.py but I will fix it asap in the am.
>> 
>> However I am not sure of the best way to interface between sfc and ovn.
>> 
>> In networking_sfc/services/src/drivers/ovn/driver.py there is a function 
>> that creates a deep copy of the port-chain dict, 
>> create_port_chain(self,contact,port_chain). 
>> 
>> Looking at networking-ovn I think it should use mech_driver.py so we can 
>> call the OVS-IDL to send the parameters to ovn. However I am not sure of the 
>> best way to do it. Could you make some suggestions or send me some sample 
>> code showing the best approach?
>> 
>> I will get the ovs/ovn cleaned up and ready. Also Louis from the 
>> networking-sfc has posted a draft blueprint.
>> 
>> Regards
>> 
>> John
>> 
>> From: Na Zhu mailto:na...@cn.ibm.com>>
>> Date: Monday, June 6, 2016 at 7:54 PM
>> To: John McDowall > <mailto:jmcdow...@paloaltonetworks.com>>, Ryan Moats > <mailto:rmo...@us.ibm.com>>
>> Cc: "disc...@openvswitch.org <mailto:disc...@openvswitch.org>" 
>> mailto:disc...@openvswitch.org>>, "OpenStack 
>> Development Mailing List (not for usage questions)" 
>> > <mailto:openstack-dev@lists.openstack.org>>, Srilatha Tangirala 
>> mailto:srila...@us.ibm.com>>
>> Subject: Re: [openstack-dev] [ovs-discuss] [OVN] [networking-ovn] 
>> [networking-sfc] SFC andOVN
>> 
>> Hi John,
>> 
>> I do not know any better approach, I think it is good to write all the 
>> parameters in the creation of a port chain, t

Re: [openstack-dev] [neutron][networking-ovn] Integration with OVN NAT gateway (Proposal)

[Amitabha Biswas]

Here is the proposal in etherpad to make it more readable:

https://etherpad.openstack.org/p/Integration_with_OVN_L3_Gateway 
<https://etherpad.openstack.org/p/Integration_with_OVN_L3_Gateway>

Thanks
Amitabha

> On Jun 7, 2016, at 5:12 PM, Amitabha Biswas  wrote:
> 
> Sorry that was a typo, it should read:
> 
>> Note that the MAC addresses of gtrp and dtrp will be the same on each OVN 
>> Join Network, but because they are in different branches of the network 
>> topology it doesn’t matter.
> Amitabha
> 
>> On Jun 7, 2016, at 4:39 PM, Bhalachandra Banavalikar 
>> mailto:bhal.banavali...@us.ibm.com>> wrote:
>> 
>> Can you please provide more details on lgrp and lip ports (last bullet in 
>> section 1)?
>> 
>> Thanks,
>> Bhal
>> 
>> Amitabha Biswas ---06/07/2016 01:56:23 PM---This proposal 
>> outlines the modifications needed in networking-ovn (addresses 
>> https://bugs.launchpad <https://bugs.launchpad/>.
>> 
>> From:  Amitabha Biswas mailto:azbis...@gmail.com>>
>> To:  "OpenStack Development Mailing List (not for usage questions)" 
>> > <mailto:openstack-dev@lists.openstack.org>>
>> Cc:  Chandra Sekhar Vejendla/San Jose/IBM@IBMUS
>> Date:  06/07/2016 01:56 PM
>> Subject:  [openstack-dev] [neutron][networking-ovn] Integration with OVN NAT 
>> gateway (Proposal)
>> 
>> 
>> 
>> 
>> This proposal outlines the modifications needed in networking-ovn (addresses 
>> https://bugs.launchpad.net/networking-ovn/+bug/1551717 
>> <https://bugs.launchpad.net/networking-ovn/+bug/1551717>) to provide 
>> Floating IP (FIP) and SNAT using the L3 gateway router patches.
>> 
>> http://patchwork.ozlabs.org/patch/624312/ 
>> <http://patchwork.ozlabs.org/patch/624312/> 
>> http://patchwork.ozlabs.org/patch/624313/ 
>> <http://patchwork.ozlabs.org/patch/624312/> 
>> http://patchwork.ozlabs.org/patch/624314/ 
>> <http://patchwork.ozlabs.org/patch/624312/> 
>> http://patchwork.ozlabs.org/patch/624315/ 
>> <http://patchwork.ozlabs.org/patch/624312/> 
>> http://patchwork.ozlabs.org/patch/629607/ 
>> <http://patchwork.ozlabs.org/patch/629607/>
>> 
>> Diagram:
>> 
>> +---+ +---+
>> | NET 1 | | NET 2 |
>> +---+ +---+
>> | |
>> | * |
>> | ** ** |
>> | ** * * ** |
>> +---RP1 * DR * RP2 --+
>> ** * * **
>> ** **  
>> *  
>> DTRP (168.254.128.2)
>> |
>> |
>> |
>> +--+
>> | Transit Network |
>> | 169.254.128.0/30 |
>> +--+
>> |
>> |
>> |
>> |
>> GTRP (169.254.128.1)
>> ***  
>> ** **  
>> ** * * ** +--+
>> * GW *-| Provider Network |
>> ** * * ** +--+
>> ** **  
>> ***  
>> 
>> New Entities:
>> OVN Join/Transit Networks
>> One per Neutron Router - /30 address space with only 2 ports for e.g. 
>> 169.254.128.0/30
>> Created when an external gateway is added to a router.
>> One extra datapath per router with an External Gateway.
>> (Alternate option - One Transit Network in a deployment, IPAM becomes a 
>> headache - Not discussed here).
>> Prevent Neutron from using that /30 address space. Specify in networking-ovn 
>> conf file.
>> Create 1 new “Join” neutron network (to represent all Join OVN Networks) in 
>> the networking-ovn.
>> Note that it may be possible to replace the Join/Transit network using 
>> Router Peering in later versions (not discussed here).
>> Allocate 2 ports in the Join network in the networking-ovn plugin.
>> Logical Gateway Transit Router Port (gtrp), 169.254.128.1
>> Logical Distributed Transit Router Port (dtrp), 169.254.128.2
>> Note that Neutron only sees 1 Join network with 2 ports; OVN sees a replica 
>> of this Join network as a new Logical Switch for each Gateway Router. The 
>> mapping of OVN Logical Switch(es) Join(s) to Gateway Router is discussed in 
>> OVN (Default) Gateway Routers below.
>> Note that the MAC addresses of gtrp and dtrp will be the same on each OVN 
>> Join Network, but because they are in different branches of the network 
>> topology it doesn’t matter.
>> OVN (Default) Gateway Routers:
>> One per Neutron Router.
>> 2 ports
>> Logical Gateway Transit Router Port (gtrp), 169.254.128.1 (same for each OVN 
>> Join network).
>> External/Provider Router Port (legwrp), this is allocated by neutron.
>> Scheduling - The current OVN gateway pr

Re: [openstack-dev] [ovs-discuss] [OVN] [networking-ovn] [networking-sfc] SFC andOVN

[Amitabha Biswas]

Hi John,

Looking at the code with Srilatha, it seems like the 
https://github.com/doonhammer/networking 
-ovn repo has gone down the path of 
having a sfc_ovn.py file in the networking-ovn/ovsdb directory. This file deals 
with the SFC specific OVSDB transactions in OVN. So to answer your question of 
invoking OVS-IDL, we can import the src_ovn.py file from 
networking_sfc/services/src/drivers/ovn/driver.py and invoke calls into IDL.

Another aspect from a networking-sfc point of view is the duplication of code 
between networking_sfc/services/src/drivers/ovn/driver.py and 
networking_sfc/services/src/drivers/ovs/driver.py in the 
https://github.com/doonhammer/networking-sfc 
 repo. There should be a 
mechanism to coalesce the common code and invoke the OVS and OVN specific parts 
separately.

Regards
Amitabha

> On Jun 7, 2016, at 9:54 PM, John McDowall  
> wrote:
> 
> Juno, Srilatha,
> 
> I need some help – I have fixed most of the obvious typo’s in the three repos 
> and merged them with mainline. There is still a problem with the build I 
> think in mech_driver.py but I will fix it asap in the am.
> 
> However I am not sure of the best way to interface between sfc and ovn.
> 
> In networking_sfc/services/src/drivers/ovn/driver.py there is a function that 
> creates a deep copy of the port-chain dict, 
> create_port_chain(self,contact,port_chain). 
> 
> Looking at networking-ovn I think it should use mech_driver.py so we can call 
> the OVS-IDL to send the parameters to ovn. However I am not sure of the best 
> way to do it. Could you make some suggestions or send me some sample code 
> showing the best approach?
> 
> I will get the ovs/ovn cleaned up and ready. Also Louis from the 
> networking-sfc has posted a draft blueprint.
> 
> Regards
> 
> John
> 
> From: Na Zhu mailto:na...@cn.ibm.com>>
> Date: Monday, June 6, 2016 at 7:54 PM
> To: John McDowall  >, Ryan Moats  >
> Cc: "disc...@openvswitch.org " 
> mailto:disc...@openvswitch.org>>, "OpenStack 
> Development Mailing List (not for usage questions)" 
>  >, Srilatha Tangirala 
> mailto:srila...@us.ibm.com>>
> Subject: Re: [openstack-dev] [ovs-discuss] [OVN] [networking-ovn] 
> [networking-sfc] SFC andOVN
> 
> Hi John,
> 
> I do not know any better approach, I think it is good to write all the 
> parameters in the creation of a port chain, this can avoid saving many data 
> in northbound db which are not used. We can do it in that way currently, if 
> the community has opposite ideas, we can change, what do you think?
> 
> Hi Ryan,
> 
> Do you agree with that?
> 
> 
> 
> Regards,
> Juno Zhu
> IBM China Development Labs (CDL) Cloud IaaS Lab
> Email: na...@cn.ibm.com 
> 5F, Building 10, 399 Keyuan Road, Zhangjiang Hi-Tech Park, Pudong New 
> District, Shanghai, China (201203)
> 
> 
> 
> From:John McDowall  >
> To:Na Zhu/China/IBM@IBMCN
> Cc:"disc...@openvswitch.org " 
> mailto:disc...@openvswitch.org>>, Ryan Moats 
> mailto:rmo...@us.ibm.com>>, Srilatha Tangirala 
> mailto:srila...@us.ibm.com>>, "OpenStack Development 
> Mailing List (not for usage questions)"  >
> Date:2016/06/06 23:36
> Subject:Re: [openstack-dev] [ovs-discuss] [OVN] [networking-ovn] 
> [networking-sfc] SFC andOVN
> 
> 
> 
> Juno,
> 
> Let me check – my intention was that the networking-sfc OVNB driver would 
> configure all aspects of the port-chain and add the parameters to the 
> networking-sfc db. Once all the parameters were in the creation of a 
> port-chain would call networking-ovn (passing a deep copy of the port-chain 
> dict). Here I see networking-ovn acting only as a bridge into ovs/ovn (I did 
> not add anything in the ovn plugin – not sure if that is the right approach). 
> Networking-ovn calls into ovs/ovn and inserts the entire port-chain.
> 
> Thoughts?
> 
> j
> 
> From: Na Zhu mailto:na...@cn.ibm.com>>
> Date: Monday, June 6, 2016 at 5:49 AM
> To: John McDowall  >
> Cc: "disc...@openvswitch.org " 
> mailto:disc...@openvswitch.org>>, Ryan Moats 
> mailto:rmo...@us.ibm.com>>, Srilatha Tangirala 
> mailto:srila...@us.ibm.com>>, "OpenStack Development 
> Mailing List (not for usage questions)"  >
> Subject: Re: [openstack-dev] [ovs-discuss] [OVN] [networking-ovn] 
> [networking-sfc] SFC andOVN
> 
> Hi John,
> 
> One question need confirm with you, I think the ovn flow classifier driver 
> and ovn port chain driver should call the APIs which you add to 
> networking-ovn to configure the northbound db sfc tables, right? I see your 
> networking-sfc ovn drivers, 

Re: [openstack-dev] [neutron][networking-ovn] Integration with OVN NAT gateway (Proposal)

[Amitabha Biswas]

Sorry that was a typo, it should read:

> Note that the MAC addresses of gtrp and dtrp will be the same on each OVN 
> Join Network, but because they are in different branches of the network 
> topology it doesn’t matter.
Amitabha

> On Jun 7, 2016, at 4:39 PM, Bhalachandra Banavalikar 
>  wrote:
> 
> Can you please provide more details on lgrp and lip ports (last bullet in 
> section 1)?
> 
> Thanks,
> Bhal
> 
> Amitabha Biswas ---06/07/2016 01:56:23 PM---This proposal 
> outlines the modifications needed in networking-ovn (addresses 
> https://bugs.launchpad <https://bugs.launchpad/>.
> 
> From: Amitabha Biswas 
> To: "OpenStack Development Mailing List (not for usage questions)" 
> 
> Cc: Chandra Sekhar Vejendla/San Jose/IBM@IBMUS
> Date: 06/07/2016 01:56 PM
> Subject: [openstack-dev] [neutron][networking-ovn] Integration with OVN NAT 
> gateway (Proposal)
> 
> 
> 
> 
> This proposal outlines the modifications needed in networking-ovn (addresses 
> https://bugs.launchpad.net/networking-ovn/+bug/1551717 
> <https://bugs.launchpad.net/networking-ovn/+bug/1551717>) to provide Floating 
> IP (FIP) and SNAT using the L3 gateway router patches.
> 
> http://patchwork.ozlabs.org/patch/624312/ 
> <http://patchwork.ozlabs.org/patch/624312/> 
> http://patchwork.ozlabs.org/patch/624313/ 
> <http://patchwork.ozlabs.org/patch/624312/> 
> http://patchwork.ozlabs.org/patch/624314/ 
> <http://patchwork.ozlabs.org/patch/624312/> 
> http://patchwork.ozlabs.org/patch/624315/ 
> <http://patchwork.ozlabs.org/patch/624312/> 
> http://patchwork.ozlabs.org/patch/629607/ 
> <http://patchwork.ozlabs.org/patch/629607/>
> 
> Diagram:
> 
> +---+ +---+
> | NET 1 | | NET 2 |
> +---+ +---+
> | |
> | * |
> | ** ** |
> | ** * * ** |
> +---RP1 * DR * RP2 --+
> ** * * **
> ** ** 
> * 
> DTRP (168.254.128.2)
> |
> |
> |
> +--+
> | Transit Network |
> | 169.254.128.0/30 |
> +--+
> |
> |
> |
> |
> GTRP (169.254.128.1)
> *** 
> ** ** 
> ** * * ** +--+
> * GW *-| Provider Network |
> ** * * ** +--+
> ** ** 
> *** 
> 
> New Entities:
> OVN Join/Transit Networks
> One per Neutron Router - /30 address space with only 2 ports for e.g. 
> 169.254.128.0/30
> Created when an external gateway is added to a router.
> One extra datapath per router with an External Gateway.
> (Alternate option - One Transit Network in a deployment, IPAM becomes a 
> headache - Not discussed here).
> Prevent Neutron from using that /30 address space. Specify in networking-ovn 
> conf file.
> Create 1 new “Join” neutron network (to represent all Join OVN Networks) in 
> the networking-ovn.
> Note that it may be possible to replace the Join/Transit network using Router 
> Peering in later versions (not discussed here).
> Allocate 2 ports in the Join network in the networking-ovn plugin.
> Logical Gateway Transit Router Port (gtrp), 169.254.128.1
> Logical Distributed Transit Router Port (dtrp), 169.254.128.2
> Note that Neutron only sees 1 Join network with 2 ports; OVN sees a replica 
> of this Join network as a new Logical Switch for each Gateway Router. The 
> mapping of OVN Logical Switch(es) Join(s) to Gateway Router is discussed in 
> OVN (Default) Gateway Routers below.
> Note that the MAC addresses of gtrp and dtrp will be the same on each OVN 
> Join Network, but because they are in different branches of the network 
> topology it doesn’t matter.
> OVN (Default) Gateway Routers:
> One per Neutron Router.
> 2 ports
> Logical Gateway Transit Router Port (gtrp), 169.254.128.1 (same for each OVN 
> Join network).
> External/Provider Router Port (legwrp), this is allocated by neutron.
> Scheduling - The current OVN gateway proposal relies on the CMS/nbctl to 
> decide on which hypervisor (HV) to schedule a particular gateway router.
> A setting on the chassis (new external_id key or a new column) that allows 
> the hypervisor admin to specify that a chassis can or cannot be used to host 
> a gateway router (similar to a network node in OpenStack). Default - Allow 
> (for compatibility purposes).
> The networking-ovn plugin picks up the list of “candidate” chassis from the 
> Southbound DB and uses an existing scheduling algorithm
> Use a simple random.choice i.e. ChanceScheduler (Version 1)
> Tap into the neutron’s LeastRouterScheduler - but that requires the 
> networking-ovn (or some a hacked up version of the L3 agent) to imitate the 
> L3 agent running on various network nodes.
> Populate the SNAT and DNAT columns in the logical router ta

[openstack-dev] [neutron][networking-ovn] Integration with OVN NAT gateway (Proposal)

[Amitabha Biswas]

This proposal outlines the modifications needed in networking-ovn (addresses 
https://bugs.launchpad.net/networking-ovn/+bug/1551717 
) to provide Floating 
IP (FIP) and SNAT using the L3 gateway router patches.

http://patchwork.ozlabs.org/patch/624312/ 
 
http://patchwork.ozlabs.org/patch/624313/ 
 
http://patchwork.ozlabs.org/patch/624314/ 
 
http://patchwork.ozlabs.org/patch/624315/ 
 
http://patchwork.ozlabs.org/patch/629607/ 


Diagram:

+---+   +---+
| NET 1 |   | NET 2 |
+---+   +---+
   |   |
   |   *   |
   | ** ** |
   |   ***  * **   |
   +---RP1 *  DR   * RP2 --+
   ***  * **
 ** **  
   *
  DTRP (168.254.128.2)
   |
   |
   |
   +--+
   | Transit Network  |
   | 169.254.128.0/30 |
   +--+
   |
   |
   |
   |
  GTRP (169.254.128.1)
*** 
  **   **   
**   *   *   ** +--+
* GW  *-| Provider Network |
**   *   *   ** +--+
  **   **   
*** 

New Entities:

OVN Join/Transit Networks
One per Neutron Router - /30 address space with only 2 ports for e.g. 
169.254.128.0/30
Created when an external gateway is added to a router.
One extra datapath per router with an External Gateway.
(Alternate option - One Transit Network in a deployment, IPAM becomes a 
headache - Not discussed here).
Prevent Neutron from using that /30 address space. Specify in networking-ovn 
conf file.
Create 1 new “Join” neutron network (to represent all Join OVN Networks) in the 
networking-ovn.
Note that it may be possible to replace the Join/Transit network using Router 
Peering in later versions  (not discussed here).
Allocate 2 ports in the Join network in the networking-ovn plugin.
Logical Gateway Transit Router Port (gtrp), 169.254.128.1
Logical Distributed Transit Router Port (dtrp), 169.254.128.2
Note that Neutron only sees 1 Join network with 2 ports; OVN sees a replica of 
this Join network as a new Logical Switch for each Gateway Router. The mapping 
of OVN Logical Switch(es) Join(s) to Gateway Router is discussed in OVN 
(Default) Gateway Routers below.
Note that the MAC addresses of lgrp and lip will be the same on each OVN Join 
Network, but because they are in different branches of the network topology it 
doesn’t matter.
OVN (Default) Gateway Routers:
One per Neutron Router.
2 ports
Logical Gateway Transit Router Port (gtrp), 169.254.128.1 (same for each OVN 
Join network).
External/Provider Router Port (legwrp), this is allocated by neutron.
Scheduling - The current OVN gateway proposal relies on the CMS/nbctl to decide 
on which hypervisor (HV) to schedule a particular gateway router.
A setting on the chassis (new external_id key or a new column) that allows the 
hypervisor admin to specify that a chassis can or cannot be used to host a 
gateway router (similar to a network node in OpenStack). Default - Allow (for 
compatibility purposes).
The networking-ovn plugin picks up the list of “candidate” chassis from the 
Southbound DB and uses an existing scheduling algorithm
Use a simple random.choice i.e. ChanceScheduler (Version 1)
Tap into the neutron’s LeastRouterScheduler - but that requires the 
networking-ovn (or some a hacked up version of the L3 agent) to imitate the L3 
agent running on various network nodes.
Populate the SNAT and DNAT columns in the logical router table. This is under 
review in OVS - http://openvswitch.org/pipermail/dev/2016-June/072169.html 

Create static routing entry in the gateway router to route tenant bound traffic 
to the distributed logical router.ar gate

Existing Entities:

Distributed Logical Routers:
Set the default gateway of the distributed logical router to the IP Address of 
the corresponding Logical Gateway Transit Router Port (169.254.128.1).

It would be good to get some feedback on this strategy. Guru mentioned that he 
saw a need for ARP response across multiple gateway routers, we don’t see that 
requirement in this design/use-case.

Thanks
Amitabha (azbiswas) and Chandra (chandrav)

__
OpenStack Development Mailing List (not for usage 

Re: [openstack-dev] [neutron] Social at the summit

[Amitabha Biswas]

+1

AZBiswas

> On Apr 25, 2016, at 1:25 PM, Michael Johnson  wrote:
> 
> +1
> 
> Michael
> 
> On Monday, April 25, 2016, Ananth  > wrote:
> Would be great to connect..
> +1
> 
> Regards
> Ananth
> Cloud & Network Solutions
> Cisco Systems
> 
> On Mon, Apr 25, 2016 at 2:13 PM, Zhipeng Huang  > wrote:
> +1 
> 
> On Mon, Apr 25, 2016 at 1:07 PM, Kyle Mestery  > wrote:
> OK, there is enough interest, I'll find a place on 6th Street for us
> and get a reservation for Thursday around 7 or so.
> 
> Thanks folks!
> 
> On Mon, Apr 25, 2016 at 12:30 PM, Zhou, Han  > wrote:
> > +1 :)
> >
> > Han Zhou
> > Irc: zhouhan
> >
> >
> > On Monday, April 25, 2016, Korzeniewski, Artur
> >  > > wrote:
> >>
> >> Sign me up :)
> >>
> >> Artur
> >> IRC: korzen
> >>
> >> -Original Message-
> >> From: Darek Smigiel [mailto:smigiel.dari...@gmail.com 
> >> ]
> >> Sent: Monday, April 25, 2016 7:19 PM
> >> To: OpenStack Development Mailing List (not for usage questions)
> >>  >> >
> >> Subject: Re: [openstack-dev] [neutron] Social at the summit
> >>
> >> Count me in!
> >> Will be good to meet all you guys!
> >>
> >> Darek (dasm) Smigiel
> >>
> >> > On Apr 25, 2016, at 12:13 PM, Doug Wiegley
> >> >  >> > > wrote:
> >> >
> >> >
> >> >> On Apr 25, 2016, at 12:01 PM, Ihar Hrachyshka  >> >> >
> >> >> wrote:
> >> >>
> >> >> WAT???
> >> >>
> >> >> It was never supposed to be core only. Everyone is welcome!
> >> >
> >> > +2
> >> >
> >> > irony intended.
> >> >
> >> > Socials are not controlled by gerrit ACLs.  :-)
> >> >
> >> > doug
> >> >
> >> >>
> >> >> Sent from my iPhone
> >> >>
> >> >>> On 25 Apr 2016, at 11:56, Edgar Magana  >> >>> >
> >> >>> wrote:
> >> >>>
> >> >>> Would you extend it to ex-cores?
> >> >>>
> >> >>> Edgar
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >>  On 4/25/16, 10:55 AM, "Kyle Mestery"  >>  > wrote:
> >> 
> >>  Ihar, Henry and I were talking and we thought Thursday night makes
> >>  sense for a Neutron social in Austin. If others agree, reply on this 
> >>  thread
> >>  and we'll find a place.
> >> 
> >>  Thanks!
> >>  Kyle
> >> 
> >>  ___
> >>  ___ OpenStack Development Mailing List (not for usage
> >>  questions)
> >>  Unsubscribe:
> >>  openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
> >>  
> >>  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev 
> >>  
> >> >>> 
> >> >>> __ OpenStack Development Mailing List (not for usage questions)
> >> >>> Unsubscribe:
> >> >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
> >> >>> 
> >> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev 
> >> >>> 
> >> >>
> >> >> _
> >> >> _ OpenStack Development Mailing List (not for usage questions)
> >> >> Unsubscribe:
> >> >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
> >> >> 
> >> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev 
> >> >> 
> >> >
> >> >
> >> > __
> >> >  OpenStack Development Mailing List (not for usage questions)
> >> > Unsubscribe:
> >> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
> >> > 
> >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev 
> >> > 
> >>
> >>
> >> __
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
> >> 
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev 
> >> 
> >>
> >> __
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
> >> 
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev 
> >> 
> >
> >

[openstack-dev] [networking-ovn] Supporting multiple routes per router interface

[Amitabha Biswas]

Hi,

The https://review.openstack.org/#/c/237820/ patch added the support for 
adding router interfaces in OVN NB. This patch only provides support for a 
single route per interface. Neutron supports multiple routes per router 
interface. While attempting to model this in OVN Northbound DB, I ran into 
some problems.

The OVN Northbound DB at this point supports only a single route per 
Logical Router Port. To solve the multiple route (per router interface) 
issue while working within this OVN NB model restriction, I created 
multiple lport and lrouter port for each subnet supported by a router 
interface.

Let's say a router interface supports both 192.168.1.0/24 and 
2001:db8:cafe::/64 routes, I added 2 lport and 2 lrouter ports in the OVN 
NB.

Logical Router Table:
UUID-PA ["fa:16:3e:59:80:ad 192.168.1.1"]   "port-1" 
{router-port="UUID-RA"} ["fa:16:3e:59:80:ad"] router
UUID-PB ["fa:16:3e:59:80:ad 2001:db8:cafe::1"]  "port-2" 
{router-port="UUID-RB"} ["fa:16:3e:59:80:ad"] router

Logical Router Port Table:
UUID-RA "fa:16:3e:59:80:ad" "port-1" "192.168.1.1/24"  []
UUID-RB "fa:16:3e:59:80:ad" "port-2" "2001:db8:cafe::1/64" []

Note that both ports (in both tables) have the same MAC corresponding to 
the neutron router interface MAC.

This results in the following logical flow:
...
table=3(switch_in_l2_lkup), priority=   50, match=(eth.dst == 
fa:16:3e:59:80:ad), action=(outport = "port-2"; output;)
table=3(switch_in_l2_lkup), priority=   50, match=(eth.dst == 
fa:16:3e:59:80:ad), action=(outport = "port-1"; output;)

As we can see there are 2 conflicting/overlapping rules for the MAC in 
table 3 and the packet could be sent to the wrong output port.

To support the multiple route per interface feature, we need to propose 
that OVN NB allow multiple routes per logical router port in ovs-dev or 
ovs-discuss. Additionally it seems that the MAC must be unique in the 
Logical Router Table per Logical Flow.

Thanks
Amitabha


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][networking-ovn][vtep] Proposal: support for vtep-gateway in ovn

[Amitabha Biswas]

Hi everyone,
I want to open up the discussion regarding how to support OVN VTEP gateway deployment and its lifecycle in Neutron. 
In the "Life Cycle of a VTEP gateway" part in the OVN architecture document (http://www.russellbryant.net/ovs-docs/ovn-architecture.7.pdf), step 3 is where the Neutron OVN plugin is involved. At a minimum, the Neutron OVN plugin will enable setting the type as "vtep" and the vtep-logical-switch and vtep-physical-switch options in the OVN_Northbound database.
 
There are 2 parts to the proposal/discussion - a short term solution and a long term one:
A short term solution (proposed by Russell Bryant) is similar to the work that was done for container support in OVN - using a binding profile http://networking-ovn.readthedocs.org/en/latest/containers.html. A ovn logical network/switch can be mapped to a vtep logical gateway by creating a port in that logical network and creating a binding profile for that port in the following manner:
neutron port-create --binding-profile '{"vtep-logical-switch":"vtep_lswitch_key", "vtep-physical-switch":"vtep_pswitch_key"}' private.
Where vtep-logical-switch and vtep-physical-switch should have been defined in the OVN_Southbound database by the previous steps (1,2) in the life cycle. 
 
For the longer term solution, there needs to be a discussion:
Should the knowledge about the physical and logical step gateway should be exposed to Neutron - if yes how? This would allow a Neutron NB API/extension to bind a “known” vtep gateway to the neutron logical network. This would be similar to the workflow done in the networking-l2gw extension https://review.openstack.org/#/c/144173/3/specs/kilo/l2-gateway-api.rst
1. Allow the admin to define and manage the vtep gateway through Neutron REST API.
2. Define connections between Neutron networks and gateways. This is conceptually similar to Step 3 of the step gateway performed by the OVN Plugin in the short term solution.
OR
Should OVN pursue it’s own Neutron extension (including vtep gateway support).
 
Thanks Amitabha



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][networking-ovn][tempest] devstack: gate-tempest-dsvm-networking-ovn failures in Openstack CI

[Amitabha Biswas]

>> Once in a while a test fails for e.g.
>> tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_two_subnets
>> that failed recently in Jenkins. But I am pretty sure it will succeed if
>> the suite is re-run.
> 
> Have you looked to see if the same test is failing for the regular
> neutron jobs?  Or does it seem to be OVN specific?

I have not seen this failure before in any of the other (around 30) ovn logs 
I’ve parsed. It was due to an existing IP Address in the create_subnet call 
(HTTP 409). OVN wasn’t in the stack trace.

  Captured traceback:
  ~~~
  Traceback (most recent call last):
File "tempest/api/network/test_dhcp_ipv6.py", line 188, in 
test_dhcpv6_two_subnets
  subnet_slaac = self.create_subnet(self.network, **kwargs)
File "tempest/api/network/base.py", line 187, in create_subnet
  **kwargs)
File "tempest/services/network/json/network_client.py", line 110, in 
create_subnet
  return self._create_resource(uri, post_data)
File "tempest/services/network/json/network_client.py", line 72, in 
_create_resource
  resp, body = self.post(req_uri, req_post_data)
File 
"/opt/stack/new/tempest/.tox/all/local/lib/python2.7/site-packages/tempest_lib/common/rest_client.py",
 line 256, in post
  return self.request('POST', url, extra_headers, headers, body)
File 
"/opt/stack/new/tempest/.tox/all/local/lib/python2.7/site-packages/tempest_lib/common/rest_client.py",
 line 643, in request
  resp, resp_body)
File 
"/opt/stack/new/tempest/.tox/all/local/lib/python2.7/site-packages/tempest_lib/common/rest_client.py",
 line 705, in _error_checker
  raise exceptions.Conflict(resp_body)
  tempest_lib.exceptions.Conflict: An object with that identifier already 
exists
  Details: {u'detail': u'', u'type': u'IpAddressInUse', u'message': 
u'Unable to complete operation for network 
a7b73f3f-37f2-4abc-8ea0-a22291f67b4e. The IP address 2003::f816:3eff:fe31:f72e 
is in use.’}

I don’t know the neutron answer yet, I will look into that.

Amitabha

> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][networking-ovn][tempest] devstack: gate-tempest-dsvm-networking-ovn failures in Openstack CI

[Amitabha Biswas]

With the recent commits it seems that the gate-tempest-dsvm-networking-ovn 
is succeeding more or less every time. The DBDeadlock issues still are 
seen on q-svc logs but are not frequent enough to cause ovsdb failures 
that were leading to the dsvm-networking failing before.

Once in a while a test fails for e.g. 
tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_two_subnets 
that failed recently in Jenkins. But I am pretty sure it will succeed if 
the suite is re-run.

Should the gate-tempest-dsvm-networking-ovn become voting at this time, 
and re-run/re-check if it fails in the Jenkins check?

Amitabha



From:   Russell Bryant 
To: openstack-dev@lists.openstack.org
Date:   08/26/2015 06:24 AM
Subject:Re: [openstack-dev] [neutron][networking-ovn][tempest] 
devstack: gate-tempest-dsvm-networking-ovn failures in Openstack CI



On 08/25/2015 03:02 PM, Assaf Muller wrote:
> 
> 
> On Tue, Aug 25, 2015 at 2:15 PM, Russell Bryant  <mailto:rbry...@redhat.com>> wrote:
> 
> On 08/25/2015 01:26 PM, Amitabha Biswas wrote:
> > Russell suggested removing the MYSQL_DRIVER=MySQL-python 
declaration
> > from local.conf https://review.openstack.org/#/c/216413/which
> results in
> > PyMySQL as the default.
> >
> > With the above change the above DB errors are no longer seen in my 
local
> > setup,
> 
> It's great to hear that resolved the errors you saw!
> 
> >  1. Is there any impact of using PyMySQL for the Jenkins check and
> gates.
> 
> As Jeremy mentioned, this is what everything else is using (and what 
OVN
> was automatically already using in OpenStack CI).
> 
> >  2. Why is the gate-networking-ovn-python27**failing (the past
> couple of
> > commits) in {0}
> > 
networking_ovn.tests.unit.test_ovn_plugin.TestOvnPlugin.test_create_port_security
> > [0.194020s] ... FAILED. Do we need another conversation to 
track this?
> 
> This is a separate issue.  The networking-ovn git repo has been 
pretty
> quiet the last few weeks and it seems something has changed that 
made
> our tests break.  We inherit a lot of base plugin tests from 
neutron, so
> it's probably some change in Neutron that we haven't synced with 
yet.  I
> haven't had time to dig into it yet.
> 
> 
> This patch was recently merged to Neutron:
> https://review.openstack.org/#/c/201141/
> 
> Looks like that unit test is trying to create a port with an invalid MAC
> address.
> I pushed a fix here:
> https://review.openstack.org/#/c/216837/

Thanks, Assaf!  Much appreciated!  :-)

-- 
Russell Bryant

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][networking-ovn][tempest] devstack: gate-tempest-dsvm-networking-ovn failures in Openstack CI

[Amitabha Biswas]

Hello everyone,

We have been investigating the cause behind the Jenkins Check 
gate-tempest-dsvm-networking-ovn failures (non-voting at the moment). The 
failures have been happening pretty consistently with every commit. I 
wanted to start a conversation to get some input as to why these errors 
may be happening.

One kind of error is related to the following (from the q-svc logs).

2015-08-04 05:40:28.313 ERROR neutron.agent.ovsdb.impl_idl 
[req-c189268a-1e1d-462f-a81e-62f0a34ff490 
tempest-FloatingIPAdminTestJSON-1706130555 
tempest-FloatingIPAdminTestJSON-1943105894] Traceback (most recent call 
last):
  File "/opt/stack/new/neutron/neutron/agent/ovsdb/native/connection.py", 
line 84, in run
txn.results.put(txn.do_commit())
  File "/opt/stack/new/neutron/neutron/agent/ovsdb/impl_idl.py", line 99, 
in do_commit
seqno)
  File "/opt/stack/new/neutron/neutron/agent/ovsdb/native/idlutils.py", 
line 125, in wait_for_change
raise Exception("Timeout")
Exception: Timeout

When this error happens - in a separate thread there is DB Deadlock. Note 
that it's not always create_port (65%), it could be delete_port (30%), 
other calls (5%). There are many more of these errors (show below) than 
the above error.

But it is always: SQL: u'UPDATE ipavailabilityranges SET first_ip=%s WHERE 
ipavailabilityranges.allocation_pool_id = %s AND 
ipavailabilityranges.first_ip = %s AND ipavailabilityranges.last_ip = %s']

2015-08-04 05:39:37.303 9407 ERROR oslo_db.api   File 
"/usr/local/lib/python2.7/dist-packages/oslo_db/api.py", line 136, in 
wrapper
2015-08-04 05:39:37.303 9407 ERROR oslo_db.api return f(*args, 
**kwargs)
2015-08-04 05:39:37.303 9407 ERROR oslo_db.api   File 
"/opt/stack/new/networking-ovn/networking_ovn/plugin.py", line 275, in 
create_port
2015-08-04 05:39:37.303 9407 ERROR oslo_db.api db_port = 
super(OVNPlugin, self).create_port(context, port)
...
2015-08-04 05:39:37.303 9407 ERROR oslo_db.api   File 
"/usr/local/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 205, in 
execute
2015-08-04 05:39:37.303 9407 ERROR oslo_db.api self.errorhandler(self, 
exc, value)
2015-08-04 05:39:37.303 9407 ERROR oslo_db.api   File 
"/usr/local/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, 
in defaulterrorhandler
2015-08-04 05:39:37.303 9407 ERROR oslo_db.api raise errorclass, 
errorvalue
2015-08-04 05:39:37.303 9407 ERROR oslo_db.api DBDeadlock: 
(_mysql_exceptions.OperationalError) (1205, 'Lock wait timeout exceeded; 
try restarting transaction') [SQL: u'UPDATE ipavailabilityranges SET 
first_ip=%s WHERE ipavailabilityranges.allocation_pool_id = %s AND 
ipavailabilityranges.first_ip = %s AND ipavailabilityranges.last_ip = %s'] 
[parameters: ('10.100.0.3', '851466c3-8d6b-4629-bf65-86be2f403e67', 
'10.100.0.2', '10.100.0.14')]
2015-08-04 05:39:37.303 9407 ERROR oslo_db.api 

Russell suggested removing the MYSQL_DRIVER=MySQL-python declaration from 
local.conf https://review.openstack.org/#/c/216413/ which results in 
PyMySQL as the default.

With the above change the above DB errors are no longer seen in my local 
setup, the CI setup is having trouble with the 
gate-networking-ovn-python27 test now therefore the 
gate-tempest-dsvm-networking-ovn never runs.

So there are 2 questions:

Is there any impact of using PyMySQL for the Jenkins check and gates.
Why is the gate-networking-ovn-python27 failing (the past couple of 
commits) in {0} 
networking_ovn.tests.unit.test_ovn_plugin.TestOvnPlugin.test_create_port_security
 
[0.194020s] ... FAILED. Do we need another conversation to track this?

Amitabha

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev