Russell, Thanks of taking the time to provide a detailed response. Let me try and answer your questions, and ask a few on my own.
1. Yes, I want to continue working through this as we see it really important to enable easy service insertion at scale. Think of the problem of how to deploy hundreds of VNF’s that can be scaled up/down with no manual intervention. 2. The Neutron API I created was done for much the same reason as your prototype - seeing how hard it was and making sure I could get something to work. Creating and supporting yet another API is rarely a good idea and so aligning with the networking-sfc API is the right thing to do. I have had a few off-line conversations with that team and will continue to try and fit this under that umbrella as it evolves. 3. I agree with you idea of a separate table, once again I just took what I could see was the shortest path to get something working for a prototype. I will take a look at your code and see how to implement an extra table. 4. The issue with routing between subnets to chain VNF’s could be looked at in three ways, 1) If each subnet has one or more VNF’s the problem is easy; 2) If the application being protected is communicating with applications/hosts on a different subnet then the traffic is, I assume, forwarded the default router; 3) The hardest case is if the topology is such that all the VNF’s are placed in a shared subnet, then we would need to add additional information in the API to direct traffic to that subnet, or OVN could look up the subnet based on logical port? - have not thought about it a lot but seems doable. 5. The toughest question here is what is a VNF and does it operate at L3, L2, or as a transparent bump in the wire. The related question is how complex is the service chain of these virtual functions. I would disagree a little here with Russell and suggest that the approach is not for “legacy” VNF’s but transparent VNF’s that act as a “bump in the wire" from a networking POV. The issue becomes what are the specific use cases we want to design for with ovn4nfv. Let me try to start to separate/segment some of the approaches: I think there are several segments of the market for VNF’s and NFV. The two I see most are: 1. Carrier networks where they are focused on virtual provider/customer edge routers (vCPE, vPE), border network gateways (BNG) , for wired to wireless networks, and SD-WAN, which is related to vCPE/PE 2. Public/private cloud data centers and virtual enterprises, where the use cases are centered around virtual load balancing/scaling and security. The needs of both are different (IMHO), the need for the carriers is to enable the creation of large scale distributed virtual routers, leveraging MPLS and/or BGP as the core transport protocol. The goal for the carriers is to be able to customize their core routing infrastructure without waiting for a new release from their hardware vendors which can take years to get into production. Here the architecture of networking-sfc has it core strengths; with its focus on dynamic routing; as it acts similarly to the ingress/egress forwarding paths of a classical router. This enables carriers to build large scale dynamic distributed virtual routing infrastructure from virtual functions. The needs in private/public cloud (and enterprises) are simpler as the requirements are to insert either a load-balancer or security VNF to scale up/down applications or secure applications. Here change can be rapid and the need is to deploy and scale up/down in minutes if not seconds in many instances. I have been focusing on the use cases for the second segment as it is where we are seeing a need for simplicity and speed. A typical use case we see a lot of is protecting east-west traffic, when an application is deployed, security is deployed with it to protect it, and when the application is removed, security is removed. The goal here is to deploy a “zero trust” security model. We have deployed a similar architecture very successfully with VMWare’s NSX product, we would now like to figure out how to do it in the open source environment. We would prefer to be transparent to networking “bump in the wire” as this simplifies the interaction between the controller (OVN) and the VNF. The only information that is shared is the logical in and out ports for the VNF. Hence configuration requires less interactions between the networking infrastructure and the VNF/Controller, moving the VNF or application also becomes easier. It becomes a simple API interaction with minimal shared state between OVN and the VNF. Does this make sense, it is just my rough attempt at breaking down the use cases into a more specific set of cases we can solve? Also what other use cases should we consider? Regards John From: Russell Bryant <russ...@ovn.org<mailto:russ...@ovn.org>> Date: Monday, March 28, 2016 at 11:32 AM To: Ben Pfaff <b...@ovn.org<mailto:b...@ovn.org>> Cc: "discuss@openvswitch.org<mailto:discuss@openvswitch.org>" <discuss@openvswitch.org<mailto:discuss@openvswitch.org>>, John McDowall <jmcdow...@paloaltonetworks.com<mailto:jmcdow...@paloaltonetworks.com>> Subject: Re: [ovs-discuss] [ovn4nfv] On Fri, Mar 25, 2016 at 11:43 AM, Ben Pfaff <b...@ovn.org<mailto:b...@ovn.org>> wrote: Russell, you had some thoughts about SFC in OVN. I had the impression that the approach you had in mind was more powerful than what John is suggesting here, yet still not very complicated. I think you said that you weren't going forward with it for now because other things were higher priority. Do you have any comments for John? Yes, I'm sorry for being so slow to respond here. Thanks for sharing your work, John! I did indeed do a prototype. I was trying to see how difficult it would be to implement OpenStack Neutron's "networking-sfc" API. The code is here: https://github.com/russellb/ovs/commits/chaining<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_russellb_ovs_commits_chaining&d=CwMFaQ&c=V9IgWpI5PvzTw83UyHGVSoW3Uc1MFWe5J8PTfkrzVSo&r=vZ6VUDaavDpfOdPQrz1ED54jEjvAE36A8TVJroVlrOQ&m=L9bSfqoxWbBnTn5DyjKllCJiwp9D6TsewCejimUcdtM&s=ZMHRnxRksz1WOxLxFSQSgtwcxSbPrkjoYwq4048Nuug&e=> It's not documented and testing was very minimal. It added a new table to OVN_Northbound for defining port chains. > $ ovn-nbctl lport-chain-list sw0 This listed the port chains defined on the logical switch "sw0". > lport-chain b5c37a7e-ffcb-4d19-bbd4-1e253d9e17a5 > match (inport == "sw0-port8" && ip && tcp && tcp.dst == 80) This is the traffic classifier. It defined the traffic to match on and send into the chain. I'd like to see something like this included in whatever solution we go with. > lport-pair c9c80cab-dade-4f29-847d-1e1d9959927a > outport 0c103dfa-ed5e-49db-944a-538e16ed0e87 (sw0-port1-101) > inport 0c103dfa-ed5e-49db-944a-538e16ed0e87 (sw0-port1-101) > lport-pair 6e0e19b7-55fe-41b6-81e1-14a29e5b5250 > outport d43e9e3d-0303-4929-8284-dc6c6cb76cc1 (sw0-port2-101) This defined the port chain. We'd start by sending the packet to sw0-port1-101. When it came back into sw0 from sw0-port1-101, we would send it to sw0-port2-101. I'd like to support N ports in the chain. It doesn't seem like supporting an arbitrary list is much more difficult than just 1 or 2. The primary limitation was that all ports on a chain had to be on the same logical switch. It sounds like your proposal is L2 focused, as well. I figured my prototype needs to be re-done to be L3 focused, instead. I never got to figuring out exactly what that would look like. If we're forwarding it to a new L3 destination, we need to remember the original destination IP as well. This could really use a detailed proposal that explains exactly how a packet is processed at each point in the network, along with whatever limitations and caveats we accept for a first iteration. I'd like to go back and better understand how networking-sfc proposed doing all of this. I see that in your proposal, you're focused on legacy VNFs that wouldn't have any knowledge of the chaining. That means we wouldn't be concerned with chain metadata in and out of VNFs. Maybe that's an easier place to start. I had proposed using nested ports as a way to identify which chain a packet is associated with. It's identical to how we define container ports that reside in a VM. It's a single VIF with many child ports. Each child port has a VLAN ID associated with it. That VLAN ID is the additonal piece of context we need. I added code in my branch to support MPLS for this as well, since networking-sfc was doing that. You mentioned hooking in at table 3. In current code, that's the logical switch pre-ACL table. I was thinking a new table after ACLs made sense. That way some traffic could be filtered out using ACLs and only hit a service chain after that point for deeper analysis if needed. You mention a new, custom REST API in OpenStack for this. I'd like to avoid that as best we can. The networking-sfc project is the closest thing to what we need. I'd like to see if we can use that, or help evolve it to something that would work for OVN. Hopefully that's helpful. I do still think this is something we should work out for OVN, so I'd be happy to continue to provide feedback if you're willing to keep working on it. Thanks, -- Russell Bryant On Mon, Mar 21, 2016 at 09:16:18PM +0000, John McDowall wrote: > As a VNF vendor we have been looking at ways to enable customers to simply > scale up (and down) VNF’s in complex virtual networks at scale. Our goal > is to help accelerate the deployment of SDN and VNF’s and more > specifically enable zero-trust security at scale for applications. This > requires the easy and fast deployment of Next Generation Firewalls (and > other VNF¹s) into the traffic path of any application. > > Over the last several weeks we have created a prototype that implements > a simple VNF insertion approach. Before we do additional work we have a > couple of questions for the community: > > Questions > ‹‹‹‹‹‹‹‹‹ > > 1. This approach has the advantage of being very simple and works with > existing VNF¹s, does it make sense to the community? > 2. If it is of interest how could it be improved and or enhanced to make > it more useful and consumable? > > Design Guidelines > ‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹ > > At the start of the effort we created a set of design guidelines to > constrain the problem space. > > > * The goal is a Service Function Insertion (SFI) approach that is simpler > and easier to deploy than Service Function Chaining and is more applicable > to single function insertion or very short chains. > * The initial design target is DC/Enterprises where the requirements are > typically for insertion of a limited set of VNF¹s in specific network > locations to act on specific applications. > * Minimal changes to existing VNF, ours and others, > * Make the solution open to all VNF¹s. > * Leverage bump in the wire connectivity as this does not require L2 or L3 > knowledge/configuration in the VNF. > * Firewalls want to inspect/classify all traffic on a link, so > pre-classifing traffic beyond ACL¹s is not necessary. > * Deploy on standard infrastructure; Openstack and Open vSwitch with > minimal changes > * Work with virtualization and containers and physical devices seamlessly. > * Insert and remove security is seconds, one of the drivers of the > requirement for speed is container deployment > * Simple to deploy and easy to debug is important - atomic insertion and > removal of VNF is an important aspect of this. > > > Approach > ‹‹‹‹‹‹‹‹ > > We have developed a prototype, roughly using the ovn4nfv model proposed by > Vikram Dham and others in OPNFV. The implemented prototype of ovn4nfv is > on OpenVSwitch 2.5 and Openstack Mitaka (development branch). I would like > to stress this is a prototype and not production code. My objective was to > prove to myself (and others) that the concept would work and then ask for > feedback from the community on level of interest and how best to design a > production implementation. > > I have called this effort service function insertion (SFI) to > differentiate from service function chaining (SFC). This approach is > simpler than SFC and requires minimal or no changes to existing VNF¹s that > act as a bump in the wire, but it will probably not handle long complex > chains or graphs. It can possibly handle chaining one or two VNF¹s in a > static manner, but not sure if it could go beyond that. I am open to > suggestions of how to extend/improve it. > > The traffic steering is implemented by inserting 2 ingress and 2 egress > rules in the ovn-nb pipeline at ingress stage 3. These rules have a higher > priority than the default rules. The changes to OVN and rules are listed > in the implementation section. > > The control plane is implemented in both Open vSwitch and in Openstack. In > Openstack there is a set of extension interfaces added to the > networking-ovn plugin. There are both CLI and REST API¹s provided for > Openstack and CLI for Open vSwitch. > > The OVN model enables logical changes to the flow rules and Openstack > neutron plugin model allows separation of changes to extensions to the > networking-OVN plugin. I have however violated a few boundaries for > expediency that would need to be fixed before > this could be easily deployed. > > We are happy to contribute the code back to the community, but would like > to gauge the level on interest and solicit feedback on the approach. We > are open to any and all suggestions for improvements in both > implementation and approach. > > Below I have given a rough overview of the implementation and the changes > I have made to the various code bases. Just to re-iterate this was done as > a quick prototype and was a learning experience in Openstack and Open > vSwitch on the way so the quality > of the code and architecture are not production ready. Links are provide > to my github repositories with the changes. > > Implementation > ‹‹‹‹‹‹‹‹‹‹‹‹‹‹ > > The approach atomically inserts rules into ovn-nb to intercept any traffic > going to or coming from a vm or a container, requires insertion of four > new rules. There are no other changes in Open vSwitch. While the prototype > is using a firewall as the VNF there is no requirement for that to be the > case any VNF that supports ³bump in the wire mode² should work. > > App-1 is the application that needs to be protected > FW-2 is the input port of the firewall > FW-1 is the egress port of the firewall > App-2 is an application talking to App-1 > > The rules look like this: > > Ingress Rules [App-1] > > * If dst-ip = app-1 and src-mac = fw-2-mac then dst = app-1 [Priority 250] > * If dst-ip = app-1 then dst = fw-2 [Priority 200] > > Egress Rules [App-1] > > * If src-ip = app-1 and src-mac = app-1 then dst = fw-1 [Priority 250] > * If src-ip = app-1 and src-mac = fw-1 then dst= app-2 [Priority 200] > > > This enables the deployment of a zero trust model of security for any > application. We would envision that the deployment of an application would > be "wrapped" with security to ensure that security is consistently and > always deployed. > > Open vSwitch Changes > ‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹ > > > In OVN the interface looks like: > > $ ovn-nbctl lservice-add LSWITCH, LAPP-PORT, LSERVICE-PORT-IN, > LSERVICE-PORT-OUT [NAME] > $ ovn-nbctl lservice-list LSWITCH > $ ovn-nbctl lservice-del NAME > > The prototype is tailored to inserting a Next Generation Firewall (such as > Palo Alto¹s implementation). However I have made no changes to our > Firewall to support ovn4nfv. The solution will work with any ³bump in the > wire² VNF. The prototype assumes that the > VNF has two data plane interfaces but the solution would work with a > single interface, if the ³add² operation only has one interface the rules > would be changed slightly internally to support an VNF with a single port. > > The only changes I have made to OVN are as follows: > > > * ovn-nb.ovsschema: Added a new table to support lservice > * ovn-nb.xml: To document the schema > * ovn-nbctl.c: Added CLI commands for lservice > * ovn-northd.c: Added new rules into stage 3 of ingress pipeline with > higher priority than existing rules. > > Personal github repo: > https://github.com/doonhammer/ovs<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_doonhammer_ovs&d=CwMFaQ&c=V9IgWpI5PvzTw83UyHGVSoW3Uc1MFWe5J8PTfkrzVSo&r=vZ6VUDaavDpfOdPQrz1ED54jEjvAE36A8TVJroVlrOQ&m=L9bSfqoxWbBnTn5DyjKllCJiwp9D6TsewCejimUcdtM&s=0ORzckDN-Xbjfo_7AkaFFyjEjUcw_UamRLZSpym89Wk&e=> > > I have done simple testing and can insert a firewall dynamically in front > of any application with no changes to application or infrastructure. This > enables the deployment of a L7 zero-trust security model with minimal > effort. > > Openstack Changes > ‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹ > > In Openstack the interface looks like: > > $ sfi-add LSWITCH, LAPP-PORT, LSERVICE-PORT-IN, LSERVICE-PORT-OUT [NAME] > $ sfi-list LSWITCH > $ sfi-del NAME > > In Openstack I have had to make slightly more changes, but this is > probably due to my lack of familiarity with Openstack. I did try and > follow the neutron plugin/extension model, but with more on making it work > rather than pure modularity. The following are > the areas where I modified Openstack: > > * networking-ovn: Added the extensions to support service function > insertion as an extension to the networking-ovn plugin. > Personal github repo: > https://github.com/doonhammer/networking-ovn<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_doonhammer_networking-2Dovn&d=CwMFaQ&c=V9IgWpI5PvzTw83UyHGVSoW3Uc1MFWe5J8PTfkrzVSo&r=vZ6VUDaavDpfOdPQrz1ED54jEjvAE36A8TVJroVlrOQ&m=L9bSfqoxWbBnTn5DyjKllCJiwp9D6TsewCejimUcdtM&s=v_L7L6H2X1Kyb6gpyH23t7iZtU28-0Wes-e8sFEju5o&e=> > > * python-neutronclient: Added the cli and REST interfaces to support > service function insertion. > Personal github repo: > https://github.com/doonhammer/python-neutronclient<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_doonhammer_python-2Dneutronclient&d=CwMFaQ&c=V9IgWpI5PvzTw83UyHGVSoW3Uc1MFWe5J8PTfkrzVSo&r=vZ6VUDaavDpfOdPQrz1ED54jEjvAE36A8TVJroVlrOQ&m=L9bSfqoxWbBnTn5DyjKllCJiwp9D6TsewCejimUcdtM&s=6ep_MWkfajb_OXLuA17foLMig7pcbz-iIu9wevBh-jw&e=> > > * neutron: Added new db table to support service function insertion > Personal github repo: > https://github.com/doonhammer/neutron<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_doonhammer_neutron&d=CwMFaQ&c=V9IgWpI5PvzTw83UyHGVSoW3Uc1MFWe5J8PTfkrzVSo&r=vZ6VUDaavDpfOdPQrz1ED54jEjvAE36A8TVJroVlrOQ&m=L9bSfqoxWbBnTn5DyjKllCJiwp9D6TsewCejimUcdtM&s=WoftbSYE5rtrgNp1hNq3_gnM9fb2WZcLd3qkpfTz5yQ&e=> > > * neutron and nova: Added changes to enable creation of L2 only ports > Personal github repo: > https://github.com/doonhammer/nova<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_doonhammer_nova&d=CwMFaQ&c=V9IgWpI5PvzTw83UyHGVSoW3Uc1MFWe5J8PTfkrzVSo&r=vZ6VUDaavDpfOdPQrz1ED54jEjvAE36A8TVJroVlrOQ&m=L9bSfqoxWbBnTn5DyjKllCJiwp9D6TsewCejimUcdtM&s=hUlwSR2GqNv3zZtS97ak_t8t7qOth2SsiQObwmE4jVU&e=> > > Known Issues > ‹‹‹‹‹‹‹‹‹‹‹‹ > > 1. To make the prototype work I had to manipulate the src and dst mac > addresses in the actions, if I could filter by switch ports this would not > be necessary. > 2. Need to add the VNF mac addresses to application ports for port > security - if 1 could be fixed this would not be necessary. > 3. Have not tested scale and performance so impact of adding new rules has > not been determined. > > > [Palo Alto Networks Ignite 2016]<http://go.paloaltonetworks.com/ignitereg> > _______________________________________________ > discuss mailing list > discuss@openvswitch.org<mailto:discuss@openvswitch.org> > http://openvswitch.org/mailman/listinfo/discuss<https://urldefense.proofpoint.com/v2/url?u=http-3A__openvswitch.org_mailman_listinfo_discuss&d=CwMFaQ&c=V9IgWpI5PvzTw83UyHGVSoW3Uc1MFWe5J8PTfkrzVSo&r=vZ6VUDaavDpfOdPQrz1ED54jEjvAE36A8TVJroVlrOQ&m=L9bSfqoxWbBnTn5DyjKllCJiwp9D6TsewCejimUcdtM&s=_ksXPh5JOVh9fxSAXGDQlB4J5sDZFFmR37NGkEgm8Cg&e=> -- Russell Bryant [Palo Alto Networks Ignite 2016]<http://go.paloaltonetworks.com/ignitereg>
_______________________________________________ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
