Hi, I see interest in OVN implementing the conventional NAT/FIP architecture from the OpenStack Networking service (neutron) and think it warrants further discussion to make sure the project takes an optimal approach to it.
For those not familiar with neutron, routers between provider and private networks primarily use NAT rather than actual routing. Each router contains at least two interfaces, one on a provider network typically using publicly routable IP space and one on each private network typically using RFC1918 IP space. Instances on private networks access the provider network and other external networks via SNAT on the router. Instances that require access from external networks can use a FIP address, essentially DNAT on the router. If you disable NAT on a router, you must create a static route for each private network or an aggregate of them pointing to the router interface on the provider network... definitely not practical or scalable. Finally, the NAT/FIP architecture does not work with IPv6. Before you think I'm just crazy, I contribute significantly to the OpenStack installation and networking guides and also provide support for a variety of operators. In my opinion, many operators implemented the NAT/FIP architecture because it was the only neutron architecture with some level of documentation for a couple of years after the initial release of neutron. The NAT/FIP architecture quickly leads to performance, scaling, and reliability problems because all network traffic between private and provider networks and among private networks must traverse a single node (network node) that contains the neutron routers. Furthermore, the architecture also runs DHCP and metadata services on the network node. For many new and existing operators, the lack of documentation and plenty of horror stories gave neutron the appearance of a complex and fragile "black box"... leading to skepticism about it that continues today. Attempting to rectify these issues with DVR only made the situation worse, but I'll spare you the details for now. A couple of years ago, some OpenStack documentation contributors with networking experience including myself decided that neutron would benefit from a separate guide [1]. Most importantly, the guide would include a variety of practical deployment architectures or scenarios with operational details and functional example configurations that operators could use as a starting point. Looking at all of the scenarios [2], the classic (NAT/FIP) scenarios offer the least performance, scaling, and reliability. The provider network scenarios offer the highest performance, scaling, and reliability... mostly due to simplicity. However, these scenarios effectively disable "self-service" networking... a cornerstone of cloud computing. So, what do we do? Suggest a hybrid architecture with the following aspects: 1) Instances that require access from external networks contain a port on the provider network. 2) Instances that only require occasional access to external networks contain a port on a private network and use SNAT on a router. If an instance requires access from external networks at a later point, add a port on the provider network. 3) Instances optionally contain a port on one or more private networks to facilitate communication among instances. 4) Use a combination of multiple network nodes, clever network scheduling, and L3 HA (neutron implementation of VRRP) to increase performance, scalability, and reliability of routing among private networks and SNAT. 5) Distribute DHCP and metadata services across multiple controller or compute nodes. What's missing? Floating IP addresses! I think OVN should support the NAT/FIP architecture, but only as a temporary measure to increase adoption by operators using it. In fact, we should use the words "classic" or "legacy" around it in any documentation. For new deployments, suggest a hybrid architecture and provide ample documentation for it similar to the scenarios in the networking guide. Meanwhile, development should focus on supporting actual routing such as advertisement of private networks via one or more dynamic routing protocols. After release of this feature, deprecate and eventually remove NAT/FIP support. P.S. - If OVN decides to support the NAT/FIP architecture, I strongly recommend also offering some form of HA such as VRRP in the initial release to increase appeal to operators, most of whom became wary of neutron's inability to eliminate single points of failure for so many releases. I don't want OVN to repeat mistakes of neutron, even for a little bit. Matt [1] http://docs.openstack.org/liberty/networking-guide/ [2] http://docs.openstack.org/liberty/networking-guide/deploy.html _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
