Re: [openstack-dev] [Neutron][ml2] Too much "shim rest proxy" mechanism drivers in ML2
On 10 June 2014 18:18, Irena Berezovsky wrote: > Hi Luke, > > Very impressive solution! > Thanks for the kind words! > > > I do not think there is a problem to keep agent out of the tree in a short > term, but would highly recommend to put it upstream in a longer term. > > You will benefit from quite valuable community review. And most important > it will allow to keep your code as much as possible aligned with neutron > code base. Once there are some general changes done by other people, your > code will be taken into account and won’t be broken accidentally. > Yes, I agree. We do want to be upstream going forward. For the Juno cycle our modest goal is that users will be able to deploy our software with an unmodified OpenStack Juno. On the OpenStack side this only requires a simple VIF type (Nova) and ML2 mech driver (Neturon). Behind the scenes we are doing a lot of work together with the upstream QEMU community to make this possible. Hopefully we will be able to expand our upstream OpenStack ambitions in the K-cycle. Looking forward to setting the next goals in Paris :-). Currently our project is relatively new and unknown, but I hope there will be a lot of involvement within the OpenStack community going forward. Our goal is to create a practical NFV solution that is 100% open source and we hope this will be useful to a lot of people. (http://snabb.co/nfv.html) P.S. Thanks for the reminder about modular ML2 agent! Cheers, -Luke ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ml2] Too much "shim rest proxy" mechanism drivers in ML2
Hi Luke,
Very impressive solution!
I do not think there is a problem to keep agent out of the tree in a short
term, but would highly recommend to put it upstream in a longer term.
You will benefit from quite valuable community review. And most important it
will allow to keep your code as much as possible aligned with neutron code
base. Once there are some general changes done by other people, your code will
be taken into account and won’t be broken accidentally.
I would like to mention that there is Modular L2 Agent initiative driven by ML2
team, you may be interested to follow:
https://etherpad.openstack.org/p/modular-l2-agent-outline
Best Regards,
Irena
From: luk...@gmail.com [mailto:luk...@gmail.com] On Behalf Of Luke Gorrie
Sent: Tuesday, June 10, 2014 12:48 PM
To: Irena Berezovsky
Cc: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Neutron][ml2] Too much "shim rest proxy"
mechanism drivers in ML2
Hi Irena,
Thanks for the very interesting perspective!
On 10 June 2014 10:57, Irena Berezovsky
mailto:ire...@mellanox.com>> wrote:
[IrenaB] The DB access approach was previously used by OVS and LinuxBridge
Agents and at some point (~Grizzly Release) was changed to use RPC
communication.
That is very interesting. I've been involved in OpenStack since the Havana
cycle and was not familiar with the old design.
I'm optimistic about the scalability of our implementation. We have
sanity-tested with 300 compute nodes and a 300ms sync interval. I am sure we
will find some parts that we need to spend optimization energy on, however.
The other scalability aspect we are being careful of is the cost of individual
update operations. (In LinuxBridge that would be the iptables, ebtables, etc
commands.) In our implementation the compute nodes preprocess the Neutron
config into a small config file for the local traffic plane and then load that
in one atomic operation ("SIGHUP" style). Again, I am sure we will find cases
that we need to spend optimization effort on, but the design seems scalable to
me thanks to the atomicity.
For concreteness, here is the agent we are running on the DB node to make the
Neutron config available:
https://github.com/SnabbCo/snabbswitch/blob/master/src/designs/neutron/neutron-sync-master
and here is the agent that pulls it onto the compute node:
https://github.com/SnabbCo/snabbswitch/blob/master/src/designs/neutron/neutron-sync-agent
TL;DR we snapshot the config with mysqldump and distribute it with git.
Here's the sanity test I referred to:
https://groups.google.com/d/msg/snabb-devel/blmDuCgoknc/PP_oMgopiB4J
I will be glad to report on our experience and what we change based on our
deployment experience during the Juno cycle.
[IrenaB] I think that for “Non SDN Controller” Mechanism Drivers there will be
need for some sort of agent to handle port update events even though it might
not be required in order to bind the port.
True. Indeed, we do have an agent running on the compute host, and it we are
synchronizing it with port updates based on the mechanism described above.
Really what I mean is: Can we keep our agent out-of-tree and apart from ML2 and
decide for ourselves how to keep it synchronized (instead of using the MQ)? Is
there a precedent for doing things this way in an ML2 mech driver (e.g. one of
the SDNs)?
Cheers!
-Luke
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ml2] Too much "shim rest proxy" mechanism drivers in ML2
Hi Irena,
Thanks for the very interesting perspective!
On 10 June 2014 10:57, Irena Berezovsky wrote:
> *[IrenaB] The DB access approach was previously used by OVS and
> LinuxBridge Agents and at some point (~Grizzly Release) was changed to use
> RPC communication.*
>
That is very interesting. I've been involved in OpenStack since the Havana
cycle and was not familiar with the old design.
I'm optimistic about the scalability of our implementation. We have
sanity-tested with 300 compute nodes and a 300ms sync interval. I am sure
we will find some parts that we need to spend optimization energy on,
however.
The other scalability aspect we are being careful of is the cost of
individual update operations. (In LinuxBridge that would be the iptables,
ebtables, etc commands.) In our implementation the compute nodes preprocess
the Neutron config into a small config file for the local traffic plane and
then load that in one atomic operation ("SIGHUP" style). Again, I am sure
we will find cases that we need to spend optimization effort on, but the
design seems scalable to me thanks to the atomicity.
For concreteness, here is the agent we are running on the DB node to make
the Neutron config available:
https://github.com/SnabbCo/snabbswitch/blob/master/src/designs/neutron/neutron-sync-master
and here is the agent that pulls it onto the compute node:
https://github.com/SnabbCo/snabbswitch/blob/master/src/designs/neutron/neutron-sync-agent
TL;DR we snapshot the config with mysqldump and distribute it with git.
Here's the sanity test I referred to:
https://groups.google.com/d/msg/snabb-devel/blmDuCgoknc/PP_oMgopiB4J
I will be glad to report on our experience and what we change based on our
deployment experience during the Juno cycle.
*[IrenaB] I think that for “Non SDN Controller” Mechanism Drivers there
> will be need for some sort of agent to handle port update events even
> though it might not be required in order to bind the port.*
>
True. Indeed, we do have an agent running on the compute host, and it we
are synchronizing it with port updates based on the mechanism described
above.
Really what I mean is: Can we keep our agent out-of-tree and apart from ML2
and decide for ourselves how to keep it synchronized (instead of using the
MQ)? Is there a precedent for doing things this way in an ML2 mech driver
(e.g. one of the SDNs)?
Cheers!
-Luke
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ml2] Too much "shim rest proxy" mechanism drivers in ML2
Hi Luke, Please see my comments inline. BR, Irena From: Luke Gorrie [mailto:l...@tail-f.com] Sent: Monday, June 09, 2014 12:29 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][ml2] Too much "shim rest proxy" mechanism drivers in ML2 On 6 June 2014 10:17, henry hly mailto:henry4...@gmail.com>> wrote: ML2 mechanism drivers are becoming another kind of "plugins". Although they can be loaded together, but can not work with each other. [...] Could we remove all device related adaption(rest/ssh/netconf/of... proxy) from these mechanism driver to the agent side, leaving only necessary code in the plugin? In the Snabb NFV mech driver [*] we are trying a design that you might find interesting. We stripped the mech driver down to bare bones and declared that the agent has to access the Neutron configuration independently. In practice this means that our out-of-tree agent is connecting to Neutron's MySQL database directly instead of being fed config changes by custom sync code in ML2. This means there are very little work for the mech driver to do (in our case check configuration and perform special port binding). [IrenaB] The DB access approach was previously used by OVS and LinuxBridge Agents and at some point (~Grizzly Release) was changed to use RPC communication. You can see the reasons and design details covered by [1] and the patch itself in [2] [1]: https://docs.google.com/document/d/1MbcBA2Os4b98ybdgAw2qe_68R1NG6KMh8zdZKgOlpvg/edit?pli=1 [2] https://review.openstack.org/#/c/9591/ We are also trying to avoid running an OVS/LinuxBridge-style agent on the compute hosts in order to keep the code footprint small. I hope we will succeed -- I'd love to hear if somebody else is running "agent-less"? Currently we depend on a really ugly workaround to make VIF binding succeed and we are looking for a clean alternative: https://github.com/lukego/neutron/commit/31d6d0657aeae9fd97a63e4d53da34fb86be92f7 [IrenaB] I think that for “Non SDN Controller” Mechanism Drivers there will be need for some sort of agent to handle port update events even though it might not be required in order to bind the port. [*] Snabb NFV mech driver code: https://review.openstack.org/95711 Cheers, -Luke ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ml2] Too much "shim rest proxy" mechanism drivers in ML2
On 6 June 2014 10:17, henry hly wrote: > ML2 mechanism drivers are becoming another kind of "plugins". Although > they can be loaded together, but can not work with each other. > [...] > Could we remove all device related adaption(rest/ssh/netconf/of... proxy) > from these mechanism driver to the agent side, leaving only necessary code > in the plugin? > In the Snabb NFV mech driver [*] we are trying a design that you might find interesting. We stripped the mech driver down to bare bones and declared that the agent has to access the Neutron configuration independently. In practice this means that our out-of-tree agent is connecting to Neutron's MySQL database directly instead of being fed config changes by custom sync code in ML2. This means there are very little work for the mech driver to do (in our case check configuration and perform special port binding). We are also trying to avoid running an OVS/LinuxBridge-style agent on the compute hosts in order to keep the code footprint small. I hope we will succeed -- I'd love to hear if somebody else is running "agent-less"? Currently we depend on a really ugly workaround to make VIF binding succeed and we are looking for a clean alternative: https://github.com/lukego/neutron/commit/31d6d0657aeae9fd97a63e4d53da34fb86be92f7 [*] Snabb NFV mech driver code: https://review.openstack.org/95711 Cheers, -Luke ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
