Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
Hi, sorry for the late reply, I was out of office for 3 weeks.
I also love the idea of having a single thread in charge of writing
dataplane actions.
As Zang described, this thread would read events in a queue, which could be
populated by agent_drivers.
The main goal would be to avoid desynchronization. While a first
greenthread can yield while performing a dataplane action (for instance,
running a ofctl command) another greenthread could potentially process
another action that should be done after the first one is terminated.
enqueuing those actions and processing them in a single thread would avoid
such a behavior.
I also think this is orthogonal with the agent/resource architecture. I
think that agent driver could populate the queue, and the singleton thread
would call the correct resource driver, depending on the impacted port, to
interpret the order filed in the queue.
regards
Mathieu
On Fri, Jun 20, 2014 at 10:38 PM, Mohammad Banikazemi m...@us.ibm.com wrote:
Zang, thanks for your comments.
I think what you are suggesting is perhaps orthogonal to having Resource
and Agent drivers. By that I mean we can have what you are suggesting and
keep the Resource and Agent drivers. The reason for having Resource drivers
is to provide the means for possibly extending what an agent does in
response to say changes to a port in a modular way. We can restrict the
access to Resource drivers from the events loop only. That restriction is
not there in the current model but would adding that address your concerns?
What are your thoughts? As Salvatore has mentioned in his email in this
thread, that is what the current OVS agent does wrt port updates. That is,
the update to ports get processed from the events loop.
As a separate but relevant issue, we can and should discuss whether having
the Resource and Agent drivers is useful in making the agent more modular.
The idea behind using these drivers is to have the agent use a collection
of drivers rather than mixin classes so we can more easily select what
(and how) functionalities an agent support and reuse as much as we can
across L2 agents. Are there better ways of achieving this? Any thoughts?
Best,
Mohammad
[image: Inactive hide details for Zang MingJie ---06/19/2014 06:27:31
AM---Hi: I don't like the idea of ResourceDriver and AgentDriver.]Zang
MingJie ---06/19/2014 06:27:31 AM---Hi: I don't like the idea of
ResourceDriver and AgentDriver. I suggested
From: Zang MingJie zealot0...@gmail.com
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org,
Date: 06/19/2014 06:27 AM
Subject: Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
--
Hi:
I don't like the idea of ResourceDriver and AgentDriver. I suggested
use a singleton worker thread to manager all underlying setup, so the
driver should do nothing other than fire a update event to the worker.
The worker thread may looks like this one:
# the only variable store all local state which survives between
different events, including lvm, fdb or whatever
state = {}
# loop forever
while True:
event = ev_queue.pop()
if not event:
sleep() # may be interrupted when new event comes
continue
origin_state = state
new_state = event.merge_state(state)
if event.is_ovsdb_changed():
if event.is_tunnel_changed():
setup_tunnel(new_state, old_state, event)
if event.is_port_tags_changed():
setup_port_tags(new_state, old_state, event)
if event.is_flow_changed():
if event.is_flow_table_1_changed():
setup_flow_table_1(new_state, old_state, event)
if event.is_flow_table_2_changed():
setup_flow_table_2(new_state, old_state, event)
if event.is_flow_table_3_changed():
setup_flow_table_3(new_state, old_state, event)
if event.is_flow_table_4_changed():
setup_flow_table_4(new_state, old_state, event)
if event.is_iptable_changed():
if event.is_iptable_nat_changed():
setup_iptable_nat(new_state, old_state, event)
if event.is_iptable_filter_changed():
setup_iptable_filter(new_state, old_state, event)
state = new_state
when any part has been changed by a event, the corresponding setup_xxx
function rebuild the whole part, then use the restore like
`iptables-restore` or `ovs-ofctl replace-flows` to reset the whole
part.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http
Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
Zang, thanks for your comments.
I think what you are suggesting is perhaps orthogonal to having Resource
and Agent drivers. By that I mean we can have what you are suggesting and
keep the Resource and Agent drivers. The reason for having Resource drivers
is to provide the means for possibly extending what an agent does in
response to say changes to a port in a modular way. We can restrict the
access to Resource drivers from the events loop only. That restriction is
not there in the current model but would adding that address your concerns?
What are your thoughts? As Salvatore has mentioned in his email in this
thread, that is what the current OVS agent does wrt port updates. That is,
the update to ports get processed from the events loop.
As a separate but relevant issue, we can and should discuss whether having
the Resource and Agent drivers is useful in making the agent more modular.
The idea behind using these drivers is to have the agent use a collection
of drivers rather than mixin classes so we can more easily select what
(and how) functionalities an agent support and reuse as much as we can
across L2 agents. Are there better ways of achieving this? Any thoughts?
Best,
Mohammad
From: Zang MingJie zealot0...@gmail.com
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org,
Date: 06/19/2014 06:27 AM
Subject:Re: [openstack-dev] [Neutron][ML2] Modular L2 agent
architecture
Hi:
I don't like the idea of ResourceDriver and AgentDriver. I suggested
use a singleton worker thread to manager all underlying setup, so the
driver should do nothing other than fire a update event to the worker.
The worker thread may looks like this one:
# the only variable store all local state which survives between
different events, including lvm, fdb or whatever
state = {}
# loop forever
while True:
event = ev_queue.pop()
if not event:
sleep() # may be interrupted when new event comes
continue
origin_state = state
new_state = event.merge_state(state)
if event.is_ovsdb_changed():
if event.is_tunnel_changed():
setup_tunnel(new_state, old_state, event)
if event.is_port_tags_changed():
setup_port_tags(new_state, old_state, event)
if event.is_flow_changed():
if event.is_flow_table_1_changed():
setup_flow_table_1(new_state, old_state, event)
if event.is_flow_table_2_changed():
setup_flow_table_2(new_state, old_state, event)
if event.is_flow_table_3_changed():
setup_flow_table_3(new_state, old_state, event)
if event.is_flow_table_4_changed():
setup_flow_table_4(new_state, old_state, event)
if event.is_iptable_changed():
if event.is_iptable_nat_changed():
setup_iptable_nat(new_state, old_state, event)
if event.is_iptable_filter_changed():
setup_iptable_filter(new_state, old_state, event)
state = new_state
when any part has been changed by a event, the corresponding setup_xxx
function rebuild the whole part, then use the restore like
`iptables-restore` or `ovs-ofctl replace-flows` to reset the whole
part.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
OVS agent manipulate not only ovs flow table, but also linux stack, which is not so easily replaced by pure openflow controller today. fastpath-slowpath separation sounds good, but really a nightmare for high concurrent connection application if we set L4 flow into OVS (in our testing, vswitchd daemon always stop working in this case). Someday when OVS can do all the L2-L4 rules in the kernel without bothering userspace classifier, pure OF controller can replace agent based solution then. OVS hooking to netfilter conntrack may come this year, but not enough yet. On Wed, Jun 18, 2014 at 12:56 AM, Armando M. arma...@gmail.com wrote: just a provocative thought: If we used the ovsdb connection instead, do we really need an L2 agent :P? On 17 June 2014 18:38, Kyle Mestery mest...@noironetworks.com wrote: Another area of improvement for the agent would be to move away from executing CLIs for port commands and instead use OVSDB. Terry Wilson and I talked about this, and re-writing ovs_lib to use an OVSDB connection instead of the CLI methods would be a huge improvement here. I'm not sure if Terry was going to move forward with this, but I'd be in favor of this for Juno if he or someone else wants to move in this direction. Thanks, Kyle On Tue, Jun 17, 2014 at 11:24 AM, Salvatore Orlando sorla...@nicira.com wrote: We've started doing this in a slightly more reasonable way for icehouse. What we've done is: - remove unnecessary notification from the server - process all port-related events, either trigger via RPC or via monitor in one place Obviously there is always a lot of room for improvement, and I agree something along the lines of what Zang suggests would be more maintainable and ensure faster event processing as well as making it easier to have some form of reliability on event processing. I was considering doing something for the ovs-agent again in Juno, but since we've moving towards a unified agent, I think any new big ticket should address this effort. Salvatore On 17 June 2014 13:31, Zang MingJie zealot0...@gmail.com wrote: Hi: Awesome! Currently we are suffering lots of bugs in ovs-agent, also intent to rebuild a more stable flexible agent. Taking the experience of ovs-agent bugs, I think the concurrency problem is also a very important problem, the agent gets lots of event from different greenlets, the rpc, the ovs monitor or the main loop. I'd suggest to serialize all event to a queue, then process events in a dedicated thread. The thread check the events one by one ordered, and resolve what has been changed, then apply the corresponding changes. If there is any error occurred in the thread, discard the current processing event, do a fresh start event, which reset everything, then apply the correct settings. The threading model is so important and may prevent tons of bugs in the future development, we should describe it clearly in the architecture On Wed, Jun 11, 2014 at 4:19 AM, Mohammad Banikazemi m...@us.ibm.com wrote: Following the discussions in the ML2 subgroup weekly meetings, I have added more information on the etherpad [1] describing the proposed architecture for modular L2 agents. I have also posted some code fragments at [2] sketching the implementation of the proposed architecture. Please have a look when you get a chance and let us know if you have any comments. [1] https://etherpad.openstack.org/p/modular-l2-agent-outline [2] https://review.openstack.org/#/c/99187/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
Hi:
I don't like the idea of ResourceDriver and AgentDriver. I suggested
use a singleton worker thread to manager all underlying setup, so the
driver should do nothing other than fire a update event to the worker.
The worker thread may looks like this one:
# the only variable store all local state which survives between
different events, including lvm, fdb or whatever
state = {}
# loop forever
while True:
event = ev_queue.pop()
if not event:
sleep() # may be interrupted when new event comes
continue
origin_state = state
new_state = event.merge_state(state)
if event.is_ovsdb_changed():
if event.is_tunnel_changed():
setup_tunnel(new_state, old_state, event)
if event.is_port_tags_changed():
setup_port_tags(new_state, old_state, event)
if event.is_flow_changed():
if event.is_flow_table_1_changed():
setup_flow_table_1(new_state, old_state, event)
if event.is_flow_table_2_changed():
setup_flow_table_2(new_state, old_state, event)
if event.is_flow_table_3_changed():
setup_flow_table_3(new_state, old_state, event)
if event.is_flow_table_4_changed():
setup_flow_table_4(new_state, old_state, event)
if event.is_iptable_changed():
if event.is_iptable_nat_changed():
setup_iptable_nat(new_state, old_state, event)
if event.is_iptable_filter_changed():
setup_iptable_filter(new_state, old_state, event)
state = new_state
when any part has been changed by a event, the corresponding setup_xxx
function rebuild the whole part, then use the restore like
`iptables-restore` or `ovs-ofctl replace-flows` to reset the whole
part.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
- Original Message - What's the progress by Terry Wilson? If not much, I'm willing to file blueprint/spec and drive it. thanks, I've been working on some proof-of-concept code to help flesh out ideas for writing the spec. I'd talked to Maru and he mentioned that he didn't think that the official OVS python library was a good base for this (the one that ryu uses). I don't remember what all of the reasons were, though. It isn't particularly well documented, but that can always be remedied. Does anyone else have any experience with the official OVS python API who could speak to its quality/stability/usefulness? It looked fairly full-featured. Terry On Wed, Jun 18, 2014 at 07:00:59PM +0900, Isaku Yamahata isaku.yamah...@gmail.com wrote: Hi. Ryu provides ovs_vsctl.py library which is python equivalent to ovs-vsctl command. It speaks OVSDB protocl. https://github.com/osrg/ryu/blob/master/ryu/lib/ovs/vsctl.py So with the library, it's mostly mechanical change to convert ovs_lib.py, I think. I'm not aware other similar library written in python. Most of ryu's library is implemented on top of the official OVS python stuff: https://github.com/openvswitch/ovs/tree/master/python/ovs Terry ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
On Thu, Jun 19, 2014 at 11:05:53AM -0400, Terry Wilson twil...@redhat.com wrote: - Original Message - What's the progress by Terry Wilson? If not much, I'm willing to file blueprint/spec and drive it. thanks, I've been working on some proof-of-concept code to help flesh out ideas for writing the spec. I'd talked to Maru and he mentioned that he didn't think that the official OVS python library was a good base for this (the one that ryu uses). I don't remember what all of the reasons were, though. It isn't particularly well documented, but that can always be remedied. Does anyone else have any experience with the official OVS python API who could speak to its quality/stability/usefulness? It looked fairly full-featured. Interesting. Can you share the technical reason? Maru? I'm curious. At least I fixed the issues I hit when I wrote ovs_vsctl.py. Thanks, Terry On Wed, Jun 18, 2014 at 07:00:59PM +0900, Isaku Yamahata isaku.yamah...@gmail.com wrote: Hi. Ryu provides ovs_vsctl.py library which is python equivalent to ovs-vsctl command. It speaks OVSDB protocl. https://github.com/osrg/ryu/blob/master/ryu/lib/ovs/vsctl.py So with the library, it's mostly mechanical change to convert ovs_lib.py, I think. I'm not aware other similar library written in python. Most of ryu's library is implemented on top of the official OVS python stuff: https://github.com/openvswitch/ovs/tree/master/python/ovs Terry ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Isaku Yamahata isaku.yamah...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
No. ovs_lib invokes both ovs-vsctl and ovs-ofctl. ovs-vsctl speaks OVSDB protocol, ovs-ofctl speaks OF-wire. thanks, On Tue, Jun 17, 2014 at 01:25:59PM -0500, Kyle Mestery mest...@noironetworks.com wrote: I don't think so. Once we implement the OVSDB support, we will deprecate using the CLI commands in ovs_lib. On Tue, Jun 17, 2014 at 12:50 PM, racha ben...@gmail.com wrote: Hi, Does it make sense also to have the choice between ovs-ofctl CLI and a direct OF1.3 connection too in the ovs-agent? Best Regards, Racha On Tue, Jun 17, 2014 at 10:25 AM, Narasimhan, Vivekanandan vivekanandan.narasim...@hp.com wrote: Managing the ports and plumbing logic is today driven by L2 Agent, with little assistance from controller. If we plan to move that functionality to the controller, the controller has to be more heavy weight (both hardware and software) since it has to do the job of L2 Agent for all the compute servers in the cloud. , We need to re-verify all scale numbers for the controller on POC’ing of such a change. That said, replacing CLI with direct OVSDB calls in the L2 Agent is certainly a good direction. Today, OVS Agent invokes flow calls of OVS-Lib but has no idea (or processing) to follow up on success or failure of such invocations. Nor there is certain guarantee that all such flow invocations would be executed by the third-process fired by OVS-Lib to execute CLI. When we transition to OVSDB calls which are more programmatic in nature, we can enhance the Flow API (OVS-Lib) to provide more fine grained errors/return codes (or content) and ovs-agent (and even other components) can act on such return state more intelligently/appropriately. -- Thanks, Vivek From: Armando M. [mailto:arma...@gmail.com] Sent: Tuesday, June 17, 2014 10:26 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture just a provocative thought: If we used the ovsdb connection instead, do we really need an L2 agent :P? On 17 June 2014 18:38, Kyle Mestery mest...@noironetworks.com wrote: Another area of improvement for the agent would be to move away from executing CLIs for port commands and instead use OVSDB. Terry Wilson and I talked about this, and re-writing ovs_lib to use an OVSDB connection instead of the CLI methods would be a huge improvement here. I'm not sure if Terry was going to move forward with this, but I'd be in favor of this for Juno if he or someone else wants to move in this direction. Thanks, Kyle On Tue, Jun 17, 2014 at 11:24 AM, Salvatore Orlando sorla...@nicira.com wrote: We've started doing this in a slightly more reasonable way for icehouse. What we've done is: - remove unnecessary notification from the server - process all port-related events, either trigger via RPC or via monitor in one place Obviously there is always a lot of room for improvement, and I agree something along the lines of what Zang suggests would be more maintainable and ensure faster event processing as well as making it easier to have some form of reliability on event processing. I was considering doing something for the ovs-agent again in Juno, but since we've moving towards a unified agent, I think any new big ticket should address this effort. Salvatore On 17 June 2014 13:31, Zang MingJie zealot0...@gmail.com wrote: Hi: Awesome! Currently we are suffering lots of bugs in ovs-agent, also intent to rebuild a more stable flexible agent. Taking the experience of ovs-agent bugs, I think the concurrency problem is also a very important problem, the agent gets lots of event from different greenlets, the rpc, the ovs monitor or the main loop. I'd suggest to serialize all event to a queue, then process events in a dedicated thread. The thread check the events one by one ordered, and resolve what has been changed, then apply the corresponding changes. If there is any error occurred in the thread, discard the current processing event, do a fresh start event, which reset everything, then apply the correct settings. The threading model is so important and may prevent tons of bugs in the future development, we should describe it clearly in the architecture On Wed, Jun 11, 2014 at 4:19 AM, Mohammad Banikazemi m...@us.ibm.com wrote: Following the discussions in the ML2 subgroup weekly meetings, I have added more information on the etherpad [1] describing the proposed architecture for modular L2 agents. I have also posted some code fragments at [2] sketching the implementation of the proposed architecture. Please have a look when you get a chance and let us know if you have any comments
Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
Hi. Ryu provides ovs_vsctl.py library which is python equivalent to ovs-vsctl command. It speaks OVSDB protocl. https://github.com/osrg/ryu/blob/master/ryu/lib/ovs/vsctl.py So with the library, it's mostly mechanical change to convert ovs_lib.py, I think. I'm not aware other similar library written in python. thanks, Isaku Yamahata On Tue, Jun 17, 2014 at 11:38:36AM -0500, Kyle Mestery mest...@noironetworks.com wrote: Another area of improvement for the agent would be to move away from executing CLIs for port commands and instead use OVSDB. Terry Wilson and I talked about this, and re-writing ovs_lib to use an OVSDB connection instead of the CLI methods would be a huge improvement here. I'm not sure if Terry was going to move forward with this, but I'd be in favor of this for Juno if he or someone else wants to move in this direction. Thanks, Kyle On Tue, Jun 17, 2014 at 11:24 AM, Salvatore Orlando sorla...@nicira.com wrote: We've started doing this in a slightly more reasonable way for icehouse. What we've done is: - remove unnecessary notification from the server - process all port-related events, either trigger via RPC or via monitor in one place Obviously there is always a lot of room for improvement, and I agree something along the lines of what Zang suggests would be more maintainable and ensure faster event processing as well as making it easier to have some form of reliability on event processing. I was considering doing something for the ovs-agent again in Juno, but since we've moving towards a unified agent, I think any new big ticket should address this effort. Salvatore On 17 June 2014 13:31, Zang MingJie zealot0...@gmail.com wrote: Hi: Awesome! Currently we are suffering lots of bugs in ovs-agent, also intent to rebuild a more stable flexible agent. Taking the experience of ovs-agent bugs, I think the concurrency problem is also a very important problem, the agent gets lots of event from different greenlets, the rpc, the ovs monitor or the main loop. I'd suggest to serialize all event to a queue, then process events in a dedicated thread. The thread check the events one by one ordered, and resolve what has been changed, then apply the corresponding changes. If there is any error occurred in the thread, discard the current processing event, do a fresh start event, which reset everything, then apply the correct settings. The threading model is so important and may prevent tons of bugs in the future development, we should describe it clearly in the architecture On Wed, Jun 11, 2014 at 4:19 AM, Mohammad Banikazemi m...@us.ibm.com wrote: Following the discussions in the ML2 subgroup weekly meetings, I have added more information on the etherpad [1] describing the proposed architecture for modular L2 agents. I have also posted some code fragments at [2] sketching the implementation of the proposed architecture. Please have a look when you get a chance and let us know if you have any comments. [1] https://etherpad.openstack.org/p/modular-l2-agent-outline [2] https://review.openstack.org/#/c/99187/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Isaku Yamahata isaku.yamah...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
What's the progress by Terry Wilson? If not much, I'm willing to file blueprint/spec and drive it. thanks, On Wed, Jun 18, 2014 at 07:00:59PM +0900, Isaku Yamahata isaku.yamah...@gmail.com wrote: Hi. Ryu provides ovs_vsctl.py library which is python equivalent to ovs-vsctl command. It speaks OVSDB protocl. https://github.com/osrg/ryu/blob/master/ryu/lib/ovs/vsctl.py So with the library, it's mostly mechanical change to convert ovs_lib.py, I think. I'm not aware other similar library written in python. thanks, Isaku Yamahata On Tue, Jun 17, 2014 at 11:38:36AM -0500, Kyle Mestery mest...@noironetworks.com wrote: Another area of improvement for the agent would be to move away from executing CLIs for port commands and instead use OVSDB. Terry Wilson and I talked about this, and re-writing ovs_lib to use an OVSDB connection instead of the CLI methods would be a huge improvement here. I'm not sure if Terry was going to move forward with this, but I'd be in favor of this for Juno if he or someone else wants to move in this direction. Thanks, Kyle On Tue, Jun 17, 2014 at 11:24 AM, Salvatore Orlando sorla...@nicira.com wrote: We've started doing this in a slightly more reasonable way for icehouse. What we've done is: - remove unnecessary notification from the server - process all port-related events, either trigger via RPC or via monitor in one place Obviously there is always a lot of room for improvement, and I agree something along the lines of what Zang suggests would be more maintainable and ensure faster event processing as well as making it easier to have some form of reliability on event processing. I was considering doing something for the ovs-agent again in Juno, but since we've moving towards a unified agent, I think any new big ticket should address this effort. Salvatore On 17 June 2014 13:31, Zang MingJie zealot0...@gmail.com wrote: Hi: Awesome! Currently we are suffering lots of bugs in ovs-agent, also intent to rebuild a more stable flexible agent. Taking the experience of ovs-agent bugs, I think the concurrency problem is also a very important problem, the agent gets lots of event from different greenlets, the rpc, the ovs monitor or the main loop. I'd suggest to serialize all event to a queue, then process events in a dedicated thread. The thread check the events one by one ordered, and resolve what has been changed, then apply the corresponding changes. If there is any error occurred in the thread, discard the current processing event, do a fresh start event, which reset everything, then apply the correct settings. The threading model is so important and may prevent tons of bugs in the future development, we should describe it clearly in the architecture On Wed, Jun 11, 2014 at 4:19 AM, Mohammad Banikazemi m...@us.ibm.com wrote: Following the discussions in the ML2 subgroup weekly meetings, I have added more information on the etherpad [1] describing the proposed architecture for modular L2 agents. I have also posted some code fragments at [2] sketching the implementation of the proposed architecture. Please have a look when you get a chance and let us know if you have any comments. [1] https://etherpad.openstack.org/p/modular-l2-agent-outline [2] https://review.openstack.org/#/c/99187/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Isaku Yamahata isaku.yamah...@gmail.com -- Isaku Yamahata isaku.yamah...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
Hi: Awesome! Currently we are suffering lots of bugs in ovs-agent, also intent to rebuild a more stable flexible agent. Taking the experience of ovs-agent bugs, I think the concurrency problem is also a very important problem, the agent gets lots of event from different greenlets, the rpc, the ovs monitor or the main loop. I'd suggest to serialize all event to a queue, then process events in a dedicated thread. The thread check the events one by one ordered, and resolve what has been changed, then apply the corresponding changes. If there is any error occurred in the thread, discard the current processing event, do a fresh start event, which reset everything, then apply the correct settings. The threading model is so important and may prevent tons of bugs in the future development, we should describe it clearly in the architecture On Wed, Jun 11, 2014 at 4:19 AM, Mohammad Banikazemi m...@us.ibm.com wrote: Following the discussions in the ML2 subgroup weekly meetings, I have added more information on the etherpad [1] describing the proposed architecture for modular L2 agents. I have also posted some code fragments at [2] sketching the implementation of the proposed architecture. Please have a look when you get a chance and let us know if you have any comments. [1] https://etherpad.openstack.org/p/modular-l2-agent-outline [2] https://review.openstack.org/#/c/99187/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
We've started doing this in a slightly more reasonable way for icehouse. What we've done is: - remove unnecessary notification from the server - process all port-related events, either trigger via RPC or via monitor in one place Obviously there is always a lot of room for improvement, and I agree something along the lines of what Zang suggests would be more maintainable and ensure faster event processing as well as making it easier to have some form of reliability on event processing. I was considering doing something for the ovs-agent again in Juno, but since we've moving towards a unified agent, I think any new big ticket should address this effort. Salvatore On 17 June 2014 13:31, Zang MingJie zealot0...@gmail.com wrote: Hi: Awesome! Currently we are suffering lots of bugs in ovs-agent, also intent to rebuild a more stable flexible agent. Taking the experience of ovs-agent bugs, I think the concurrency problem is also a very important problem, the agent gets lots of event from different greenlets, the rpc, the ovs monitor or the main loop. I'd suggest to serialize all event to a queue, then process events in a dedicated thread. The thread check the events one by one ordered, and resolve what has been changed, then apply the corresponding changes. If there is any error occurred in the thread, discard the current processing event, do a fresh start event, which reset everything, then apply the correct settings. The threading model is so important and may prevent tons of bugs in the future development, we should describe it clearly in the architecture On Wed, Jun 11, 2014 at 4:19 AM, Mohammad Banikazemi m...@us.ibm.com wrote: Following the discussions in the ML2 subgroup weekly meetings, I have added more information on the etherpad [1] describing the proposed architecture for modular L2 agents. I have also posted some code fragments at [2] sketching the implementation of the proposed architecture. Please have a look when you get a chance and let us know if you have any comments. [1] https://etherpad.openstack.org/p/modular-l2-agent-outline [2] https://review.openstack.org/#/c/99187/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
Another area of improvement for the agent would be to move away from executing CLIs for port commands and instead use OVSDB. Terry Wilson and I talked about this, and re-writing ovs_lib to use an OVSDB connection instead of the CLI methods would be a huge improvement here. I'm not sure if Terry was going to move forward with this, but I'd be in favor of this for Juno if he or someone else wants to move in this direction. Thanks, Kyle On Tue, Jun 17, 2014 at 11:24 AM, Salvatore Orlando sorla...@nicira.com wrote: We've started doing this in a slightly more reasonable way for icehouse. What we've done is: - remove unnecessary notification from the server - process all port-related events, either trigger via RPC or via monitor in one place Obviously there is always a lot of room for improvement, and I agree something along the lines of what Zang suggests would be more maintainable and ensure faster event processing as well as making it easier to have some form of reliability on event processing. I was considering doing something for the ovs-agent again in Juno, but since we've moving towards a unified agent, I think any new big ticket should address this effort. Salvatore On 17 June 2014 13:31, Zang MingJie zealot0...@gmail.com wrote: Hi: Awesome! Currently we are suffering lots of bugs in ovs-agent, also intent to rebuild a more stable flexible agent. Taking the experience of ovs-agent bugs, I think the concurrency problem is also a very important problem, the agent gets lots of event from different greenlets, the rpc, the ovs monitor or the main loop. I'd suggest to serialize all event to a queue, then process events in a dedicated thread. The thread check the events one by one ordered, and resolve what has been changed, then apply the corresponding changes. If there is any error occurred in the thread, discard the current processing event, do a fresh start event, which reset everything, then apply the correct settings. The threading model is so important and may prevent tons of bugs in the future development, we should describe it clearly in the architecture On Wed, Jun 11, 2014 at 4:19 AM, Mohammad Banikazemi m...@us.ibm.com wrote: Following the discussions in the ML2 subgroup weekly meetings, I have added more information on the etherpad [1] describing the proposed architecture for modular L2 agents. I have also posted some code fragments at [2] sketching the implementation of the proposed architecture. Please have a look when you get a chance and let us know if you have any comments. [1] https://etherpad.openstack.org/p/modular-l2-agent-outline [2] https://review.openstack.org/#/c/99187/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
just a provocative thought: If we used the ovsdb connection instead, do we really need an L2 agent :P? On 17 June 2014 18:38, Kyle Mestery mest...@noironetworks.com wrote: Another area of improvement for the agent would be to move away from executing CLIs for port commands and instead use OVSDB. Terry Wilson and I talked about this, and re-writing ovs_lib to use an OVSDB connection instead of the CLI methods would be a huge improvement here. I'm not sure if Terry was going to move forward with this, but I'd be in favor of this for Juno if he or someone else wants to move in this direction. Thanks, Kyle On Tue, Jun 17, 2014 at 11:24 AM, Salvatore Orlando sorla...@nicira.com wrote: We've started doing this in a slightly more reasonable way for icehouse. What we've done is: - remove unnecessary notification from the server - process all port-related events, either trigger via RPC or via monitor in one place Obviously there is always a lot of room for improvement, and I agree something along the lines of what Zang suggests would be more maintainable and ensure faster event processing as well as making it easier to have some form of reliability on event processing. I was considering doing something for the ovs-agent again in Juno, but since we've moving towards a unified agent, I think any new big ticket should address this effort. Salvatore On 17 June 2014 13:31, Zang MingJie zealot0...@gmail.com wrote: Hi: Awesome! Currently we are suffering lots of bugs in ovs-agent, also intent to rebuild a more stable flexible agent. Taking the experience of ovs-agent bugs, I think the concurrency problem is also a very important problem, the agent gets lots of event from different greenlets, the rpc, the ovs monitor or the main loop. I'd suggest to serialize all event to a queue, then process events in a dedicated thread. The thread check the events one by one ordered, and resolve what has been changed, then apply the corresponding changes. If there is any error occurred in the thread, discard the current processing event, do a fresh start event, which reset everything, then apply the correct settings. The threading model is so important and may prevent tons of bugs in the future development, we should describe it clearly in the architecture On Wed, Jun 11, 2014 at 4:19 AM, Mohammad Banikazemi m...@us.ibm.com wrote: Following the discussions in the ML2 subgroup weekly meetings, I have added more information on the etherpad [1] describing the proposed architecture for modular L2 agents. I have also posted some code fragments at [2] sketching the implementation of the proposed architecture. Please have a look when you get a chance and let us know if you have any comments. [1] https://etherpad.openstack.org/p/modular-l2-agent-outline [2] https://review.openstack.org/#/c/99187/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
Not if you use ODL, and we don't want to reinvent that wheel. But by skipping CLI commands and instead using OVSDB programmatically from agent to ovs-vswitchd, that's a decent improvement. On Tue, Jun 17, 2014 at 11:56 AM, Armando M. arma...@gmail.com wrote: just a provocative thought: If we used the ovsdb connection instead, do we really need an L2 agent :P? On 17 June 2014 18:38, Kyle Mestery mest...@noironetworks.com wrote: Another area of improvement for the agent would be to move away from executing CLIs for port commands and instead use OVSDB. Terry Wilson and I talked about this, and re-writing ovs_lib to use an OVSDB connection instead of the CLI methods would be a huge improvement here. I'm not sure if Terry was going to move forward with this, but I'd be in favor of this for Juno if he or someone else wants to move in this direction. Thanks, Kyle On Tue, Jun 17, 2014 at 11:24 AM, Salvatore Orlando sorla...@nicira.com wrote: We've started doing this in a slightly more reasonable way for icehouse. What we've done is: - remove unnecessary notification from the server - process all port-related events, either trigger via RPC or via monitor in one place Obviously there is always a lot of room for improvement, and I agree something along the lines of what Zang suggests would be more maintainable and ensure faster event processing as well as making it easier to have some form of reliability on event processing. I was considering doing something for the ovs-agent again in Juno, but since we've moving towards a unified agent, I think any new big ticket should address this effort. Salvatore On 17 June 2014 13:31, Zang MingJie zealot0...@gmail.com wrote: Hi: Awesome! Currently we are suffering lots of bugs in ovs-agent, also intent to rebuild a more stable flexible agent. Taking the experience of ovs-agent bugs, I think the concurrency problem is also a very important problem, the agent gets lots of event from different greenlets, the rpc, the ovs monitor or the main loop. I'd suggest to serialize all event to a queue, then process events in a dedicated thread. The thread check the events one by one ordered, and resolve what has been changed, then apply the corresponding changes. If there is any error occurred in the thread, discard the current processing event, do a fresh start event, which reset everything, then apply the correct settings. The threading model is so important and may prevent tons of bugs in the future development, we should describe it clearly in the architecture On Wed, Jun 11, 2014 at 4:19 AM, Mohammad Banikazemi m...@us.ibm.com wrote: Following the discussions in the ML2 subgroup weekly meetings, I have added more information on the etherpad [1] describing the proposed architecture for modular L2 agents. I have also posted some code fragments at [2] sketching the implementation of the proposed architecture. Please have a look when you get a chance and let us know if you have any comments. [1] https://etherpad.openstack.org/p/modular-l2-agent-outline [2] https://review.openstack.org/#/c/99187/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
Managing the ports and plumbing logic is today driven by L2 Agent, with little assistance from controller. If we plan to move that functionality to the controller, the controller has to be more heavy weight (both hardware and software) since it has to do the job of L2 Agent for all the compute servers in the cloud. , We need to re-verify all scale numbers for the controller on POC’ing of such a change. That said, replacing CLI with direct OVSDB calls in the L2 Agent is certainly a good direction. Today, OVS Agent invokes flow calls of OVS-Lib but has no idea (or processing) to follow up on success or failure of such invocations. Nor there is certain guarantee that all such flow invocations would be executed by the third-process fired by OVS-Lib to execute CLI. When we transition to OVSDB calls which are more programmatic in nature, we can enhance the Flow API (OVS-Lib) to provide more fine grained errors/return codes (or content) and ovs-agent (and even other components) can act on such return state more intelligently/appropriately. -- Thanks, Vivek From: Armando M. [mailto:arma...@gmail.com] Sent: Tuesday, June 17, 2014 10:26 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture just a provocative thought: If we used the ovsdb connection instead, do we really need an L2 agent :P? On 17 June 2014 18:38, Kyle Mestery mest...@noironetworks.commailto:mest...@noironetworks.com wrote: Another area of improvement for the agent would be to move away from executing CLIs for port commands and instead use OVSDB. Terry Wilson and I talked about this, and re-writing ovs_lib to use an OVSDB connection instead of the CLI methods would be a huge improvement here. I'm not sure if Terry was going to move forward with this, but I'd be in favor of this for Juno if he or someone else wants to move in this direction. Thanks, Kyle On Tue, Jun 17, 2014 at 11:24 AM, Salvatore Orlando sorla...@nicira.commailto:sorla...@nicira.com wrote: We've started doing this in a slightly more reasonable way for icehouse. What we've done is: - remove unnecessary notification from the server - process all port-related events, either trigger via RPC or via monitor in one place Obviously there is always a lot of room for improvement, and I agree something along the lines of what Zang suggests would be more maintainable and ensure faster event processing as well as making it easier to have some form of reliability on event processing. I was considering doing something for the ovs-agent again in Juno, but since we've moving towards a unified agent, I think any new big ticket should address this effort. Salvatore On 17 June 2014 13:31, Zang MingJie zealot0...@gmail.commailto:zealot0...@gmail.com wrote: Hi: Awesome! Currently we are suffering lots of bugs in ovs-agent, also intent to rebuild a more stable flexible agent. Taking the experience of ovs-agent bugs, I think the concurrency problem is also a very important problem, the agent gets lots of event from different greenlets, the rpc, the ovs monitor or the main loop. I'd suggest to serialize all event to a queue, then process events in a dedicated thread. The thread check the events one by one ordered, and resolve what has been changed, then apply the corresponding changes. If there is any error occurred in the thread, discard the current processing event, do a fresh start event, which reset everything, then apply the correct settings. The threading model is so important and may prevent tons of bugs in the future development, we should describe it clearly in the architecture On Wed, Jun 11, 2014 at 4:19 AM, Mohammad Banikazemi m...@us.ibm.commailto:m...@us.ibm.com wrote: Following the discussions in the ML2 subgroup weekly meetings, I have added more information on the etherpad [1] describing the proposed architecture for modular L2 agents. I have also posted some code fragments at [2] sketching the implementation of the proposed architecture. Please have a look when you get a chance and let us know if you have any comments. [1] https://etherpad.openstack.org/p/modular-l2-agent-outline [2] https://review.openstack.org/#/c/99187/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
Hi, Does it make sense also to have the choice between ovs-ofctl CLI and a direct OF1.3 connection too in the ovs-agent? Best Regards, Racha On Tue, Jun 17, 2014 at 10:25 AM, Narasimhan, Vivekanandan vivekanandan.narasim...@hp.com wrote: Managing the ports and plumbing logic is today driven by L2 Agent, with little assistance from controller. If we plan to move that functionality to the controller, the controller has to be more heavy weight (both hardware and software) since it has to do the job of L2 Agent for all the compute servers in the cloud. , We need to re-verify all scale numbers for the controller on POC’ing of such a change. That said, replacing CLI with direct OVSDB calls in the L2 Agent is certainly a good direction. Today, OVS Agent invokes flow calls of OVS-Lib but has no idea (or processing) to follow up on success or failure of such invocations. Nor there is certain guarantee that all such flow invocations would be executed by the third-process fired by OVS-Lib to execute CLI. When we transition to OVSDB calls which are more programmatic in nature, we can enhance the Flow API (OVS-Lib) to provide more fine grained errors/return codes (or content) and ovs-agent (and even other components) can act on such return state more intelligently/appropriately. -- Thanks, Vivek *From:* Armando M. [mailto:arma...@gmail.com] *Sent:* Tuesday, June 17, 2014 10:26 PM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture just a provocative thought: If we used the ovsdb connection instead, do we really need an L2 agent :P? On 17 June 2014 18:38, Kyle Mestery mest...@noironetworks.com wrote: Another area of improvement for the agent would be to move away from executing CLIs for port commands and instead use OVSDB. Terry Wilson and I talked about this, and re-writing ovs_lib to use an OVSDB connection instead of the CLI methods would be a huge improvement here. I'm not sure if Terry was going to move forward with this, but I'd be in favor of this for Juno if he or someone else wants to move in this direction. Thanks, Kyle On Tue, Jun 17, 2014 at 11:24 AM, Salvatore Orlando sorla...@nicira.com wrote: We've started doing this in a slightly more reasonable way for icehouse. What we've done is: - remove unnecessary notification from the server - process all port-related events, either trigger via RPC or via monitor in one place Obviously there is always a lot of room for improvement, and I agree something along the lines of what Zang suggests would be more maintainable and ensure faster event processing as well as making it easier to have some form of reliability on event processing. I was considering doing something for the ovs-agent again in Juno, but since we've moving towards a unified agent, I think any new big ticket should address this effort. Salvatore On 17 June 2014 13:31, Zang MingJie zealot0...@gmail.com wrote: Hi: Awesome! Currently we are suffering lots of bugs in ovs-agent, also intent to rebuild a more stable flexible agent. Taking the experience of ovs-agent bugs, I think the concurrency problem is also a very important problem, the agent gets lots of event from different greenlets, the rpc, the ovs monitor or the main loop. I'd suggest to serialize all event to a queue, then process events in a dedicated thread. The thread check the events one by one ordered, and resolve what has been changed, then apply the corresponding changes. If there is any error occurred in the thread, discard the current processing event, do a fresh start event, which reset everything, then apply the correct settings. The threading model is so important and may prevent tons of bugs in the future development, we should describe it clearly in the architecture On Wed, Jun 11, 2014 at 4:19 AM, Mohammad Banikazemi m...@us.ibm.com wrote: Following the discussions in the ML2 subgroup weekly meetings, I have added more information on the etherpad [1] describing the proposed architecture for modular L2 agents. I have also posted some code fragments at [2] sketching the implementation of the proposed architecture. Please have a look when you get a chance and let us know if you have any comments. [1] https://etherpad.openstack.org/p/modular-l2-agent-outline [2] https://review.openstack.org/#/c/99187/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
I don't think so. Once we implement the OVSDB support, we will deprecate using the CLI commands in ovs_lib. On Tue, Jun 17, 2014 at 12:50 PM, racha ben...@gmail.com wrote: Hi, Does it make sense also to have the choice between ovs-ofctl CLI and a direct OF1.3 connection too in the ovs-agent? Best Regards, Racha On Tue, Jun 17, 2014 at 10:25 AM, Narasimhan, Vivekanandan vivekanandan.narasim...@hp.com wrote: Managing the ports and plumbing logic is today driven by L2 Agent, with little assistance from controller. If we plan to move that functionality to the controller, the controller has to be more heavy weight (both hardware and software) since it has to do the job of L2 Agent for all the compute servers in the cloud. , We need to re-verify all scale numbers for the controller on POC’ing of such a change. That said, replacing CLI with direct OVSDB calls in the L2 Agent is certainly a good direction. Today, OVS Agent invokes flow calls of OVS-Lib but has no idea (or processing) to follow up on success or failure of such invocations. Nor there is certain guarantee that all such flow invocations would be executed by the third-process fired by OVS-Lib to execute CLI. When we transition to OVSDB calls which are more programmatic in nature, we can enhance the Flow API (OVS-Lib) to provide more fine grained errors/return codes (or content) and ovs-agent (and even other components) can act on such return state more intelligently/appropriately. -- Thanks, Vivek From: Armando M. [mailto:arma...@gmail.com] Sent: Tuesday, June 17, 2014 10:26 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture just a provocative thought: If we used the ovsdb connection instead, do we really need an L2 agent :P? On 17 June 2014 18:38, Kyle Mestery mest...@noironetworks.com wrote: Another area of improvement for the agent would be to move away from executing CLIs for port commands and instead use OVSDB. Terry Wilson and I talked about this, and re-writing ovs_lib to use an OVSDB connection instead of the CLI methods would be a huge improvement here. I'm not sure if Terry was going to move forward with this, but I'd be in favor of this for Juno if he or someone else wants to move in this direction. Thanks, Kyle On Tue, Jun 17, 2014 at 11:24 AM, Salvatore Orlando sorla...@nicira.com wrote: We've started doing this in a slightly more reasonable way for icehouse. What we've done is: - remove unnecessary notification from the server - process all port-related events, either trigger via RPC or via monitor in one place Obviously there is always a lot of room for improvement, and I agree something along the lines of what Zang suggests would be more maintainable and ensure faster event processing as well as making it easier to have some form of reliability on event processing. I was considering doing something for the ovs-agent again in Juno, but since we've moving towards a unified agent, I think any new big ticket should address this effort. Salvatore On 17 June 2014 13:31, Zang MingJie zealot0...@gmail.com wrote: Hi: Awesome! Currently we are suffering lots of bugs in ovs-agent, also intent to rebuild a more stable flexible agent. Taking the experience of ovs-agent bugs, I think the concurrency problem is also a very important problem, the agent gets lots of event from different greenlets, the rpc, the ovs monitor or the main loop. I'd suggest to serialize all event to a queue, then process events in a dedicated thread. The thread check the events one by one ordered, and resolve what has been changed, then apply the corresponding changes. If there is any error occurred in the thread, discard the current processing event, do a fresh start event, which reset everything, then apply the correct settings. The threading model is so important and may prevent tons of bugs in the future development, we should describe it clearly in the architecture On Wed, Jun 11, 2014 at 4:19 AM, Mohammad Banikazemi m...@us.ibm.com wrote: Following the discussions in the ML2 subgroup weekly meetings, I have added more information on the etherpad [1] describing the proposed architecture for modular L2 agents. I have also posted some code fragments at [2] sketching the implementation of the proposed architecture. Please have a look when you get a chance and let us know if you have any comments. [1] https://etherpad.openstack.org/p/modular-l2-agent-outline [2] https://review.openstack.org/#/c/99187/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture
Mine wasn't really a serious suggestion, Neutron's controlling logic is already bloated as it is, and my personal opinion would be in favor of a leaner Neutron Server rather than a more complex one; adding more controller-like logic to it certainly goes against that direction :) Having said that and as Vivek pointed out, using ovsdb gives us finer control and ability to react more effectively, however, with the current server-agent rpc framework there's no way of leveraging that...so in a grand scheme of things I'd rather see it prioritized lower rather than higher, to give precedence to rearchitecting the framework first. Armando On 17 June 2014 19:25, Narasimhan, Vivekanandan vivekanandan.narasim...@hp.com wrote: Managing the ports and plumbing logic is today driven by L2 Agent, with little assistance from controller. If we plan to move that functionality to the controller, the controller has to be more heavy weight (both hardware and software) since it has to do the job of L2 Agent for all the compute servers in the cloud. , We need to re-verify all scale numbers for the controller on POC’ing of such a change. That said, replacing CLI with direct OVSDB calls in the L2 Agent is certainly a good direction. Today, OVS Agent invokes flow calls of OVS-Lib but has no idea (or processing) to follow up on success or failure of such invocations. Nor there is certain guarantee that all such flow invocations would be executed by the third-process fired by OVS-Lib to execute CLI. When we transition to OVSDB calls which are more programmatic in nature, we can enhance the Flow API (OVS-Lib) to provide more fine grained errors/return codes (or content) and ovs-agent (and even other components) can act on such return state more intelligently/appropriately. -- Thanks, Vivek *From:* Armando M. [mailto:arma...@gmail.com] *Sent:* Tuesday, June 17, 2014 10:26 PM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [Neutron][ML2] Modular L2 agent architecture just a provocative thought: If we used the ovsdb connection instead, do we really need an L2 agent :P? On 17 June 2014 18:38, Kyle Mestery mest...@noironetworks.com wrote: Another area of improvement for the agent would be to move away from executing CLIs for port commands and instead use OVSDB. Terry Wilson and I talked about this, and re-writing ovs_lib to use an OVSDB connection instead of the CLI methods would be a huge improvement here. I'm not sure if Terry was going to move forward with this, but I'd be in favor of this for Juno if he or someone else wants to move in this direction. Thanks, Kyle On Tue, Jun 17, 2014 at 11:24 AM, Salvatore Orlando sorla...@nicira.com wrote: We've started doing this in a slightly more reasonable way for icehouse. What we've done is: - remove unnecessary notification from the server - process all port-related events, either trigger via RPC or via monitor in one place Obviously there is always a lot of room for improvement, and I agree something along the lines of what Zang suggests would be more maintainable and ensure faster event processing as well as making it easier to have some form of reliability on event processing. I was considering doing something for the ovs-agent again in Juno, but since we've moving towards a unified agent, I think any new big ticket should address this effort. Salvatore On 17 June 2014 13:31, Zang MingJie zealot0...@gmail.com wrote: Hi: Awesome! Currently we are suffering lots of bugs in ovs-agent, also intent to rebuild a more stable flexible agent. Taking the experience of ovs-agent bugs, I think the concurrency problem is also a very important problem, the agent gets lots of event from different greenlets, the rpc, the ovs monitor or the main loop. I'd suggest to serialize all event to a queue, then process events in a dedicated thread. The thread check the events one by one ordered, and resolve what has been changed, then apply the corresponding changes. If there is any error occurred in the thread, discard the current processing event, do a fresh start event, which reset everything, then apply the correct settings. The threading model is so important and may prevent tons of bugs in the future development, we should describe it clearly in the architecture On Wed, Jun 11, 2014 at 4:19 AM, Mohammad Banikazemi m...@us.ibm.com wrote: Following the discussions in the ML2 subgroup weekly meetings, I have added more information on the etherpad [1] describing the proposed architecture for modular L2 agents. I have also posted some code fragments at [2] sketching the implementation of the proposed architecture. Please have a look when you get a chance and let us know if you have any comments. [1] https://etherpad.openstack.org/p/modular-l2-agent
