Re: [ovs-discuss] Source Code - Multiple Controllers Round Robin Load Balancing
It seems to me like a bad idea, in general, to use round robin load balancing to send packets to controllers because that is likely to reorder packets. I think that it would probably be a better idea to balance packets across controllers on a flow-by-flow basis. I think that you could use an OpenFlow "select" group for that, by using a bucket for each controller. That would not require any change to OVS. On Sun, Nov 19, 2017 at 01:47:01PM -0800, Brian Perry wrote: > Summary: OvS last commit 52f793b > I am trying to modify the switch's code to use a round robin scheduler for > sending asynchronous messages (especially PACKET_IN messages) to one of the > controllers within a multiple controller SDN setup. After traversing the > code I think I found where I need to insert this round robin scheduler, > line 4766 and 6225 of ofproto-dpif-xlate.c. > > I plan on modifying this part of the code and testing it to see if I was > correct but it would be great to have some feedback from the community on > this. > 1) Am I on the right path, do I only need to modify the code around line > 4766? Or do I need to modify another part of the code? > 2) What is OFPACT_CONTROLLER case for? From what I can gather macro > OFPACT_FOR_EACH uses a macro that goes through a list that deal with table > entry actions but it could call the execute_controller_action() function > and I don't know why it would. > 3) What does emit_continuation() function do? It looked like it had to do > with table lookups instead of sending asynchronous messages to the > controller. If it only does table lookups why would it call the > execute_controller_action() function. > > > Details: > I setup a simple Mininet topology with a single switch, 3 user computers, > and 2 controllers, where everything is connected to the switch. Currently > when the switch receives a packet with no corresponding forwarding rule it > sends a request to both controllers. I would like it so that it sends the > forwarding rule request in a round robin method. So the first forwarding > rule request will be sent to only controller 1, then the next forwarding > rule request will be sent to only controller 2, then the next one to only > controller 1, then the next one only to controller 2, etc. > > Along side other documents I've read a description of the Open vSwitch > architecture (https://www.slideshare.net/rajdeep/openvswitch-deep-dive) and > pages 25-28 of Pavel's master thesis (https://mail.openvswitch.org/ > pipermail/ovs-discuss/2017-February/043763.html) to get a better > understanding of the internals of OvS. This information paired with the > following post (https://mail.openvswitch.org/pipermail/ovs-discuss/2016- > March/040236.html) informed me that the source code I am looking for is > used within the Userspace and located within the ofproto directory. I > started looking in the ofproto directory for the handle_openflow() > function. This eventually lead me to look at the structure and usage of > ofconn which lead me to line 1741 of connmgr.c where I found the structure > ofproto_async_msg and function connmgr_send_async_msg. After following the > function I noticed that ofproto_async_msg.controller_id is assigned in only > 2 different places; within the execute_controller_action() and > emit_continuation() functions. I continued following the > execute_controller_action() function and noticed that it's called in only 5 > different locations all within the file ofproto-dpif-xlate.c. Within those > 5 locations only lines 4766 and 6225 use some sort of loop to craft and > send asynchronous messages to multiple controllers. > > So my questions become: > 1) Am I on the right path, do I only need to modify the code around line > 4766? Or do I need to modify another part of the code? > 2) What is OFPACT_CONTROLLER case for? From what I can gather macro > OFPACT_FOR_EACH uses a macro that goes through a list that deal with table > entry actions but it could call the execute_controller_action() function > and I don't know why it would. > 3) What does emit_continuation() function do? It looked like it had to do > with table lookups instead of sending asynchronous messages to the > controller. If it only does table lookups why would it call the > execute_controller_action() function. > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] Source Code - Multiple Controllers Round Robin Load Balancing
Summary: OvS last commit 52f793b I am trying to modify the switch's code to use a round robin scheduler for sending asynchronous messages (especially PACKET_IN messages) to one of the controllers within a multiple controller SDN setup. After traversing the code I think I found where I need to insert this round robin scheduler, line 4766 and 6225 of ofproto-dpif-xlate.c. I plan on modifying this part of the code and testing it to see if I was correct but it would be great to have some feedback from the community on this. 1) Am I on the right path, do I only need to modify the code around line 4766? Or do I need to modify another part of the code? 2) What is OFPACT_CONTROLLER case for? From what I can gather macro OFPACT_FOR_EACH uses a macro that goes through a list that deal with table entry actions but it could call the execute_controller_action() function and I don't know why it would. 3) What does emit_continuation() function do? It looked like it had to do with table lookups instead of sending asynchronous messages to the controller. If it only does table lookups why would it call the execute_controller_action() function. Details: I setup a simple Mininet topology with a single switch, 3 user computers, and 2 controllers, where everything is connected to the switch. Currently when the switch receives a packet with no corresponding forwarding rule it sends a request to both controllers. I would like it so that it sends the forwarding rule request in a round robin method. So the first forwarding rule request will be sent to only controller 1, then the next forwarding rule request will be sent to only controller 2, then the next one to only controller 1, then the next one only to controller 2, etc. Along side other documents I've read a description of the Open vSwitch architecture (https://www.slideshare.net/rajdeep/openvswitch-deep-dive) and pages 25-28 of Pavel's master thesis (https://mail.openvswitch.org/ pipermail/ovs-discuss/2017-February/043763.html) to get a better understanding of the internals of OvS. This information paired with the following post (https://mail.openvswitch.org/pipermail/ovs-discuss/2016- March/040236.html) informed me that the source code I am looking for is used within the Userspace and located within the ofproto directory. I started looking in the ofproto directory for the handle_openflow() function. This eventually lead me to look at the structure and usage of ofconn which lead me to line 1741 of connmgr.c where I found the structure ofproto_async_msg and function connmgr_send_async_msg. After following the function I noticed that ofproto_async_msg.controller_id is assigned in only 2 different places; within the execute_controller_action() and emit_continuation() functions. I continued following the execute_controller_action() function and noticed that it's called in only 5 different locations all within the file ofproto-dpif-xlate.c. Within those 5 locations only lines 4766 and 6225 use some sort of loop to craft and send asynchronous messages to multiple controllers. So my questions become: 1) Am I on the right path, do I only need to modify the code around line 4766? Or do I need to modify another part of the code? 2) What is OFPACT_CONTROLLER case for? From what I can gather macro OFPACT_FOR_EACH uses a macro that goes through a list that deal with table entry actions but it could call the execute_controller_action() function and I don't know why it would. 3) What does emit_continuation() function do? It looked like it had to do with table lookups instead of sending asynchronous messages to the controller. If it only does table lookups why would it call the execute_controller_action() function. ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Regarding Buffer size
Hello Prabhu, Thanks for your detailed reply. I have understood your points and will try to work in that direction. Can you just let me know how can I manage the buffer size using Linux tc utility, as I tried to understand Linux tc but was not able to figure out the way for managing the buffer size for each queue configured for each port of the switch. I read somewhere that txqueuelen can be used for reducing the size of the queue for each port which is 1000 by default, but could not verify it. I am able to shape the traffic as per the bandwidth requirements by using minimum and maximum rates using tc linux and htb, but I am not able to figure out the way for buffer size. I appreciate your time and cooperation for my research. Regards, Ankur From: Prabhu Sent: Sunday, November 19, 2017 12:03 PM To: ANKUR SHETH; Ben Pfaff; ryu-de...@lists.sourceforge.net; ovs-discuss@openvswitch.org; la...@vger.kernel.org Subject: Re: Regarding Buffer size Hello Ankur, Buffer Size management is out of scope for Openvswitch and RYU SDN Controller. I have been investigating on several approaches to use queue based solutions for one of the problem that I was looking to solve. >From my understanding, 1. You should be use tc linux utility to define queue size (or) buffer size with limits such as no.of.packets it can buffer and associate it with the port 2. Openvswitch do not have direct control over the property that you are looking for (manipulating buffer properties which can be done using tc). 3. RYU SDN controller does not have API complex queue management Also, you should be aware that OVS process packets in two modes fast path and slow path. You may have to decide where to place the buffer and depending on that you may have to either modify kernel or you can completely achieve you goal with performance tradeoff. NOTE: The above given description is based on my understanding, please correct it in case of being wrong. Thanks, Prabhu On 11/19/2017 01:38 AM, ANKUR SHETH wrote: Hello all, I actually intend to control the buffer size for the ingress as well as egress traffic from the openvswitch. I know that ovs internally uses linux tc utility to control the traffic, but even after referring to tc command manual I am not able to figure out how to manage the queue size of the buffers. I am excited about working on queue management on ovs switches using RYU SDN controller, but I am unable to find the correct direction even after understanding linux tc. Could you please suggest on how I can achieve it or guide me to some documentation. Any help is appreciated. Regards, Ankur ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Regarding Buffer size
Hello Ankur, Buffer Size management is out of scope for Openvswitch and RYU SDN Controller. I have been investigating on several approaches to use queue based solutions for one of the problem that I was looking to solve. From my understanding, 1. You should be use *tc *linux utility to define queue size (or) buffer size with limits such as no.of.packets it can buffer and associate it with the port 2. Openvswitch do not have direct control over the property that you are looking for (manipulating buffer properties which can be done using tc). 3. RYU SDN controller does not have API complex queue management Also, you should be aware that OVS process packets in two modes fast path and slow path. You may have to decide where to place the buffer and depending on that you may have to either modify kernel or you can completely achieve you goal with performance tradeoff. NOTE: The above given description is based on my understanding, please correct it in case of being wrong. Thanks, Prabhu On 11/19/2017 01:38 AM, ANKUR SHETH wrote: Hello all, I actually intend to control the buffer size for the ingress as well as egress traffic from the openvswitch. I know that ovs internally uses linux tc utility to control the traffic, but even after referring to tc command manual I am not able to figure out how to manage the queue size of the buffers. I am excited about working on queue management on ovs switches using RYU SDN controller, but I am unable to find the correct direction even after understanding linux tc. Could you please suggest on how I can achieve it or guide me to some documentation. Any help is appreciated. Regards, Ankur ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
