Re: [ovs-dev] [RFC] Introduce "OpenFlow Controller as Shared Library"

2019-04-16 Thread Ben Pfaff
On Sun, Mar 24, 2019 at 09:54:02PM -0700, Ansis Atteka wrote:
> From: Ansis Atteka 
> 
> Currently ovs-vswitchd process can communicate with an OpenFlow
> controller only through tcp, unix and ssl sockets.
> 
> This patch would allow ovs-vswitchd process to communicate
> with an OpenFlow controller by directly calling into its
> code that provides interface similar to a socket (ie
> implements read() and write() functions).
> 
> There are few benefits of using shared library as OpenFlow
> controller:
> 
> 1. Better performance by
>a) avoiding copying OpenFlow messages to socket buffers; AND
>b) reducing context switches.
>The preliminary tests that I did improved performance by ~30% for
>an OpenFlow controller that handles PACKET_INs and resubmits packets
>with PACKET_OUTs.
> 2. Better parallelization in future by distributing the load
>over ovs-vswitchd handler threads (currently only one thread calls into
>the shared library code).
> 3. Eliminate undeterministic thread blocking that may be caused when
>socket buffers are full.
> 4. In some cases better security (e.g. by allowing to confine ovs-vswitchd
>process to a stricter Access Control policy). Although, In some cases
>security may get worse (e.g. because controller would run in the same
>virtual memory space as ovs-vswitchd process).
> 
> While the code is enough to demonstrate PoC, I have left some TODOs.
> Because of that I am sending this code as RFC to hear more feedback
> from community on subjects as
> 1. what should be the API that shared libraries should export.
> 2. if I am possibly missing something critical that makes this approach
>not feasible (e.g. race conditions, something that ovs does behind
>scenes (like vlog module initialization for plugin) and would require
>overhaul in other components, impossible to integrate code with event
>loop, inpractical cleanups that would make it impossible
>to unload plugins)
> 
> In this patch I am proposing an API that requires library to export
> socket functions that mimic socket read() and write() functions.  In some
> cases this allows to easy retrofit existing OpenFlow controllers as
> shared library plugins.  To test out one can set test controller with
> 
> ovs-vsctl add-br brX
> ovs-vsctl set-controller brX dl:/ovs/tests/.libs/libtest-dl.so
> sudo ovs-ofctl add-flow brX "actions=controller"
> 
> And observe that packet-ins get to OpenFlow controller plugin:
> 
> 2019-03-25T04:10:35.615Z|01230|libtestdl|INFO|received 255 byte message from 
> Open vSwitch
> 2019-03-25T04:10:35.615Z|01231|libtestdl|INFO|Received OFPTYPE_PACKET_IN
> 2019-03-25T04:10:36.616Z|01232|libtestdl|INFO|received 255 byte message from 
> Open vSwitch
> 2019-03-25T04:10:36.616Z|01233|libtestdl|INFO|Received OFPTYPE_PACKET_IN
> 2019-03-25T04:10:38.143Z|01234|libtestdl|INFO|received 8 byte message from 
> Open vSwitch
> 
> Another approach would be for shared libraries to export init() and
> finit() functions that would self register and self-unregisters certain class
> implementations.
> 
> Signed-off-by: Ansis Atteka 

This approach seems pretty reasonable overall.  We've had people talk
about plugin interfaces before, but the assumption was always that it
would be essentially a new API with a broad set of definitions.  This
approach is much simpler, API-wise, since OpenFlow is still the core and
it's just a new transport.

You might be able to use a "seq" object (lib/seq.[ch]) for notification
between the controller and the main OVS code.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev


[ovs-dev] [RFC] Introduce "OpenFlow Controller as Shared Library"

2019-03-24 Thread Ansis Atteka
From: Ansis Atteka 

Currently ovs-vswitchd process can communicate with an OpenFlow
controller only through tcp, unix and ssl sockets.

This patch would allow ovs-vswitchd process to communicate
with an OpenFlow controller by directly calling into its
code that provides interface similar to a socket (ie
implements read() and write() functions).

There are few benefits of using shared library as OpenFlow
controller:

1. Better performance by
   a) avoiding copying OpenFlow messages to socket buffers; AND
   b) reducing context switches.
   The preliminary tests that I did improved performance by ~30% for
   an OpenFlow controller that handles PACKET_INs and resubmits packets
   with PACKET_OUTs.
2. Better parallelization in future by distributing the load
   over ovs-vswitchd handler threads (currently only one thread calls into
   the shared library code).
3. Eliminate undeterministic thread blocking that may be caused when
   socket buffers are full.
4. In some cases better security (e.g. by allowing to confine ovs-vswitchd
   process to a stricter Access Control policy). Although, In some cases
   security may get worse (e.g. because controller would run in the same
   virtual memory space as ovs-vswitchd process).

While the code is enough to demonstrate PoC, I have left some TODOs.
Because of that I am sending this code as RFC to hear more feedback
from community on subjects as
1. what should be the API that shared libraries should export.
2. if I am possibly missing something critical that makes this approach
   not feasible (e.g. race conditions, something that ovs does behind
   scenes (like vlog module initialization for plugin) and would require
   overhaul in other components, impossible to integrate code with event
   loop, inpractical cleanups that would make it impossible
   to unload plugins)

In this patch I am proposing an API that requires library to export
socket functions that mimic socket read() and write() functions.  In some
cases this allows to easy retrofit existing OpenFlow controllers as
shared library plugins.  To test out one can set test controller with

ovs-vsctl add-br brX
ovs-vsctl set-controller brX dl:/ovs/tests/.libs/libtest-dl.so
sudo ovs-ofctl add-flow brX "actions=controller"

And observe that packet-ins get to OpenFlow controller plugin:

2019-03-25T04:10:35.615Z|01230|libtestdl|INFO|received 255 byte message from 
Open vSwitch
2019-03-25T04:10:35.615Z|01231|libtestdl|INFO|Received OFPTYPE_PACKET_IN
2019-03-25T04:10:36.616Z|01232|libtestdl|INFO|received 255 byte message from 
Open vSwitch
2019-03-25T04:10:36.616Z|01233|libtestdl|INFO|Received OFPTYPE_PACKET_IN
2019-03-25T04:10:38.143Z|01234|libtestdl|INFO|received 8 byte message from Open 
vSwitch

Another approach would be for shared libraries to export init() and
finit() functions that would self register and self-unregisters certain class
implementations.

Signed-off-by: Ansis Atteka 
---
 Makefile.am   |   2 +-
 lib/automake.mk   |   3 +
 lib/stream-dl.c   | 161 +++
 lib/stream-dl.h   |  28 +
 lib/stream-dlopen.c   |  79 +++
 lib/stream-provider.h |   3 +
 lib/stream.c  |   1 +
 lib/vconn-provider.h  |   1 +
 lib/vconn-stream.c|   2 +
 lib/vconn.c   |   1 +
 tests/automake.mk |   6 ++
 tests/test-dl.c   | 169 ++
 12 files changed, 455 insertions(+), 1 deletion(-)
 create mode 100644 lib/stream-dl.c
 create mode 100644 lib/stream-dl.h
 create mode 100644 lib/stream-dlopen.c
 create mode 100644 tests/test-dl.c

diff --git a/Makefile.am b/Makefile.am
index ff1f94b48..7cb0a6b55 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -338,7 +338,7 @@ thread-safety-check:
if test -e .git && (git --version) >/dev/null 2>&1 && \
  grep -n -f build-aux/thread-safety-blacklist \
`git ls-files | grep '\.[ch]$$' \
- | $(EGREP) -v '^datapath|^lib/sflow|^third-party'` /dev/null \
+ | $(EGREP) -v '^datapath|^lib/sflow|^lib/stream-dl|^third-party'` 
/dev/null \
  | $(EGREP) -v ':[ ]*/?\*'; \
then \
  echo "See above for list of calls to functions that are"; \
diff --git a/lib/automake.mk b/lib/automake.mk
index cc5dccf39..1e9a6eefa 100644
--- a/lib/automake.mk
+++ b/lib/automake.mk
@@ -269,6 +269,8 @@ lib_libopenvswitch_la_SOURCES = \
lib/sset.h \
lib/stp.c \
lib/stp.h \
+lib/stream-dl.c \
+lib/stream-dl.h \
lib/stream-fd.c \
lib/stream-fd.h \
lib/stream-provider.h \
@@ -346,6 +348,7 @@ lib_libopenvswitch_la_SOURCES += \
lib/signals.c \
lib/signals.h \
lib/socket-util-unix.c \
+lib/stream-dlopen.c \
lib/stream-unix.c
 endif
 
diff --git a/lib/stream-dl.c b/lib/stream-dl.c
new file mode 100644
index 0..1ff7dfd8e
--- /dev/null
+++ b/lib/stream-dl.c
@@ -0,0 +1,161 @@
+/*
+ * Copyr