I've now performed tests where I measured the wall clock time of each iteration through the test. I've created a plot of the C vs. the DDlog ovn-northd here: https://i.imgur.com/bIsbeMN.png

This confirms for me that the DDlog implementation does a better job of staying consistent in its execution time even as the network grows larger. You can see that the C implementation gets noticeably slower as the network grows. The DDlog implementation also gets slower but the slope of the trend is much less severe compared to C.

Through my initial tests, I'm happy with what DDlog is doing. It appears to be handling large-scale networks much better than the C implementation. The incremental processing allows for the network size not to affect the performance of ovn-northd as much.

On 7/18/19 1:18 PM, Mark Michelson wrote:
Hi, just wanted to send a follow-up and a test result.

I got everything up and running (yay!). So I set up a server and ran a sandbox test using ovn with traditional C-based ovn-northd and then ran the same test using ovn with DDlog-based ovn-northd.

The test script can be found here: https://gist.github.com/putnopvut/3df7156b6d44b81d75a598262c5e959b

The test sets up a network with 159 logical switches all connected to a single logical router. Each logical switch has 92 logical switch ports. For every 2 logical switch ports we add, we create a new port group, and ACLs are created for each port group. The important part of this test is that the final call to ovn-nbctl in each iteration of the loop uses --wait=sb. In other words, each new logical switch port we add has to wait for ovn-northd to process it completely before we can start adding the next logical switch port.

With traditional C-based ovn-northd:
[root@wsfd-netdev67 tutorial]# time ./scale-daemon-new-pg.sh

real    128m59.280s
user    1m27.601s
sys    2m0.986s

With DDlog-based ovn-northd:
[root@wsfd-netdev67 tutorial]# time ./scale-daemon-new-pg.sh

real    33m40.586s
user    1m26.976s
sys    1m59.784s

As you can see, the test takes about a quarter of the time to complete when running with DDlog. So far, it seems like a good improvement.

As a point of reference, if I remove the --wait=sb from the test, then the result is:

[root@wsfd-netdev67 tutorial]# time ./scale-daemon-new-pg.sh

real    3m59.696s
user    1m18.937s
sys    1m44.669s

This test was run with DDlog-based ovn-northd, but the result is nearly identical with C-based ovn-northd since the test no longer is bound by ovn-northd's performance.

I'm a bit surprised that with DDlog, adding --wait=sb to each iteration causes such a large difference in the overall test time (~4 minutes up to ~34 minutes).

My next step is going to be to plot the time of each individual iteration during the test. I'll share those results when I have them ready. I don't want to speculate on anything until I have that data to share.

Thanks,
Mark Michelson

On 7/15/19 12:34 PM, Leonid Ryzhyk wrote:
Hi Mark,

Thanks for the feedback. We will add the `--version` flag.

Yes, `stack install` merely copies the two DDlog executable to `~/.local/bin/` (or a custom path specified using `--local-bin-path <custom_path>`).

Also, just want to mention that we maintain up-to-date binary releases of DDlog at `https://github.com/vmware/differential-datalog/releases`, so most users should not need to install the Haskell tool stack and compile DDlog. When using binary releases, all you have to do is add the `ddlog/bin` directory to `$PATH`.

Leonid

On Mon, Jul 15, 2019 at 6:30 AM Mark Michelson <mmich...@redhat.com <mailto:mmich...@redhat.com>> wrote:

    I did install DDlog originally back in early December last year. So
    there are probably remnants of that still present. However, I did
    perform a `stack install` using an updated pull of master on Friday. So
    I guess the `stack install` didn't get rid of the old installation?

    Also, there's no `ddlog --version` or anything similar to see what
    version of DDlog is installed. That could be a nice feature to have in
    the near future.

    On 7/12/19 12:40 PM, Leonid Ryzhyk wrote:
     > Thanks for trying it out!  Sounds like you have a very old
    version of
     > ddlog. If you install from source, please make sure that you run
    `stack
     > install` and that there is no other version off ddlog in your
    path other
     > than the one created by `stack install`.
     >
     > Leonid
     >
     > On Fri, Jul 12, 2019, 9:05 AM Mark Michelson <mmich...@redhat.com
    <mailto:mmich...@redhat.com>
     > <mailto:mmich...@redhat.com <mailto:mmich...@redhat.com>>> wrote:
     >
     >     On 7/12/19 3:29 AM, Leonid Ryzhyk wrote:
     >      > Dear OVN developers,
     >      >
     >      > This is a brief update on the state of the DDlog port of
    ovn-northd.
     >      >
     >      > We completed the initial implementation of ovn-northd in DDlog
     >     few months
     >      > ago.  Justin kindly
     >      > helped to integrate it with OVN, so that it can be used as
    a drop-in
     >      > replacement for the C
     >      > version (and passes all the tests in the OVN test suite).     The DDlog
     >      > implementation does
     >      > not have any of the new features/improvements added in
    April 2019
     >     or later.
     >      >
     >      > ## Repository
     >      >
     >      > The code is in the `ddlog-dev` branch of the `ovn-org/ovn`
     >     repository:
     >      > https://github.com/ovn-org/ovn/tree/ddlog-dev
     >
     >     Hi Leonid,
     >
     >     I ran into an issue when attempting to build ovn-northd. I
    successfully
     >     installed DDLog, but then I encountered this issue when
    building OVN:
     >
     >     ddlog -i ovn/northd/ovn_northd.dl -L
     >     /home/putnopvut/differential-datalog/lib
     >     ddlog: Failed to parse input file: "ovn/northd/ovn_northd.dl"
    (line 94,
     >     column 5):
     >     unexpected "&"
     >     expecting "not", variable name, relation name, "var",
    expression or "."
     >
     >     The line in question looks like this:
     >
     >           &SwitchPort(.lsp = lsp, .sw = &sw),
     >
     >     Any idea what's gone wrong here?
     >
     >     Thanks,
     >
     >     Mark Michelson
     >
     >
     >
     >      >
     >      > ## Documentation
     >      >
     >      > Building and using ovn-northd-ddlog:
     >      >
     >
https://github.com/ovn-org/ovn/blob/ddlog-dev/ovn/northd/docs/design.md
     >      >
     >      > Debugging ovn-northd-ddlog:
     >      >
     >
https://github.com/ovn-org/ovn/blob/ddlog-dev/ovn/northd/docs/debugging.md
     >      >
     >      > ## Preliminary performance results
     >      >
     >      > Han Zhou kindly tested ovn-northd-ddlog with his OVN scale
    test
     >     and even
     >      > found a nasty
     >      > performance bug in the process (thanks, Han!).  He reports
    that DDlog
     >      > speeds up the test
     >      > by almost a factor of 10:
     >      >
     >      > - ddlog version: 7:39min
     >      > - C version: 67:47min
     >      >
     >      > This is great, and in fact profiling shows that there is still
     >     plenty of
     >      > space for
     >      > improvement.  He also reports a 10+ times increase in memory
     >     footprint:
     >      >
     >      > - ddlog version: 1944696KB
     >      > - C version: 147984KB
     >      >
     >      > Again, we are working on a number of optimizations, which
    should
     >     reduce
     >      > this overhead; although it will never be as low as C,
    since DDlog
     >      > fundamentally
     >      > needs to cache more state to enable fast incremental
    computation.
     >      >
     >      > Han also used DDlog's record&replay feature to capture all
    northd
     >      > transactions
     >      > performed by the scale test in a format that can be replayed
     >     against the
     >      > standalone DDlog executable without having to reproduce Han's
     >     OpenStack
     >      > setup.
     >      > The replay file is here: http://ryzhyk.net/replay.tgz
     >      >
     >      > Instructions for replaying this script:
     >      >
     >
https://github.com/ovn-org/ovn/blob/ddlog-dev/ovn/northd/docs/debugging.md#record-and-replay-ddlog-execution
     >      > The script will run for a few
     >      > minutes and finally print some profiling information,
    including the
     >      > breakdown of
     >      > DDlog's CPU and memory usage.
     >      >
     >      > ## Next steps
     >      >
     >      > We seek help from the OVN community in maintaining
     >     ovn-northd-ddlog.  The
     >      > first
     >      > step is to start porting new OVN features introduced in
    the last
     >     few months
     >      > to
     >      > DDlog.
     >      >
     >      > Leonid
     >
     >     Hi Leonid,
     >
     >     I attempted
     >
     >      > _______________________________________________
     >      > dev mailing list
     >      > d...@openvswitch.org <mailto:d...@openvswitch.org>
    <mailto:d...@openvswitch.org <mailto:d...@openvswitch.org>>
     >      > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
     >      >
     >



_______________________________________________
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Reply via email to