Re: [ovs-dev] [ovs-discuss] ovsdb-server core dump and ovsdb corruption using raft cluster

2018-08-06 Thread Ben Pfaff
With Guru's help, I believe I have fixed it: https://patchwork.ozlabs.org/patch/954247/ On Wed, Aug 01, 2018 at 11:46:38AM -0700, Guru Shetty wrote: > I was able to reproduce it. I will work with Ben to get this fixed. > > On 26 July 2018 at 23:14, Girish Moodalbail wrote: > > > Hello B

Re: [ovs-dev] [ovs-discuss] ovsdb-server core dump and ovsdb corruption using raft cluster

2018-08-01 Thread Guru Shetty
I was able to reproduce it. I will work with Ben to get this fixed. On 26 July 2018 at 23:14, Girish Moodalbail wrote: > Hello Ben, > > Sorry, got distracted with something else at work. I am still able to > reproduce the issue, and this is what I have and what I did > (if you need the core, let

Re: [ovs-dev] [ovs-discuss] ovsdb-server core dump and ovsdb corruption using raft cluster

2018-07-31 Thread Girish Moodalbail
Hello Ben/Guru, Wanted to check if you were able to reproduce the issue on your end, and whether you guys needed any more info from me. If you guys have any patch, then we are more than happy to verify it. regards, ~Girish On Thu, Jul 26, 2018 at 11:14 PM, Girish Moodalbail wrote: > Hello Ben,

Re: [ovs-dev] [ovs-discuss] ovsdb-server core dump and ovsdb corruption using raft cluster

2018-07-26 Thread Girish Moodalbail
Hello Ben, Sorry, got distracted with something else at work. I am still able to reproduce the issue, and this is what I have and what I did (if you need the core, let me know and I can share it with you) - 3-cluster RAFT setup in Ubuntu VM (2 VCPUs with 8GB RAM) $ uname -r Linux u1804-HVM-do

Re: [ovs-dev] [ovs-discuss] ovsdb-server core dump and ovsdb corruption using raft cluster

2018-07-25 Thread Ben Pfaff
On Wed, Jul 18, 2018 at 10:48:08AM -0700, Girish Moodalbail wrote: > Hello all, > > We are able to reproduce this issue on OVS 2.9.2 at will. The OVSDB NB > server or OVSDB SB server dumps core while it is trying to compact the > database. > > You can reproduce the issue by using: > > root@u1804

Re: [ovs-dev] [ovs-discuss] ovsdb-server core dump and ovsdb corruption using raft cluster

2018-07-25 Thread Girish Moodalbail
On one occasion, we saw ovsdb-server crash with a similar stack (i.e., at the same location but different code path). This time, the raft node (leader) was trying to reply to a follower with a snapshot of its database (I am guessing from the stack trace, so I might be way off here). See below, for

Re: [ovs-dev] [ovs-discuss] ovsdb-server core dump and ovsdb corruption using raft cluster

2018-07-24 Thread aginwala
Hi: Glad to see more people picking up on raft testing. Just to add on, you can also refer to https://mail.openvswitch.org/pipermail/ovs-dev/2018-May/347765.html and https://mail.openvswitch.org/pipermail/ovs-dev/2018-April/346375.html where there are couple of suggestions given by Ben too. See

Re: [ovs-dev] [ovs-discuss] ovsdb-server core dump and ovsdb corruption using raft cluster

2018-07-24 Thread Yifeng Sun
My apologize, the patch has some issue. I need to dig further. Yifeng On Tue, Jul 24, 2018 at 1:40 PM, Yifeng Sun wrote: > Hi Yun and Girish, > > I submitted a patch, do you mind testing and reviewing it? Thanks. > > [PATCH] dynamic-string: Fix a bug that leads to assertion fail > > diff --git

Re: [ovs-dev] [ovs-discuss] ovsdb-server core dump and ovsdb corruption using raft cluster

2018-07-24 Thread Yifeng Sun
Hi Yun and Girish, I submitted a patch, do you mind testing and reviewing it? Thanks. [PATCH] dynamic-string: Fix a bug that leads to assertion fail diff --git a/lib/dynamic-string.c b/lib/dynamic-string.c index 6f7b610a9908..4564e420544d 100644 --- a/lib/dynamic-string.c +++ b/lib/dynamic-strin

Re: [ovs-dev] [ovs-discuss] ovsdb-server core dump and ovsdb corruption using raft cluster

2018-07-18 Thread Girish Moodalbail
Hello all, We are able to reproduce this issue on OVS 2.9.2 at will. The OVSDB NB server or OVSDB SB server dumps core while it is trying to compact the database. You can reproduce the issue by using: root@u1804-HVM-domU:/var/crash# ovs-appctl -t /var/run/openvswitch/ovnsb_db.ctl ovsdb-server/co