After upgrading from Open vSwitch 3.1.2-4 to 3.2.0-2 we've been consistently seeing new OVS crashes when setting QoS on ports. Both packages were taken from the Debian distribution ( https://packages.debian.org/source/sid/openvswitch) we're running on. From the core dump we're seeing the following backtrace:
# gdb --batch -ex bt /usr/sbin/ovs-vswitchd /core [New LWP 67669] [New LWP 67682] [New LWP 67681] [New LWP 67671] [New LWP 67679] [New LWP 67680] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `ovs-vswitchd unix:/tmp/ovs/db.sock -vconsole:emer -vsyslog:err -vfile:info --ml'. Program terminated with signal SIGABRT, Aborted. #0 0x00007fcacecbb0fc in ?? () from /lib/x86_64-linux-gnu/libc.so.6 [Current thread is 1 (Thread 0x7fcacf4cfa80 (LWP 67669))] #0 0x00007fcacecbb0fc in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007fcacec6d472 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00007fcacec574b2 in abort () from /lib/x86_64-linux-gnu/libc.so.6 #3 0x0000560787952c7e in ovs_abort_valist (err_no=<optimized out>, format=<optimized out>, args=args@entry=0x7ffd14c3bce0) at ../lib/util.c:447 #4 0x0000560787952d14 in ovs_abort (err_no=<optimized out>, format=format@entry=0x5607879f3ec7 "%s: pthread_%s_%s failed") at ../lib/util.c:439 #5 0x000056078791ee11 in ovs_mutex_lock_at (l_=l_@entry=0x56078934c6c8, where=where@entry=0x5607879f864b "../lib/netdev-linux.c:2575") at ../lib/ovs-thread.c:76 #6 0x000056078796d03d in netdev_linux_get_speed (netdev_=0x56078934c640, current=0x7ffd14c3be64, max=0x7ffd14c3be1c) at ../lib/netdev-linux.c:2575 #7 0x00005607878c04f3 in netdev_get_speed (netdev=netdev@entry=0x56078934c640, current=current@entry=0x7ffd14c3be64, max=0x7ffd14c3be1c, max@entry=0x0) at ../lib/netdev.c:1175 #8 0x0000560787968d67 in htb_parse_qdisc_details__ (netdev=netdev@entry=0x56078934c640, details=details@entry=0x56078934a880, hc=hc@entry=0x7ffd14c3beb0) at ../lib/netdev-linux.c:4804 #9 0x00005607879755da in htb_tc_install (details=0x56078934a880, netdev=0x56078934c640) at ../lib/netdev-linux.c:4883 #10 htb_tc_install (netdev=0x56078934c640, details=0x56078934a880) at ../lib/netdev-linux.c:4875 #11 0x0000560787974937 in netdev_linux_set_qos (netdev_=0x56078934c640, type=<optimized out>, details=0x56078934a880) at ../lib/netdev-linux.c:3054 #12 0x0000560787814ea5 in iface_configure_qos (qos=0x56078934a780, iface=0x560789349fd0) at ../vswitchd/bridge.c:4845 #13 bridge_reconfigure (ovs_cfg=ovs_cfg@entry=0x5607892dbf90) at ../vswitchd/bridge.c:928 #14 0x0000560787817f7d in bridge_run () at ../vswitchd/bridge.c:3321 #15 0x000056078780d205 in main (argc=<optimized out>, argv=<optimized out>) at ../vswitchd/ovs-vswitchd.c:130 A shell script to reproduce the issue is: #!/bin/sh apt-get install openvswitch-{common,switch}{,-dbgsym} # Don't need it running on the system systemctl stop openvswitch-switch set -e cleanup() { ip link del veth0 rm /tmp/ovs/conf.db } trap cleanup EXIT # Setup our environment ip link add veth0 type veth peer veth1 mkdir -p /tmp/ovs export OVS_RUNDIR=/tmp/ovs export OVS_LOGDIR=/tmp/ovs export OVS_DBDIR=/tmp/ovs /usr/share/openvswitch/scripts/ovs-ctl start ovs-vsctl add-br demo ovs-vsctl add-port demo veth1 # Make it crash ovs-vsctl set Port veth1 qos=@qos \ -- --id=@qos create QoS type=linux-htb queues:1=@high queues:2=@low \ -- --id=@high create Queue other-config:priority=1 other-config:min-rate= 0.1 \ -- --id=@low create Queue other-config:priority=6 other-config:min-rate= 0.05 We built the reproduction script based on speculation that https://github.com/openvswitch/ovs/commit/6240c0b4c80ea3d8dd1bf77526b04b55742de2ce is related to the crash. Notably we don't seem to run into the problem when we pass in a specific maximum bandwidth instead of relying on the interface's maximum bandwidth. Sincerely, Daryl Wang
_______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss