Hello Experts, I am doing stress with k8s cluster with ovn, one thing I am seeing is that when raft nodes got update for large data in short time from ovn-northd, 3 raft nodes will trigger voting and leader role switched from one node to another.
>From ovn-northd side, I can see ovn-northd will trigger the BACKOFF, RECONNECT... Since ovn-northd only connect to NB/SB leader only and how can we make ovn-northd more available in most of the time? Is it possible to make ovn-northd have established connections to all raft nodes to avoid the reconnect mechanism? Since the backoff time 8s is not configurable for now. Test logs: 2020-04-29T17:03:08.296Z|41861|ovsdb_idl|INFO|tcp:10.0.2.152:6642: clustered database server is not cluster leader; trying another server 2020-04-29T17:03:08.296Z|41862|reconnect|DBG|tcp:10.0.2.152:6642: entering RECONNECT 2020-04-29T17:03:08.304Z|41863|reconnect|DBG|tcp:10.0.2.152:6642: entering BACKOFF 2020-04-29T17:03:09.708Z|41867|coverage|INFO|Dropped 2 log messages in last 78 seconds (most recently, 71 seconds ago) due to excessive rate 2020-04-29T17:03:09.708Z|41868|coverage|INFO|Skipping details of duplicate event coverage for hash=ceada91f 2020-04-29T17:03:16.304Z|41869|reconnect|DBG|tcp:10.0.2.153:6642: entering CONNECTING 2020-04-29T17:03:16.308Z|41870|reconnect|INFO|tcp:10.0.2.153:6642: connected 2020-04-29T17:03:16.308Z|41871|reconnect|DBG|tcp:10.0.2.153:6642: entering ACTIVE 2020-04-29T17:03:16.308Z|41872|ovn_northd|INFO|ovn-northd lock lost. This ovn-northd instance is now on standby. 2020-04-29T17:03:16.309Z|41873|ovn_northd|INFO|ovn-northd lock acquired. This ovn-northd instance is now active. 2020-04-29T17:03:16.311Z|41874|ovsdb_idl|INFO|tcp:10.0.2.153:6642: clustered database server is disconnected from cluster; trying another server 2020-04-29T17:03:16.311Z|41875|reconnect|DBG|tcp:10.0.2.153:6642: entering RECONNECT 2020-04-29T17:03:16.312Z|41876|reconnect|DBG|tcp:10.0.2.153:6642: entering BACKOFF 2020-04-29T17:03:24.316Z|41877|reconnect|DBG|tcp:10.0.2.151:6642: entering CONNECTING 2020-04-29T17:03:24.321Z|41878|reconnect|INFO|tcp:10.0.2.151:6642: connected 2020-04-29T17:03:24.321Z|41879|reconnect|DBG|tcp:10.0.2.151:6642: entering ACTIVE 2020-04-29T17:03:24.321Z|41880|ovn_northd|INFO|ovn-northd lock lost. This ovn-northd instance is now on standby. 2020-04-29T17:03:24.354Z|41881|ovn_northd|INFO|ovn-northd lock acquired. This ovn-northd instance is now active. 2020-04-29T17:03:24.358Z|41882|ovsdb_idl|INFO|tcp:10.0.2.151:6642: clustered database server is not cluster leader; trying another server 2020-04-29T17:03:24.358Z|41883|reconnect|DBG|tcp:10.0.2.151:6642: entering RECONNECT 2020-04-29T17:03:24.360Z|41884|reconnect|DBG|tcp:10.0.2.151:6642: entering BACKOFF 2020-04-29T17:03:32.367Z|41885|reconnect|DBG|tcp:10.0.2.152:6642: entering CONNECTING 2020-04-29T17:03:32.372Z|41886|reconnect|INFO|tcp:10.0.2.152:6642: connected 2020-04-29T17:03:32.372Z|41887|reconnect|DBG|tcp:10.0.2.152:6642: entering ACTIVE 2020-04-29T17:03:32.372Z|41888|ovn_northd|INFO|ovn-northd lock lost. This ovn-northd instance is now on standby. 2020-04-29T17:03:32.373Z|41889|ovn_northd|INFO|ovn-northd lock acquired. This ovn-northd instance is now active. 2020-04-29T17:03:32.376Z|41890|ovsdb_idl|INFO|tcp:10.0.2.152:6642: clustered database server is not cluster leader; trying another server 2020-04-29T17:03:32.376Z|41891|reconnect|DBG|tcp:10.0.2.152:6642: entering RECONNECT 2020-04-29T17:03:32.378Z|41892|reconnect|DBG|tcp:10.0.2.152:6642: entering BACKOFF 2020-04-29T17:03:40.381Z|41893|reconnect|DBG|tcp:10.0.2.153:6642: entering CONNECTING 2020-04-29T17:03:40.385Z|41894|reconnect|INFO|tcp:10.0.2.153:6642: connected 2020-04-29T17:03:40.385Z|41895|reconnect|DBG|tcp:10.0.2.153:6642: entering ACTIVE 2020-04-29T17:03:40.385Z|41896|ovn_northd|INFO|ovn-northd lock lost. This ovn-northd instance is now on standby. 2020-04-29T17:03:40.385Z|41897|ovn_northd|INFO|ovn-northd lock acquired. This ovn-northd instance is now active. -- Winson
_______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss