Public bug reported: OSP 14
3 controllers + 3 computes + dvr several vms in one compute with fip. Problem 1 : root@compute-2 heat-admin]# systemctl restart openvswitch fip conectivity with undercloud vm is lost and no recover conectivity with other computes is lost, but it is recovered restarting neutron openvswitch agent container root@compute-2 heat-admin]# systemctl restart openvswitch. Problem 2: After kill -9 "pid ovs switch" Sometimes the ovs switch in not restarted automatically Same problems that in the scenario1 [root@compute-2 heat-admin]# ps -ef | grep ovs root 10558 7292 0 12:09 ? 00:00:01 /usr/bin/python2 /bin/privsep-helper --config-file /usr/share/nova/nova-dist.conf --config-file /etc/nova/nova.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpATdioG/privsep.sock 42435 46886 46871 0 13:17 ? 00:00:00 /bin/bash /neutron_ovs_agent_launcher.sh openvsw+ 49054 1 1 13:20 ? 00:00:00 ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --user openvswitch:hugetlbfs --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach root 49217 17666 0 13:21 pts/0 00:00:00 grep --color=auto ovs [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# ps -ef | grep ovs root 10558 7292 0 12:09 ? 00:00:01 /usr/bin/python2 /bin/privsep-helper --config-file /usr/share/nova/nova-dist.conf --config-file /etc/nova/nova.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpATdioG/privsep.sock 42435 46886 46871 0 13:17 ? 00:00:00 /bin/bash /neutron_ovs_agent_launcher.sh openvsw+ 49054 1 0 13:20 ? 00:00:00 ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --user openvswitch:hugetlbfs --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach root 49421 17666 0 13:21 pts/0 00:00:00 grep --color=auto ovs [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# ps -ef | grep ovs root 10558 7292 0 12:09 ? 00:00:01 /usr/bin/python2 /bin/privsep-helper --config-file /usr/share/nova/nova-dist.conf --config-file /etc/nova/nova.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpATdioG/privsep.sock 42435 46886 46871 0 13:17 ? 00:00:00 /bin/bash /neutron_ovs_agent_launcher.sh openvsw+ 49054 1 0 13:20 ? 00:00:00 ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --user openvswitch:hugetlbfs --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach root 49423 17666 0 13:21 pts/0 00:00:00 grep --color=auto ovs [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# kill -9 49054 [root@compute-2 heat-admin]# ps -ef | grep ovs root 10558 7292 0 12:09 ? 00:00:01 /usr/bin/python2 /bin/privsep-helper --config-file /usr/share/nova/nova-dist.conf --config-file /etc/nova/nova.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpATdioG/privsep.sock 42435 46886 46871 0 13:17 ? 00:00:00 /bin/bash /neutron_ovs_agent_launcher.sh root 49610 17666 0 13:22 pts/0 00:00:00 grep --color=auto ovs [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# ps -ef | grep ovs root 10558 7292 0 12:09 ? 00:00:01 /usr/bin/python2 /bin/privsep-helper --config-file /usr/share/nova/nova-dist.conf --config-file /etc/nova/nova.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpATdioG/privsep.sock 42435 46886 46871 0 13:17 ? 00:00:00 /bin/bash /neutron_ovs_agent_launcher.sh root 49628 17666 0 13:22 pts/0 00:00:00 grep --color=auto ovs [root@compute-2 heat-admin]# date Fri Nov 23 13:22:22 UTC 2018 [root@compute-2 heat-admin]# ps -ef | grep ovs root 10558 7292 0 12:09 ? 00:00:01 /usr/bin/python2 /bin/privsep-helper --config-file /usr/share/nova/nova-dist.conf --config-file /etc/nova/nova.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpATdioG/privsep.sock 42435 46886 46871 0 13:17 ? 00:00:00 /bin/bash /neutron_ovs_agent_launcher.sh root 49788 17666 0 13:22 pts/0 00:00:00 grep --color=auto ovs [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# ps -ef | grep ovs root 10558 7292 0 12:09 ? 00:00:01 /usr/bin/python2 /bin/privsep-helper --config-file /usr/share/nova/nova-dist.conf --config-file /etc/nova/nova.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpATdioG/privsep.sock 42435 46886 46871 0 13:17 ? 00:00:00 /bin/bash /neutron_ovs_agent_launcher.sh root 49790 17666 0 13:22 pts/0 00:00:00 grep --color=auto ovs overcloud) [stack@undercloud-0 ~]$ openstack versions show +-------------+----------------+---------+------------+-----------------------------------+------------------+------------------+ | Region Name | Service Type | Version | Status | Endpoint | Min Microversion | Max Microversion | +-------------+----------------+---------+------------+-----------------------------------+------------------+------------------+ | regionOne | block-storage | 2.0 | DEPRECATED | http://10.0.0.101:8776/v2/ | None | None | | regionOne | block-storage | 3.0 | CURRENT | http://10.0.0.101:8776/v3/ | 3.0 | 3.55 | | regionOne | placement | None | CURRENT | http://10.0.0.101:8778/placement/ | None | None | | regionOne | network | 2.0 | CURRENT | http://10.0.0.101:9696/v2.0/ | None | None | | regionOne | alarm | 2.0 | CURRENT | http://10.0.0.101:8042/v2 | None | None | | regionOne | cloudformation | 1.0 | CURRENT | http://10.0.0.101:8000/v1/ | None | None | | regionOne | event | 2.0 | CURRENT | http://10.0.0.101:8977/v2 | None | None | | regionOne | orchestration | 1.0 | CURRENT | http://10.0.0.101:8004/v1/ | None | None | | regionOne | object-store | 1.0 | CURRENT | http://10.0.0.101:8080/v1/ | None | None | | regionOne | compute | 2.0 | SUPPORTED | http://10.0.0.101:8774/v2/ | None | None | | regionOne | compute | 2.1 | CURRENT | http://10.0.0.101:8774/v2.1/ | 2.1 | 2.65 | | regionOne | image | 2.0 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None | | regionOne | image | 2.1 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None | | regionOne | image | 2.2 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None | | regionOne | image | 2.3 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None | | regionOne | image | 2.4 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None | | regionOne | image | 2.5 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None | | regionOne | image | 2.6 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None | | regionOne | image | 2.7 | CURRENT | http://10.0.0.101:9292/v2/ | None | None | | regionOne | metric | 1.0 | CURRENT | http://10.0.0.101:8041/v1/ | None | None | | regionOne | identity | 3.10 | CURRENT | http://10.0.0.101:5000/v3/ | None | None | +-------------+----------------+---------+------------+-----------------------------------+------------------+------------------+ (overcloud) [stack@undercloud-0 ~]$ cat /etc/re redhat-lsb/ redhat-release request-key.conf request-key.d/ resolv.conf (overcloud) [stack@undercloud-0 ~]$ cat /etc/re redhat-lsb/ redhat-release request-key.conf request-key.d/ resolv.conf (overcloud) [stack@undercloud-0 ~]$ cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.6 (Maipo) ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1804842 Title: When kill(sometines doesn't restart) the ovs switch or restart it in the compute nodes vm conectivity is lost Status in neutron: New Bug description: OSP 14 3 controllers + 3 computes + dvr several vms in one compute with fip. Problem 1 : root@compute-2 heat-admin]# systemctl restart openvswitch fip conectivity with undercloud vm is lost and no recover conectivity with other computes is lost, but it is recovered restarting neutron openvswitch agent container root@compute-2 heat-admin]# systemctl restart openvswitch. Problem 2: After kill -9 "pid ovs switch" Sometimes the ovs switch in not restarted automatically Same problems that in the scenario1 [root@compute-2 heat-admin]# ps -ef | grep ovs root 10558 7292 0 12:09 ? 00:00:01 /usr/bin/python2 /bin/privsep-helper --config-file /usr/share/nova/nova-dist.conf --config-file /etc/nova/nova.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpATdioG/privsep.sock 42435 46886 46871 0 13:17 ? 00:00:00 /bin/bash /neutron_ovs_agent_launcher.sh openvsw+ 49054 1 1 13:20 ? 00:00:00 ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --user openvswitch:hugetlbfs --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach root 49217 17666 0 13:21 pts/0 00:00:00 grep --color=auto ovs [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# ps -ef | grep ovs root 10558 7292 0 12:09 ? 00:00:01 /usr/bin/python2 /bin/privsep-helper --config-file /usr/share/nova/nova-dist.conf --config-file /etc/nova/nova.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpATdioG/privsep.sock 42435 46886 46871 0 13:17 ? 00:00:00 /bin/bash /neutron_ovs_agent_launcher.sh openvsw+ 49054 1 0 13:20 ? 00:00:00 ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --user openvswitch:hugetlbfs --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach root 49421 17666 0 13:21 pts/0 00:00:00 grep --color=auto ovs [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# ps -ef | grep ovs root 10558 7292 0 12:09 ? 00:00:01 /usr/bin/python2 /bin/privsep-helper --config-file /usr/share/nova/nova-dist.conf --config-file /etc/nova/nova.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpATdioG/privsep.sock 42435 46886 46871 0 13:17 ? 00:00:00 /bin/bash /neutron_ovs_agent_launcher.sh openvsw+ 49054 1 0 13:20 ? 00:00:00 ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --user openvswitch:hugetlbfs --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach root 49423 17666 0 13:21 pts/0 00:00:00 grep --color=auto ovs [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# kill -9 49054 [root@compute-2 heat-admin]# ps -ef | grep ovs root 10558 7292 0 12:09 ? 00:00:01 /usr/bin/python2 /bin/privsep-helper --config-file /usr/share/nova/nova-dist.conf --config-file /etc/nova/nova.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpATdioG/privsep.sock 42435 46886 46871 0 13:17 ? 00:00:00 /bin/bash /neutron_ovs_agent_launcher.sh root 49610 17666 0 13:22 pts/0 00:00:00 grep --color=auto ovs [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# ps -ef | grep ovs root 10558 7292 0 12:09 ? 00:00:01 /usr/bin/python2 /bin/privsep-helper --config-file /usr/share/nova/nova-dist.conf --config-file /etc/nova/nova.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpATdioG/privsep.sock 42435 46886 46871 0 13:17 ? 00:00:00 /bin/bash /neutron_ovs_agent_launcher.sh root 49628 17666 0 13:22 pts/0 00:00:00 grep --color=auto ovs [root@compute-2 heat-admin]# date Fri Nov 23 13:22:22 UTC 2018 [root@compute-2 heat-admin]# ps -ef | grep ovs root 10558 7292 0 12:09 ? 00:00:01 /usr/bin/python2 /bin/privsep-helper --config-file /usr/share/nova/nova-dist.conf --config-file /etc/nova/nova.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpATdioG/privsep.sock 42435 46886 46871 0 13:17 ? 00:00:00 /bin/bash /neutron_ovs_agent_launcher.sh root 49788 17666 0 13:22 pts/0 00:00:00 grep --color=auto ovs [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# [root@compute-2 heat-admin]# ps -ef | grep ovs root 10558 7292 0 12:09 ? 00:00:01 /usr/bin/python2 /bin/privsep-helper --config-file /usr/share/nova/nova-dist.conf --config-file /etc/nova/nova.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpATdioG/privsep.sock 42435 46886 46871 0 13:17 ? 00:00:00 /bin/bash /neutron_ovs_agent_launcher.sh root 49790 17666 0 13:22 pts/0 00:00:00 grep --color=auto ovs overcloud) [stack@undercloud-0 ~]$ openstack versions show +-------------+----------------+---------+------------+-----------------------------------+------------------+------------------+ | Region Name | Service Type | Version | Status | Endpoint | Min Microversion | Max Microversion | +-------------+----------------+---------+------------+-----------------------------------+------------------+------------------+ | regionOne | block-storage | 2.0 | DEPRECATED | http://10.0.0.101:8776/v2/ | None | None | | regionOne | block-storage | 3.0 | CURRENT | http://10.0.0.101:8776/v3/ | 3.0 | 3.55 | | regionOne | placement | None | CURRENT | http://10.0.0.101:8778/placement/ | None | None | | regionOne | network | 2.0 | CURRENT | http://10.0.0.101:9696/v2.0/ | None | None | | regionOne | alarm | 2.0 | CURRENT | http://10.0.0.101:8042/v2 | None | None | | regionOne | cloudformation | 1.0 | CURRENT | http://10.0.0.101:8000/v1/ | None | None | | regionOne | event | 2.0 | CURRENT | http://10.0.0.101:8977/v2 | None | None | | regionOne | orchestration | 1.0 | CURRENT | http://10.0.0.101:8004/v1/ | None | None | | regionOne | object-store | 1.0 | CURRENT | http://10.0.0.101:8080/v1/ | None | None | | regionOne | compute | 2.0 | SUPPORTED | http://10.0.0.101:8774/v2/ | None | None | | regionOne | compute | 2.1 | CURRENT | http://10.0.0.101:8774/v2.1/ | 2.1 | 2.65 | | regionOne | image | 2.0 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None | | regionOne | image | 2.1 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None | | regionOne | image | 2.2 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None | | regionOne | image | 2.3 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None | | regionOne | image | 2.4 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None | | regionOne | image | 2.5 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None | | regionOne | image | 2.6 | SUPPORTED | http://10.0.0.101:9292/v2/ | None | None | | regionOne | image | 2.7 | CURRENT | http://10.0.0.101:9292/v2/ | None | None | | regionOne | metric | 1.0 | CURRENT | http://10.0.0.101:8041/v1/ | None | None | | regionOne | identity | 3.10 | CURRENT | http://10.0.0.101:5000/v3/ | None | None | +-------------+----------------+---------+------------+-----------------------------------+------------------+------------------+ (overcloud) [stack@undercloud-0 ~]$ cat /etc/re redhat-lsb/ redhat-release request-key.conf request-key.d/ resolv.conf (overcloud) [stack@undercloud-0 ~]$ cat /etc/re redhat-lsb/ redhat-release request-key.conf request-key.d/ resolv.conf (overcloud) [stack@undercloud-0 ~]$ cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.6 (Maipo) To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1804842/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp