[Bug 633392] Re: Bridged Guests losing network connectivity
Serge, Something different happens with this. Now, I get the network instability that I saw before with SSH sessions exiting with Broken Pipe, and the netcat session you showed in the link just hangs. If I try to re-SSH into the box, however, it works again. It looks like the same thing that was happening, only the ARP entry doesn't change (i'm assuming it's because there's no guest) after it does. -- Bridged Guests losing network connectivity https://bugs.launchpad.net/bugs/633392 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 633392] Re: Bridged Guests losing network connectivity
Serge, It takes anywhere from maybe 10-30 minutes during the tests I ran, but it always happens. You think this is an issue with the kernel? -- Bridged Guests losing network connectivity https://bugs.launchpad.net/bugs/633392 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 633392] Re: Bridged Guests losing network connectivity
Serge, No luck, still having the same issues with the new kernel. -- Bridged Guests losing network connectivity https://bugs.launchpad.net/bugs/633392 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 633392] [NEW] Bridged Guests losing network connectivity
Public bug reported: Binary package hint: qemu-kvm Opening at the request of Serge Hallyn. Running 10.04 with KVM and Libvirt, starting VMs with virsh start domain I've tested with OpenBSD 4.7 and Debian Lenny 5.0.5, both of them are having this issue. I notice that networking with the bridge works fine at first for my guests, but after a while, the bridge stops functioning, and I can't reach it via SSH, nor can I ping/ssh out. Also, when this happens, the ARP entry on the firewall I'm using to SSH to the guest changes from the Guest's MAC to the Host's MAC address. On OpenBSD, if I run the /etc/netstart script to re-initialize the interface, the bridge starts functioning again, as if I restarted the VM, but quickly fails again (10-20 minutes). I have to reboot the Debian machine to get networking functioning again. Here is my /etc/network/interfaces on the host: # The loopback network interface auto lo iface lo inet loopback # The primary network interface auto eth0 iface eth0 inet dhcp auto br0 iface br0 inet static address 192.168.8.166 netmask 255.255.252.0 network 192.168.8.0 broadcast 192.168.11.255 gateway 192.168.8.1 bridge_ports eth1 bridge_fd 9 bridge_hello 2 bridge_maxage 12 bridge_stp off And my /etc/libvirt/qemu/networks/autostart/default.xml: network namedefault/name bridge name=virbr%d / forward/ ip address=192.168.122.1 netmask=255.255.255.0 dhcp range start=192.168.122.2 end=192.168.122.254 / /dhcp /ip /network # iptables -L Chain INPUT (policy ACCEPT) target prot opt source destination ACCEPT udp -- anywhere anywhereudp dpt:domain ACCEPT tcp -- anywhere anywheretcp dpt:domain ACCEPT udp -- anywhere anywhereudp dpt:bootps ACCEPT tcp -- anywhere anywheretcp dpt:bootps Chain FORWARD (policy ACCEPT) target prot opt source destination ACCEPT all -- anywhere 192.168.122.0/24state RELATED,ESTABLISHED ACCEPT all -- 192.168.122.0/24 anywhere ACCEPT all -- anywhere anywhere REJECT all -- anywhere anywherereject-with icmp-port-unreachable REJECT all -- anywhere anywherereject-with icmp-port-unreachable Chain OUTPUT (policy ACCEPT) target prot opt source destination # netstat -nr Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0 192.168.8.0 0.0.0.0 255.255.252.0 U 0 0 0 br0 0.0.0.0 192.168.1.1 0.0.0.0 UG0 0 0 eth0 0.0.0.0 192.168.8.1 0.0.0.0 UG0 0 0 br0 # brctl show bridge name bridge id STP enabled interfaces br0 8000.00188b3e8dc0 no eth1 vnet0 virbr0 8000. yes IFCONFIG -- With Guest OS Running # ifconfig -a br0 Link encap:Ethernet HWaddr 00:18:8b:3e:8d:c0 inet addr:192.168.8.166 Bcast:192.168.11.255 Mask:255.255.252.0 inet6 addr: fe80::218:8bff:fe3e:8dc0/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:236817 errors:0 dropped:0 overruns:0 frame:0 TX packets:2654 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:17692813 (17.6 MB) TX bytes:362935 (362.9 KB) eth0 Link encap:Ethernet HWaddr 00:18:8b:3e:8d:be inet addr:192.168.1.151 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::218:8bff:fe3e:8dbe/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:76541 errors:0 dropped:0 overruns:0 frame:0 TX packets:62301 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:10233726 (10.2 MB) TX bytes:21694280 (21.6 MB) Interrupt:16 Memory:f800-f8012800 eth1 Link encap:Ethernet HWaddr 00:18:8b:3e:8d:c0 inet6 addr: fe80::218:8bff:fe3e:8dc0/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:241821 errors:0 dropped:0 overruns:0 frame:0 TX packets:10075 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:22612242 (22.6 MB) TX bytes:1391493 (1.3 MB) Interrupt:16 Memory:f400-f4012800 loLink encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1
[Bug 584048] Re: kvm images losing connectivity w/bridged network
Serge: I opened the new bug report as you requested, here: https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/633392 -- kvm images losing connectivity w/bridged network https://bugs.launchpad.net/bugs/584048 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 584048] Re: kvm images losing connectivity w/bridged network
Serge, I'm testing the new packages now, I'll let you know what I find. -- kvm images losing connectivity w/bridged network https://bugs.launchpad.net/bugs/584048 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 584048] Re: kvm images losing connectivity w/bridged network
Serge, No dice; the same problem occurs. Curiously, running the /etc/netstart script in OpenBSD works to restore connectivity, which fails again after a bit. -- kvm images losing connectivity w/bridged network https://bugs.launchpad.net/bugs/584048 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 584048] Re: kvm images losing connectivity w/bridged network
I'll go ahead and test the package when I have a minute here; I've multiple other servers that need attention. -- kvm images losing connectivity w/bridged network https://bugs.launchpad.net/bugs/584048 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 584048] Re: kvm images losing connectivity w/bridged network
I am also not modifying the bridge at all. Where my problem differs from Frank's, however, is my connectivity is lost even if there is steady network traffic to or from the guest. -- kvm images losing connectivity w/bridged network https://bugs.launchpad.net/bugs/584048 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 584048] Re: kvm images losing connectivity w/bridged network
If you don't mind, once you verify the host's /proc/net/arp entries before and after it goes bad, I'm going to whip up a recipe for trying to reproduce this without kvm, so we can reclassify it approrpriately. I don't see the ARP entry for the guest at all on the host machine before the networking fails, it's just not there; even after pinging the host machine from the guest, and vice-versa. It doesn't make a difference whether or not STP is enabled on the bridge. BEFORE NETWORKING FAILS: # cat /proc/net/arp IP address HW type Flags HW addressMask Device 192.168.1.1540x1 0x2 00:30:1b:be:63:4f *eth0 192.168.8.1 0x1 0x2 00:15:17:dc:2f:a2 *br0 AFTER NETWORKING FAILS: # cat /proc/net/arp IP address HW type Flags HW addressMask Device 192.168.1.1540x1 0x2 00:30:1b:be:63:4f *eth0 192.168.8.1 0x1 0x2 00:15:17:dc:2f:a2 *br0 -- kvm images losing connectivity w/bridged network https://bugs.launchpad.net/bugs/584048 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 584048] Re: kvm images losing connectivity w/bridged network
I'm also having this problem. Running 10.04 with KVM and Libvirt, starting VMs with virsh start domain I've tested with OpenBSD 4.7 and Debian Lenny 5.0.5, both of them are having this issue. I notice that networking with the bridge works fine at first for my guests, but after a while, the bridge stops functioning, and I can't reach it via SSH, nor can I ping/ssh out. Also, when this happens, the ARP entry on the firewall I'm using to SSH to the guest changes from the Guest's MAC to the Host's MAC address. On OpenBSD, if I run the /etc/netstart script to re-initialize the interface, the bridge starts functioning again, as if I restarted the VM, but quickly fails again (10-20 minutes). I have to reboot the Debian machine to get networking functioning again. Here is my /etc/network/interfaces on the host: # The loopback network interface auto lo iface lo inet loopback # The primary network interface auto eth0 iface eth0 inet dhcp auto br0 iface br0 inet static address 192.168.8.166 netmask 255.255.252.0 network 192.168.8.0 broadcast 192.168.11.255 gateway 192.168.8.1 bridge_ports eth1 bridge_fd 9 bridge_hello 2 bridge_maxage 12 bridge_stp off And my /etc/libvirt/qemu/networks/autostart/default.xml: network namedefault/name bridge name=virbr%d / forward/ ip address=192.168.122.1 netmask=255.255.255.0 dhcp range start=192.168.122.2 end=192.168.122.254 / /dhcp /ip /network -- kvm images losing connectivity w/bridged network https://bugs.launchpad.net/bugs/584048 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 584048] Re: kvm images losing connectivity w/bridged network
General instability exists with this bug as well. For example, SSHing to the VM: $ ssh r...@192.168.8.166 r...@192.168.8.166's password: Linux deb64 2.6.26-2-amd64 #1 SMP Sun Jun 20 20:16:30 UTC 2010 x86_64 The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. Last login: Thu Jul 22 15:57:28 2010 from 192.168.8.1 deb64:~# /// After waiting a few minutes, and pressing Enter... deb64:~# Write failed: Broken pipe /// Then, trying to re-SSH into the server... $ ssh r...@192.168.8.166 r...@192.168.8.166's password: Read from socket failed: Connection reset by peer $ ssh r...@192.168.8.166 ssh: connect to host 192.168.8.166 port 22: Connection refused This is when the ARP entry changes on the firewall I'm SSHing from, and the bridge connectivity freezes, and I have to restart the VM. -- kvm images losing connectivity w/bridged network https://bugs.launchpad.net/bugs/584048 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 584048] Re: kvm images losing connectivity w/bridged network
So now is this going through the firewall? This is from the firewall. What kind of firewall? Do other hosts inside the firewall have the right ARP entry for the guest? What about the host itself? It's a simple i386 OpenBSD 4.7 box with PF enabled. All traffic to and from the Host and the VM is unfiltered. I'll have to check whether the ARP entry changes for the host, but other hosts show the ARP entry change as well. Note that this sounds like a different problem from what others are reporting. Others are saying that their bridge hangs when new devices are added or removed. Note that the OP has said nothing about adding or removing new devices. -- kvm images losing connectivity w/bridged network https://bugs.launchpad.net/bugs/584048 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
