Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
After moving to 3.5 kernel I haven't seen it, even at 3x traffic which used to cause it On Mar 25, 2014 9:06 PM, Dewey McDonnell de...@poslinker.com wrote: My web search reveals this problem has been around for years and most posts conclude that the problem continues. After solving my tap0 TX packets dropped problem, I find that my VM network has random freezes a few times each day. Many bloggers on this subject say this VM network freeze is difficult to reproduce. Not for me! I can cause a network freeze on my VM in a heartbeat. Like many others, all I need to do is start a data transfer over the bridge (FTP 2Kb file or larger). When the freeze happens, ifconfig tap0 says overruns:1. This is QEMU version 1.6.2 without KVM on CentOS 6.5 version 2.6.32-431. -- You received this bug notification because you are subscribed to the bug report. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network Status in OpenStack Compute (Nova): Invalid Status in qemu-kvm package in Ubuntu: Fix Released Status in qemu-kvm source package in Precise: Fix Released Bug description: = SRU Justification: 1. Impact: networking breaks after awhile in kvm guests using virtio networking 2. Development fix: The bug was fixed upstream and the fix picked up in a new merge. 3. Stable fix: 3 virtio patches are cherrypicked from upstream: a821ce5 virtio: order index/descriptor reads 92045d8 virtio: add missing mb() on enable notification a281ebc virtio: add missing mb() on notification 4. Test case: Create a bridge enslaving the real NIC, and use that as the bridge for a kvm instance with virtio networking. See comment #44 for specific test case. 5. Regression potential: Should be low as several people have tested the fixed package under heavy load. = System: --- Dell R410 Dual processor 2.4Ghz w/16G RAM Distributor ID: Ubuntu Description:Ubuntu 12.04 LTS Release:12.04 Codename: precise Setup: - We're running 3 KVM guests, all Ubuntu 12.04 LTS using bridged networking. From the host: # cat /etc/network/interfaces auto br0 iface br0 inet static address 212.XX.239.98 netmask 255.255.255.240 gateway 212.XX.239.97 bridge_ports eth0 bridge_fd 9 bridge_hello 2 bridge_maxage 12 bridge_stp off # ifconfig eth0 eth0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:11278363 errors:0 dropped:3128 overruns:0 frame:0 TX packets:14437384 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:4115980743 (4.1 GB) TX bytes:5451961979 (5.4 GB) Interrupt:36 Memory:da00-da012800 # ifconfig br0 br0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a inet addr:212.XX.239.98 Bcast:212.XX.239.111 Mask:255.255.255.240 inet6 addr: fe80::d6ae:52ff:fe84:2d5a/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1720861 errors:0 dropped:0 overruns:0 frame:0 TX packets:1708622 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:210152198 (210.1 MB) TX bytes:300858508 (300.8 MB) # brctl show bridge name bridge id STP enabled interfaces br0 8000.d4ae52842d5a no eth0 I have no default network configured to autostart in libvirt as we're using bridged networking: # virsh net-list --all Name State Autostart - default inactive no # arp Address HWtype HWaddress Flags Mask Iface mailer03..com ether 52:54:00:82:5f:0f C br0 mailer01..com ether 52:54:00:d2:f7:31 C br0 mailer02..com ether 52:54:00:d3:8f:91 C br0 dxi-gw2..comether 00:1a:30:2a:b1:c0 C br0 From one of the guests: domain type='kvm' id='4' namemailer01/name uuidd41d1355-84e8-ae23-e84e-227bc0231b97/uuid memory2097152/memory currentMemory2097152/currentMemory vcpu1/vcpu os type arch='x86_64' machine='pc-1.0'hvm/type boot dev='hd'/ /os features acpi/ /features clock offset='utc'/ on_poweroffdestroy/on_poweroff on_rebootrestart/on_reboot on_crashdestroy/on_crash devices emulator/usr/bin/kvm/emulator disk type='file' device='disk' driver name='qemu' type='raw'/ source file='/dev/mapper/vg_main-mailer01--root'/
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
After moving to 3.5 kernel I haven't seen it, even at 3x traffic which used to cause it On Mar 25, 2014 9:06 PM, Dewey McDonnell de...@poslinker.com wrote: My web search reveals this problem has been around for years and most posts conclude that the problem continues. After solving my tap0 TX packets dropped problem, I find that my VM network has random freezes a few times each day. Many bloggers on this subject say this VM network freeze is difficult to reproduce. Not for me! I can cause a network freeze on my VM in a heartbeat. Like many others, all I need to do is start a data transfer over the bridge (FTP 2Kb file or larger). When the freeze happens, ifconfig tap0 says overruns:1. This is QEMU version 1.6.2 without KVM on CentOS 6.5 version 2.6.32-431. -- You received this bug notification because you are subscribed to the bug report. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network Status in OpenStack Compute (Nova): Invalid Status in qemu-kvm package in Ubuntu: Fix Released Status in qemu-kvm source package in Precise: Fix Released Bug description: = SRU Justification: 1. Impact: networking breaks after awhile in kvm guests using virtio networking 2. Development fix: The bug was fixed upstream and the fix picked up in a new merge. 3. Stable fix: 3 virtio patches are cherrypicked from upstream: a821ce5 virtio: order index/descriptor reads 92045d8 virtio: add missing mb() on enable notification a281ebc virtio: add missing mb() on notification 4. Test case: Create a bridge enslaving the real NIC, and use that as the bridge for a kvm instance with virtio networking. See comment #44 for specific test case. 5. Regression potential: Should be low as several people have tested the fixed package under heavy load. = System: --- Dell R410 Dual processor 2.4Ghz w/16G RAM Distributor ID: Ubuntu Description:Ubuntu 12.04 LTS Release:12.04 Codename: precise Setup: - We're running 3 KVM guests, all Ubuntu 12.04 LTS using bridged networking. From the host: # cat /etc/network/interfaces auto br0 iface br0 inet static address 212.XX.239.98 netmask 255.255.255.240 gateway 212.XX.239.97 bridge_ports eth0 bridge_fd 9 bridge_hello 2 bridge_maxage 12 bridge_stp off # ifconfig eth0 eth0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:11278363 errors:0 dropped:3128 overruns:0 frame:0 TX packets:14437384 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:4115980743 (4.1 GB) TX bytes:5451961979 (5.4 GB) Interrupt:36 Memory:da00-da012800 # ifconfig br0 br0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a inet addr:212.XX.239.98 Bcast:212.XX.239.111 Mask:255.255.255.240 inet6 addr: fe80::d6ae:52ff:fe84:2d5a/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1720861 errors:0 dropped:0 overruns:0 frame:0 TX packets:1708622 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:210152198 (210.1 MB) TX bytes:300858508 (300.8 MB) # brctl show bridge name bridge id STP enabled interfaces br0 8000.d4ae52842d5a no eth0 I have no default network configured to autostart in libvirt as we're using bridged networking: # virsh net-list --all Name State Autostart - default inactive no # arp Address HWtype HWaddress Flags Mask Iface mailer03..com ether 52:54:00:82:5f:0f C br0 mailer01..com ether 52:54:00:d2:f7:31 C br0 mailer02..com ether 52:54:00:d3:8f:91 C br0 dxi-gw2..comether 00:1a:30:2a:b1:c0 C br0 From one of the guests: domain type='kvm' id='4' namemailer01/name uuidd41d1355-84e8-ae23-e84e-227bc0231b97/uuid memory2097152/memory currentMemory2097152/currentMemory vcpu1/vcpu os type arch='x86_64' machine='pc-1.0'hvm/type boot dev='hd'/ /os features acpi/ /features clock offset='utc'/ on_poweroffdestroy/on_poweroff on_rebootrestart/on_reboot on_crashdestroy/on_crash devices emulator/usr/bin/kvm/emulator disk type='file' device='disk' driver name='qemu' type='raw'/ source file='/dev/mapper/vg_main-mailer01--root'/
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
Same here, thank you everyone! -- Kraig Amador On Friday, November 2, 2012 at 5:42 AM, Jonathan Tullett wrote: This bug is considered fixed for me. Not a single network glitch since installing the package from PPA. Many thanks to the development team! -- You received this bug notification because you are subscribed to the bug report. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network Status in OpenStack Compute (Nova): Invalid Status in “qemu-kvm” package in Ubuntu: Fix Released Status in “qemu-kvm” source package in Precise: Fix Released Bug description: = SRU Justification: 1. Impact: networking breaks after awhile in kvm guests using virtio networking 2. Development fix: The bug was fixed upstream and the fix picked up in a new merge. 3. Stable fix: 3 virtio patches are cherrypicked from upstream: a821ce5 virtio: order index/descriptor reads 92045d8 virtio: add missing mb() on enable notification a281ebc virtio: add missing mb() on notification 4. Test case: Create a bridge enslaving the real NIC, and use that as the bridge for a kvm instance with virtio networking. See comment #44 for specific test case. 5. Regression potential: Should be low as several people have tested the fixed package under heavy load. = System: --- Dell R410 Dual processor 2.4Ghz w/16G RAM Distributor ID: Ubuntu Description: Ubuntu 12.04 LTS Release: 12.04 Codename: precise Setup: - We're running 3 KVM guests, all Ubuntu 12.04 LTS using bridged networking. From the host: # cat /etc/network/interfaces auto br0 iface br0 inet static address 212.XX.239.98 netmask 255.255.255.240 gateway 212.XX.239.97 bridge_ports eth0 bridge_fd 9 bridge_hello 2 bridge_maxage 12 bridge_stp off # ifconfig eth0 eth0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:11278363 errors:0 dropped:3128 overruns:0 frame:0 TX packets:14437384 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:4115980743 (4.1 GB) TX bytes:5451961979 (5.4 GB) Interrupt:36 Memory:da00-da012800 # ifconfig br0 br0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a inet addr:212.XX.239.98 Bcast:212.XX.239.111 Mask:255.255.255.240 inet6 addr: fe80::d6ae:52ff:fe84:2d5a/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1720861 errors:0 dropped:0 overruns:0 frame:0 TX packets:1708622 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:210152198 (210.1 MB) TX bytes:300858508 (300.8 MB) # brctl show bridge name bridge id STP enabled interfaces br0 8000.d4ae52842d5a no eth0 I have no default network configured to autostart in libvirt as we're using bridged networking: # virsh net-list --all Name State Autostart - default inactive no # arp Address HWtype HWaddress Flags Mask Iface mailer03..com (http://mailer03..com) ether 52:54:00:82:5f:0f C br0 mailer01..com (http://mailer01..com) ether 52:54:00:d2:f7:31 C br0 mailer02..com (http://mailer02..com) ether 52:54:00:d3:8f:91 C br0 dxi-gw2..com (http://dxi-gw2..com) ether 00:1a:30:2a:b1:c0 C br0 From one of the guests: domain type='kvm' id='4' namemailer01/name uuidd41d1355-84e8-ae23-e84e-227bc0231b97/uuid memory2097152/memory currentMemory2097152/currentMemory vcpu1/vcpu os type arch='x86_64' machine='pc-1.0'hvm/type boot dev='hd'/ /os features acpi/ /features clock offset='utc'/ on_poweroffdestroy/on_poweroff on_rebootrestart/on_reboot on_crashdestroy/on_crash devices emulator/usr/bin/kvm/emulator disk type='file' device='disk' driver name='qemu' type='raw'/ source file='/dev/mapper/vg_main-mailer01--root'/ target dev='hda' bus='ide'/ alias name='ide0-0-0'/ address type='drive' controller='0' bus='0' unit='0'/ /disk disk type='file' device='disk' driver name='qemu' type='raw'/ source file='/dev/mapper/vg_main-mailer01--swap'/ target dev='hdb' bus='ide'/ alias name='ide0-0-1'/ address type='drive' controller='0' bus='0' unit='1'/ /disk controller type='ide' index='0' alias name='ide0'/ address type='pci' domain='0x' bus='0x00' slot='0x01' function='0x1'/ /controller interface type='bridge' mac address='52:54:00:d2:f7:31'/ source bridge='br0'/ target dev='vnet0'/ model type='virtio'/ alias name='net0'/ address type='pci' domain='0x' bus='0x00' slot='0x03'
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
Same here, thank you everyone! -- Kraig Amador On Friday, November 2, 2012 at 5:42 AM, Jonathan Tullett wrote: This bug is considered fixed for me. Not a single network glitch since installing the package from PPA. Many thanks to the development team! -- You received this bug notification because you are subscribed to the bug report. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network Status in OpenStack Compute (Nova): Invalid Status in “qemu-kvm” package in Ubuntu: Fix Released Status in “qemu-kvm” source package in Precise: Fix Released Bug description: = SRU Justification: 1. Impact: networking breaks after awhile in kvm guests using virtio networking 2. Development fix: The bug was fixed upstream and the fix picked up in a new merge. 3. Stable fix: 3 virtio patches are cherrypicked from upstream: a821ce5 virtio: order index/descriptor reads 92045d8 virtio: add missing mb() on enable notification a281ebc virtio: add missing mb() on notification 4. Test case: Create a bridge enslaving the real NIC, and use that as the bridge for a kvm instance with virtio networking. See comment #44 for specific test case. 5. Regression potential: Should be low as several people have tested the fixed package under heavy load. = System: --- Dell R410 Dual processor 2.4Ghz w/16G RAM Distributor ID: Ubuntu Description: Ubuntu 12.04 LTS Release: 12.04 Codename: precise Setup: - We're running 3 KVM guests, all Ubuntu 12.04 LTS using bridged networking. From the host: # cat /etc/network/interfaces auto br0 iface br0 inet static address 212.XX.239.98 netmask 255.255.255.240 gateway 212.XX.239.97 bridge_ports eth0 bridge_fd 9 bridge_hello 2 bridge_maxage 12 bridge_stp off # ifconfig eth0 eth0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:11278363 errors:0 dropped:3128 overruns:0 frame:0 TX packets:14437384 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:4115980743 (4.1 GB) TX bytes:5451961979 (5.4 GB) Interrupt:36 Memory:da00-da012800 # ifconfig br0 br0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a inet addr:212.XX.239.98 Bcast:212.XX.239.111 Mask:255.255.255.240 inet6 addr: fe80::d6ae:52ff:fe84:2d5a/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1720861 errors:0 dropped:0 overruns:0 frame:0 TX packets:1708622 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:210152198 (210.1 MB) TX bytes:300858508 (300.8 MB) # brctl show bridge name bridge id STP enabled interfaces br0 8000.d4ae52842d5a no eth0 I have no default network configured to autostart in libvirt as we're using bridged networking: # virsh net-list --all Name State Autostart - default inactive no # arp Address HWtype HWaddress Flags Mask Iface mailer03..com (http://mailer03..com) ether 52:54:00:82:5f:0f C br0 mailer01..com (http://mailer01..com) ether 52:54:00:d2:f7:31 C br0 mailer02..com (http://mailer02..com) ether 52:54:00:d3:8f:91 C br0 dxi-gw2..com (http://dxi-gw2..com) ether 00:1a:30:2a:b1:c0 C br0 From one of the guests: domain type='kvm' id='4' namemailer01/name uuidd41d1355-84e8-ae23-e84e-227bc0231b97/uuid memory2097152/memory currentMemory2097152/currentMemory vcpu1/vcpu os type arch='x86_64' machine='pc-1.0'hvm/type boot dev='hd'/ /os features acpi/ /features clock offset='utc'/ on_poweroffdestroy/on_poweroff on_rebootrestart/on_reboot on_crashdestroy/on_crash devices emulator/usr/bin/kvm/emulator disk type='file' device='disk' driver name='qemu' type='raw'/ source file='/dev/mapper/vg_main-mailer01--root'/ target dev='hda' bus='ide'/ alias name='ide0-0-0'/ address type='drive' controller='0' bus='0' unit='0'/ /disk disk type='file' device='disk' driver name='qemu' type='raw'/ source file='/dev/mapper/vg_main-mailer01--swap'/ target dev='hdb' bus='ide'/ alias name='ide0-0-1'/ address type='drive' controller='0' bus='0' unit='1'/ /disk controller type='ide' index='0' alias name='ide0'/ address type='pci' domain='0x' bus='0x00' slot='0x01' function='0x1'/ /controller interface type='bridge' mac address='52:54:00:d2:f7:31'/ source bridge='br0'/ target dev='vnet0'/ model type='virtio'/ alias name='net0'/ address type='pci' domain='0x' bus='0x00' slot='0x03'
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
I don't believe just rebooting a guest will cause a new KVM instance to load. As a test, I just rebooted a guest VM on a system here and the pid of the kvm process did not change. I think it may be possible that you are still running on the old software. Also, to update my data point... On my server which was experiencing issues, I rebooted the host just to make sure everything was fresh. It's been about a month and I have not experienced the failure again. I was typically going a few weeks between issues. gary - Original Message - From: Matt Hilt mjh...@gmail.com To: g...@isgsoftware.net Sent: Tuesday, September 25, 2012 12:07:44 PM Subject: [Bug 997978] Re: KVM images lose connectivity with bridged network Soren, We have a 12.04 based OpenStack cluster with 4 host nodes running about 30 VMs currently. We performed the steps to add the kvm-network-hang repo and updated to the latest version on the host machines, then rebooted the instances. My understanding is that this should catch the update, since a new KVM command is run on reboot. I caught the first failure ~12 hours after the upgrade. It had the usual symptoms: networking loss, but the VM is still up and an active VNC session was possible. I thought I just might have missed a reboot on one of the VMs, so I didn't report anything. The second failure happened yesterday, but someone else caught it and rebooted the VM. As best we can tell after the fact, it looks like the usual failure (no full harddrive, or kernel panic, or anything that got logged). As I mentioned before, we used to see at least one failure per day, usually much more. This patch has at least reduced the occurence to a minimal amount. These non-deterministic bugs are hard to track down. -- You received this bug notification because you are subscribed to the bug report. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network Status in OpenStack Compute (Nova): Invalid Status in “qemu-kvm” package in Ubuntu: Fix Released Status in “qemu-kvm” source package in Precise: In Progress Bug description: = SRU Justification: 1. Impact: networking breaks after awhile in kvm guests using virtio networking 2. Development fix: The bug was fixed upstream and the fix picked up in a new merge. 3. Stable fix: 3 virtio patches are cherrypicked from upstream: a821ce5 virtio: order index/descriptor reads 92045d8 virtio: add missing mb() on enable notification a281ebc virtio: add missing mb() on notification 4. Test case: Create a bridge enslaving the real NIC, and use that as the bridge for a kvm instance with virtio networking. See comment #44 for specific test case. 5. Regression potential: Should be low as several people have tested the fixed package under heavy load. = System: --- Dell R410 Dual processor 2.4Ghz w/16G RAM Distributor ID: Ubuntu Description:Ubuntu 12.04 LTS Release:12.04 Codename: precise Setup: - We're running 3 KVM guests, all Ubuntu 12.04 LTS using bridged networking. From the host: # cat /etc/network/interfaces auto br0 iface br0 inet static address 212.XX.239.98 netmask 255.255.255.240 gateway 212.XX.239.97 bridge_ports eth0 bridge_fd 9 bridge_hello 2 bridge_maxage 12 bridge_stp off # ifconfig eth0 eth0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:11278363 errors:0 dropped:3128 overruns:0 frame:0 TX packets:14437384 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:4115980743 (4.1 GB) TX bytes:5451961979 (5.4 GB) Interrupt:36 Memory:da00-da012800 # ifconfig br0 br0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a inet addr:212.XX.239.98 Bcast:212.XX.239.111 Mask:255.255.255.240 inet6 addr: fe80::d6ae:52ff:fe84:2d5a/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1720861 errors:0 dropped:0 overruns:0 frame:0 TX packets:1708622 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:210152198 (210.1 MB) TX bytes:300858508 (300.8 MB) # brctl show bridge name bridge id STP enabled interfaces br0 8000.d4ae52842d5a no eth0 I have no default network configured to autostart in libvirt as we're using bridged networking: # virsh net-list --all Name State Autostart - default inactive no # arp Address HWtype HWaddress Flags Mask Iface
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
I don't believe just rebooting a guest will cause a new KVM instance to load. As a test, I just rebooted a guest VM on a system here and the pid of the kvm process did not change. I think it may be possible that you are still running on the old software. Also, to update my data point... On my server which was experiencing issues, I rebooted the host just to make sure everything was fresh. It's been about a month and I have not experienced the failure again. I was typically going a few weeks between issues. gary - Original Message - From: Matt Hilt mjh...@gmail.com To: g...@isgsoftware.net Sent: Tuesday, September 25, 2012 12:07:44 PM Subject: [Bug 997978] Re: KVM images lose connectivity with bridged network Soren, We have a 12.04 based OpenStack cluster with 4 host nodes running about 30 VMs currently. We performed the steps to add the kvm-network-hang repo and updated to the latest version on the host machines, then rebooted the instances. My understanding is that this should catch the update, since a new KVM command is run on reboot. I caught the first failure ~12 hours after the upgrade. It had the usual symptoms: networking loss, but the VM is still up and an active VNC session was possible. I thought I just might have missed a reboot on one of the VMs, so I didn't report anything. The second failure happened yesterday, but someone else caught it and rebooted the VM. As best we can tell after the fact, it looks like the usual failure (no full harddrive, or kernel panic, or anything that got logged). As I mentioned before, we used to see at least one failure per day, usually much more. This patch has at least reduced the occurence to a minimal amount. These non-deterministic bugs are hard to track down. -- You received this bug notification because you are subscribed to the bug report. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network Status in OpenStack Compute (Nova): Invalid Status in “qemu-kvm” package in Ubuntu: Fix Released Status in “qemu-kvm” source package in Precise: In Progress Bug description: = SRU Justification: 1. Impact: networking breaks after awhile in kvm guests using virtio networking 2. Development fix: The bug was fixed upstream and the fix picked up in a new merge. 3. Stable fix: 3 virtio patches are cherrypicked from upstream: a821ce5 virtio: order index/descriptor reads 92045d8 virtio: add missing mb() on enable notification a281ebc virtio: add missing mb() on notification 4. Test case: Create a bridge enslaving the real NIC, and use that as the bridge for a kvm instance with virtio networking. See comment #44 for specific test case. 5. Regression potential: Should be low as several people have tested the fixed package under heavy load. = System: --- Dell R410 Dual processor 2.4Ghz w/16G RAM Distributor ID: Ubuntu Description:Ubuntu 12.04 LTS Release:12.04 Codename: precise Setup: - We're running 3 KVM guests, all Ubuntu 12.04 LTS using bridged networking. From the host: # cat /etc/network/interfaces auto br0 iface br0 inet static address 212.XX.239.98 netmask 255.255.255.240 gateway 212.XX.239.97 bridge_ports eth0 bridge_fd 9 bridge_hello 2 bridge_maxage 12 bridge_stp off # ifconfig eth0 eth0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:11278363 errors:0 dropped:3128 overruns:0 frame:0 TX packets:14437384 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:4115980743 (4.1 GB) TX bytes:5451961979 (5.4 GB) Interrupt:36 Memory:da00-da012800 # ifconfig br0 br0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a inet addr:212.XX.239.98 Bcast:212.XX.239.111 Mask:255.255.255.240 inet6 addr: fe80::d6ae:52ff:fe84:2d5a/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1720861 errors:0 dropped:0 overruns:0 frame:0 TX packets:1708622 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:210152198 (210.1 MB) TX bytes:300858508 (300.8 MB) # brctl show bridge name bridge id STP enabled interfaces br0 8000.d4ae52842d5a no eth0 I have no default network configured to autostart in libvirt as we're using bridged networking: # virsh net-list --all Name State Autostart - default inactive no # arp Address HWtype HWaddress Flags Mask Iface
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
I can test that. I had to use the hardware to deal with a separate issue but I loaded up the backported package on Friday and I have been running a test over the weekend. No problems yet. Sent from my iPhone On Sep 3, 2012, at 1:21 AM, Soren Hansen so...@linux2go.dk wrote: As Serge says, we think we've narrowed in on the set of commits that will address this problem: a821ce5 virtio: order index/descriptor reads 92045d8 virtio: add missing mb() on enable notification a281ebc virtio: add missing mb() on notification I'd be happy to provide a SRU candidate in a PPA with just those patches applied if anyone is willing to test? Serge, would it be ok for me to use the ubuntu-virt/backports ppa for this, or would you rather I create a new PPA? -- You received this bug notification because you are subscribed to the bug report. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network Status in OpenStack Compute (Nova): New Status in “bridge-utils” package in Ubuntu: Invalid Status in “ifenslave” package in Ubuntu: Confirmed Status in “libvirt” package in Ubuntu: Confirmed Status in “linux” package in Ubuntu: Confirmed Status in “qemu-kvm” package in Ubuntu: Confirmed Bug description: System: --- Dell R410 Dual processor 2.4Ghz w/16G RAM Distributor ID: Ubuntu Description:Ubuntu 12.04 LTS Release:12.04 Codename: precise Setup: - We're running 3 KVM guests, all Ubuntu 12.04 LTS using bridged networking. From the host: # cat /etc/network/interfaces auto br0 iface br0 inet static address 212.XX.239.98 netmask 255.255.255.240 gateway 212.XX.239.97 bridge_ports eth0 bridge_fd 9 bridge_hello 2 bridge_maxage 12 bridge_stp off # ifconfig eth0 eth0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:11278363 errors:0 dropped:3128 overruns:0 frame:0 TX packets:14437384 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:4115980743 (4.1 GB) TX bytes:5451961979 (5.4 GB) Interrupt:36 Memory:da00-da012800 # ifconfig br0 br0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a inet addr:212.XX.239.98 Bcast:212.XX.239.111 Mask:255.255.255.240 inet6 addr: fe80::d6ae:52ff:fe84:2d5a/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1720861 errors:0 dropped:0 overruns:0 frame:0 TX packets:1708622 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:210152198 (210.1 MB) TX bytes:300858508 (300.8 MB) # brctl show bridge name bridge id STP enabled interfaces br0 8000.d4ae52842d5a no eth0 I have no default network configured to autostart in libvirt as we're using bridged networking: # virsh net-list --all Name State Autostart - default inactive no # arp Address HWtype HWaddress Flags Mask Iface mailer03..com ether 52:54:00:82:5f:0f C br0 mailer01..com ether 52:54:00:d2:f7:31 C br0 mailer02..com ether 52:54:00:d3:8f:91 C br0 dxi-gw2..comether 00:1a:30:2a:b1:c0 C br0 From one of the guests: domain type='kvm' id='4' namemailer01/name uuidd41d1355-84e8-ae23-e84e-227bc0231b97/uuid memory2097152/memory currentMemory2097152/currentMemory vcpu1/vcpu os type arch='x86_64' machine='pc-1.0'hvm/type boot dev='hd'/ /os features acpi/ /features clock offset='utc'/ on_poweroffdestroy/on_poweroff on_rebootrestart/on_reboot on_crashdestroy/on_crash devices emulator/usr/bin/kvm/emulator disk type='file' device='disk' driver name='qemu' type='raw'/ source file='/dev/mapper/vg_main-mailer01--root'/ target dev='hda' bus='ide'/ alias name='ide0-0-0'/ address type='drive' controller='0' bus='0' unit='0'/ /disk disk type='file' device='disk' driver name='qemu' type='raw'/ source file='/dev/mapper/vg_main-mailer01--swap'/ target dev='hdb' bus='ide'/ alias name='ide0-0-1'/ address type='drive' controller='0' bus='0' unit='1'/ /disk controller type='ide' index='0' alias name='ide0'/ address type='pci' domain='0x' bus='0x00' slot='0x01' function='0x1'/ /controller interface type='bridge' mac address='52:54:00:d2:f7:31'/ source
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
I can test that. I had to use the hardware to deal with a separate issue but I loaded up the backported package on Friday and I have been running a test over the weekend. No problems yet. Sent from my iPhone On Sep 3, 2012, at 1:21 AM, Soren Hansen so...@linux2go.dk wrote: As Serge says, we think we've narrowed in on the set of commits that will address this problem: a821ce5 virtio: order index/descriptor reads 92045d8 virtio: add missing mb() on enable notification a281ebc virtio: add missing mb() on notification I'd be happy to provide a SRU candidate in a PPA with just those patches applied if anyone is willing to test? Serge, would it be ok for me to use the ubuntu-virt/backports ppa for this, or would you rather I create a new PPA? -- You received this bug notification because you are subscribed to the bug report. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network Status in OpenStack Compute (Nova): New Status in “bridge-utils” package in Ubuntu: Invalid Status in “ifenslave” package in Ubuntu: Confirmed Status in “libvirt” package in Ubuntu: Confirmed Status in “linux” package in Ubuntu: Confirmed Status in “qemu-kvm” package in Ubuntu: Confirmed Bug description: System: --- Dell R410 Dual processor 2.4Ghz w/16G RAM Distributor ID: Ubuntu Description:Ubuntu 12.04 LTS Release:12.04 Codename: precise Setup: - We're running 3 KVM guests, all Ubuntu 12.04 LTS using bridged networking. From the host: # cat /etc/network/interfaces auto br0 iface br0 inet static address 212.XX.239.98 netmask 255.255.255.240 gateway 212.XX.239.97 bridge_ports eth0 bridge_fd 9 bridge_hello 2 bridge_maxage 12 bridge_stp off # ifconfig eth0 eth0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:11278363 errors:0 dropped:3128 overruns:0 frame:0 TX packets:14437384 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:4115980743 (4.1 GB) TX bytes:5451961979 (5.4 GB) Interrupt:36 Memory:da00-da012800 # ifconfig br0 br0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a inet addr:212.XX.239.98 Bcast:212.XX.239.111 Mask:255.255.255.240 inet6 addr: fe80::d6ae:52ff:fe84:2d5a/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1720861 errors:0 dropped:0 overruns:0 frame:0 TX packets:1708622 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:210152198 (210.1 MB) TX bytes:300858508 (300.8 MB) # brctl show bridge name bridge id STP enabled interfaces br0 8000.d4ae52842d5a no eth0 I have no default network configured to autostart in libvirt as we're using bridged networking: # virsh net-list --all Name State Autostart - default inactive no # arp Address HWtype HWaddress Flags Mask Iface mailer03..com ether 52:54:00:82:5f:0f C br0 mailer01..com ether 52:54:00:d2:f7:31 C br0 mailer02..com ether 52:54:00:d3:8f:91 C br0 dxi-gw2..comether 00:1a:30:2a:b1:c0 C br0 From one of the guests: domain type='kvm' id='4' namemailer01/name uuidd41d1355-84e8-ae23-e84e-227bc0231b97/uuid memory2097152/memory currentMemory2097152/currentMemory vcpu1/vcpu os type arch='x86_64' machine='pc-1.0'hvm/type boot dev='hd'/ /os features acpi/ /features clock offset='utc'/ on_poweroffdestroy/on_poweroff on_rebootrestart/on_reboot on_crashdestroy/on_crash devices emulator/usr/bin/kvm/emulator disk type='file' device='disk' driver name='qemu' type='raw'/ source file='/dev/mapper/vg_main-mailer01--root'/ target dev='hda' bus='ide'/ alias name='ide0-0-0'/ address type='drive' controller='0' bus='0' unit='0'/ /disk disk type='file' device='disk' driver name='qemu' type='raw'/ source file='/dev/mapper/vg_main-mailer01--swap'/ target dev='hdb' bus='ide'/ alias name='ide0-0-1'/ address type='drive' controller='0' bus='0' unit='1'/ /disk controller type='ide' index='0' alias name='ide0'/ address type='pci' domain='0x' bus='0x00' slot='0x01' function='0x1'/ /controller interface type='bridge' mac address='52:54:00:d2:f7:31'/ source
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
Quoting Gary Cuozzo (g...@isgsoftware.net): I installed the new packages on Monday and have not had any issues. That said, I was only experiencing failures once every 3-4 weeks. So I don't think my data point will be valid for at least a few weeks. Thanks, Gary, will wait before making assumptions. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to bridge-utils in Ubuntu. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/997978/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
Quoting Gary Cuozzo (g...@isgsoftware.net): I installed the new packages on Monday and have not had any issues. That said, I was only experiencing failures once every 3-4 weeks. So I don't think my data point will be valid for at least a few weeks. Thanks, Gary, will wait before making assumptions. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/997978/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
I think the big is in the guest code. Did you try 10.04 LTS? -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to bridge-utils in Ubuntu. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/997978/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
Quoting Metin Akat (997...@bugs.launchpad.net): No I might try it later today (as a guest) I installed the PPA to the host. From what I can see, the situation is even worse. Had several failures in the first 2 hours since. Thanks, that's valuable information. (Yes, the ppa was to be installed on the host, not the guests) -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to bridge-utils in Ubuntu. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/997978/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
I think the big is in the guest code. Did you try 10.04 LTS? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/997978/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
Quoting Metin Akat (997...@bugs.launchpad.net): No I might try it later today (as a guest) I installed the PPA to the host. From what I can see, the situation is even worse. Had several failures in the first 2 hours since. Thanks, that's valuable information. (Yes, the ppa was to be installed on the host, not the guests) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/997978/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
Quoting Gary Cuozzo (g...@isgsoftware.net): Regarding comment #55: I don't believe this is a guest OS issue. In my comment #51 I also indicate that I have had non-precise vm's (specifically 11.10) experience the same issue. I have only experienced this issue when using 12.04 precise as the host OS. On a customer server which I back-dated by reinstalling 11.10 as the host OS, the issue went away. Ah, thanks for that info. If I push an updated version of qemu-kvm to ppa:ubuntu-virt/ppa, would you (would anyone) be able to test it to confirm whether it fixes the issue? If so then we could be sure the problem is in qemu userspace. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to bridge-utils in Ubuntu. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/997978/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
I can test that today. Sent from my iPhone On Aug 16, 2012, at 7:11 AM, Serge Hallyn 997...@bugs.launchpad.net wrote: Quoting Gary Cuozzo (g...@isgsoftware.net): Regarding comment #55: I don't believe this is a guest OS issue. In my comment #51 I also indicate that I have had non-precise vm's (specifically 11.10) experience the same issue. I have only experienced this issue when using 12.04 precise as the host OS. On a customer server which I back-dated by reinstalling 11.10 as the host OS, the issue went away. Ah, thanks for that info. If I push an updated version of qemu-kvm to ppa:ubuntu-virt/ppa, would you (would anyone) be able to test it to confirm whether it fixes the issue? If so then we could be sure the problem is in qemu userspace. -- You received this bug notification because you are subscribed to the bug report. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network Status in OpenStack Compute (Nova): New Status in “bridge-utils” package in Ubuntu: Invalid Status in “ifenslave” package in Ubuntu: Confirmed Status in “libvirt” package in Ubuntu: Confirmed Status in “linux” package in Ubuntu: Confirmed Status in “qemu-kvm” package in Ubuntu: Confirmed Bug description: System: --- Dell R410 Dual processor 2.4Ghz w/16G RAM Distributor ID: Ubuntu Description:Ubuntu 12.04 LTS Release:12.04 Codename: precise Setup: - We're running 3 KVM guests, all Ubuntu 12.04 LTS using bridged networking. From the host: # cat /etc/network/interfaces auto br0 iface br0 inet static address 212.XX.239.98 netmask 255.255.255.240 gateway 212.XX.239.97 bridge_ports eth0 bridge_fd 9 bridge_hello 2 bridge_maxage 12 bridge_stp off # ifconfig eth0 eth0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:11278363 errors:0 dropped:3128 overruns:0 frame:0 TX packets:14437384 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:4115980743 (4.1 GB) TX bytes:5451961979 (5.4 GB) Interrupt:36 Memory:da00-da012800 # ifconfig br0 br0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a inet addr:212.XX.239.98 Bcast:212.XX.239.111 Mask:255.255.255.240 inet6 addr: fe80::d6ae:52ff:fe84:2d5a/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1720861 errors:0 dropped:0 overruns:0 frame:0 TX packets:1708622 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:210152198 (210.1 MB) TX bytes:300858508 (300.8 MB) # brctl show bridge name bridge id STP enabled interfaces br0 8000.d4ae52842d5a no eth0 I have no default network configured to autostart in libvirt as we're using bridged networking: # virsh net-list --all Name State Autostart - default inactive no # arp Address HWtype HWaddress Flags Mask Iface mailer03..com ether 52:54:00:82:5f:0f C br0 mailer01..com ether 52:54:00:d2:f7:31 C br0 mailer02..com ether 52:54:00:d3:8f:91 C br0 dxi-gw2..comether 00:1a:30:2a:b1:c0 C br0 From one of the guests: domain type='kvm' id='4' namemailer01/name uuidd41d1355-84e8-ae23-e84e-227bc0231b97/uuid memory2097152/memory currentMemory2097152/currentMemory vcpu1/vcpu os type arch='x86_64' machine='pc-1.0'hvm/type boot dev='hd'/ /os features acpi/ /features clock offset='utc'/ on_poweroffdestroy/on_poweroff on_rebootrestart/on_reboot on_crashdestroy/on_crash devices emulator/usr/bin/kvm/emulator disk type='file' device='disk' driver name='qemu' type='raw'/ source file='/dev/mapper/vg_main-mailer01--root'/ target dev='hda' bus='ide'/ alias name='ide0-0-0'/ address type='drive' controller='0' bus='0' unit='0'/ /disk disk type='file' device='disk' driver name='qemu' type='raw'/ source file='/dev/mapper/vg_main-mailer01--swap'/ target dev='hdb' bus='ide'/ alias name='ide0-0-1'/ address type='drive' controller='0' bus='0' unit='1'/ /disk controller type='ide' index='0' alias name='ide0'/ address type='pci' domain='0x' bus='0x00' slot='0x01' function='0x1'/ /controller interface type='bridge' mac address='52:54:00:d2:f7:31'/
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
Quoting Gary Cuozzo (g...@isgsoftware.net): Regarding comment #55: I don't believe this is a guest OS issue. In my comment #51 I also indicate that I have had non-precise vm's (specifically 11.10) experience the same issue. I have only experienced this issue when using 12.04 precise as the host OS. On a customer server which I back-dated by reinstalling 11.10 as the host OS, the issue went away. Ah, thanks for that info. If I push an updated version of qemu-kvm to ppa:ubuntu-virt/ppa, would you (would anyone) be able to test it to confirm whether it fixes the issue? If so then we could be sure the problem is in qemu userspace. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/997978/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
I can test that today. Sent from my iPhone On Aug 16, 2012, at 7:11 AM, Serge Hallyn 997...@bugs.launchpad.net wrote: Quoting Gary Cuozzo (g...@isgsoftware.net): Regarding comment #55: I don't believe this is a guest OS issue. In my comment #51 I also indicate that I have had non-precise vm's (specifically 11.10) experience the same issue. I have only experienced this issue when using 12.04 precise as the host OS. On a customer server which I back-dated by reinstalling 11.10 as the host OS, the issue went away. Ah, thanks for that info. If I push an updated version of qemu-kvm to ppa:ubuntu-virt/ppa, would you (would anyone) be able to test it to confirm whether it fixes the issue? If so then we could be sure the problem is in qemu userspace. -- You received this bug notification because you are subscribed to the bug report. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network Status in OpenStack Compute (Nova): New Status in “bridge-utils” package in Ubuntu: Invalid Status in “ifenslave” package in Ubuntu: Confirmed Status in “libvirt” package in Ubuntu: Confirmed Status in “linux” package in Ubuntu: Confirmed Status in “qemu-kvm” package in Ubuntu: Confirmed Bug description: System: --- Dell R410 Dual processor 2.4Ghz w/16G RAM Distributor ID: Ubuntu Description:Ubuntu 12.04 LTS Release:12.04 Codename: precise Setup: - We're running 3 KVM guests, all Ubuntu 12.04 LTS using bridged networking. From the host: # cat /etc/network/interfaces auto br0 iface br0 inet static address 212.XX.239.98 netmask 255.255.255.240 gateway 212.XX.239.97 bridge_ports eth0 bridge_fd 9 bridge_hello 2 bridge_maxage 12 bridge_stp off # ifconfig eth0 eth0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:11278363 errors:0 dropped:3128 overruns:0 frame:0 TX packets:14437384 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:4115980743 (4.1 GB) TX bytes:5451961979 (5.4 GB) Interrupt:36 Memory:da00-da012800 # ifconfig br0 br0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a inet addr:212.XX.239.98 Bcast:212.XX.239.111 Mask:255.255.255.240 inet6 addr: fe80::d6ae:52ff:fe84:2d5a/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1720861 errors:0 dropped:0 overruns:0 frame:0 TX packets:1708622 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:210152198 (210.1 MB) TX bytes:300858508 (300.8 MB) # brctl show bridge name bridge id STP enabled interfaces br0 8000.d4ae52842d5a no eth0 I have no default network configured to autostart in libvirt as we're using bridged networking: # virsh net-list --all Name State Autostart - default inactive no # arp Address HWtype HWaddress Flags Mask Iface mailer03..com ether 52:54:00:82:5f:0f C br0 mailer01..com ether 52:54:00:d2:f7:31 C br0 mailer02..com ether 52:54:00:d3:8f:91 C br0 dxi-gw2..comether 00:1a:30:2a:b1:c0 C br0 From one of the guests: domain type='kvm' id='4' namemailer01/name uuidd41d1355-84e8-ae23-e84e-227bc0231b97/uuid memory2097152/memory currentMemory2097152/currentMemory vcpu1/vcpu os type arch='x86_64' machine='pc-1.0'hvm/type boot dev='hd'/ /os features acpi/ /features clock offset='utc'/ on_poweroffdestroy/on_poweroff on_rebootrestart/on_reboot on_crashdestroy/on_crash devices emulator/usr/bin/kvm/emulator disk type='file' device='disk' driver name='qemu' type='raw'/ source file='/dev/mapper/vg_main-mailer01--root'/ target dev='hda' bus='ide'/ alias name='ide0-0-0'/ address type='drive' controller='0' bus='0' unit='0'/ /disk disk type='file' device='disk' driver name='qemu' type='raw'/ source file='/dev/mapper/vg_main-mailer01--swap'/ target dev='hdb' bus='ide'/ alias name='ide0-0-1'/ address type='drive' controller='0' bus='0' unit='1'/ /disk controller type='ide' index='0' alias name='ide0'/ address type='pci' domain='0x' bus='0x00' slot='0x01' function='0x1'/ /controller interface type='bridge' mac address='52:54:00:d2:f7:31'/
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
Thanks, Eugene - that sounds like a great test case! I will try that. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to bridge-utils in Ubuntu. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/bridge-utils/+bug/997978/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
Thanks, Eugene - that sounds like a great test case! I will try that. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/bridge-utils/+bug/997978/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
Quoting Georg Leciejewski (vespaschor...@gmx.de): here it is. already posted it above with ip's xx: auto lo iface lo inet loopback # device: eth0 auto eth0 iface eth0 inet static Hi, I believe this is wrong. Could you change the eth0 bit to simply auto eth0 iface eth0 inet manual The fact that I also did an mtr during downtimes and it shows that packages are lost on dom0 - .79 Supports the idea that that might solve your problem. (It also suggests that yours is not the same as the original bug reporter's problem). -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to bridge-utils in Ubuntu. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/bridge-utils/+bug/997978/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
Quoting Georg Leciejewski (vespaschor...@gmx.de): here it is. already posted it above with ip's xx: auto lo iface lo inet loopback # device: eth0 auto eth0 iface eth0 inet static Hi, I believe this is wrong. Could you change the eth0 bit to simply auto eth0 iface eth0 inet manual The fact that I also did an mtr during downtimes and it shows that packages are lost on dom0 - .79 Supports the idea that that might solve your problem. (It also suggests that yours is not the same as the original bug reporter's problem). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/bridge-utils/+bug/997978/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
Quoting Georg Leciejewski (vespaschor...@gmx.de): @serge, thanks for your input. here Dom0 ifconfig, with 2 machines running. i setup br0 myself because the machines are running at hetzner, with special addon IP's so the vm's can be reached from the outside. (wiki.hetzner.de/index.php/KVM_mit_Nutzung_aller_IPs_-_the_easy_way) i am also evaluating the problem with hetzner support to ensure its not one of their routers in from of the Dom0. vnet0, vnet1 are the bridges created by libvirt, but the vm have static ip entries in /network/interfaces and are bound to br0 via libvirt. -- br0 Link encap:Ethernet HWaddr c8:60:00:e9:4a:2e inet addr:176.9.126.xx Bcast:0.0.0.0 Mask:255.255.255.255 inet6 addr: fe80::ca60:ff:fee9:4a2e/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:5852477 errors:0 dropped:0 overruns:0 frame:0 TX packets:4403013 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:2310013511 (2.3 GB) TX bytes:2588138678 (2.5 GB) eth0 Link encap:Ethernet HWaddr c8:60:00:e9:4a:2e inet addr:176.9.126.xx Bcast:176.9.126.95 Mask:255.255.255.192 UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:6623242 errors:0 dropped:35190 overruns:0 frame:0 TX packets:4668377 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:2484375973 (2.4 GB) TX bytes:1715146205 (1.7 GB) Interrupt:17 Memory:fe50-fe52 One thing I notice here is that you have an ip address on eth0, which I assume is bridged with br0? When I bridge eth0 to br0 using the following /etc/network/interfaces: = auto lo iface lo inet loopback auto br0 iface br0 inet dhcp bridge_ports eth0 # The primary network interface auto eth0 iface eth0 inet manual = I get the following ifconfig -a output: = br0 Link encap:Ethernet HWaddr fa:16:3e:59:27:16 inet addr:10.55.60.89 Bcast:10.55.60.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fe59:2716/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:223 errors:0 dropped:0 overruns:0 frame:0 TX packets:178 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:21627 (21.6 KB) TX bytes:19555 (19.5 KB) eth0 Link encap:Ethernet HWaddr fa:16:3e:59:27:16 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:259 errors:0 dropped:0 overruns:0 frame:0 TX packets:178 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:26871 (26.8 KB) TX bytes:19487 (19.4 KB) = What does your /etc/network/interfaces look like? -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to bridge-utils in Ubuntu. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/bridge-utils/+bug/997978/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
Quoting Georg Leciejewski (vespaschor...@gmx.de): @serge, thanks for your input. here Dom0 ifconfig, with 2 machines running. i setup br0 myself because the machines are running at hetzner, with special addon IP's so the vm's can be reached from the outside. (wiki.hetzner.de/index.php/KVM_mit_Nutzung_aller_IPs_-_the_easy_way) i am also evaluating the problem with hetzner support to ensure its not one of their routers in from of the Dom0. vnet0, vnet1 are the bridges created by libvirt, but the vm have static ip entries in /network/interfaces and are bound to br0 via libvirt. -- br0 Link encap:Ethernet HWaddr c8:60:00:e9:4a:2e inet addr:176.9.126.xx Bcast:0.0.0.0 Mask:255.255.255.255 inet6 addr: fe80::ca60:ff:fee9:4a2e/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:5852477 errors:0 dropped:0 overruns:0 frame:0 TX packets:4403013 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:2310013511 (2.3 GB) TX bytes:2588138678 (2.5 GB) eth0 Link encap:Ethernet HWaddr c8:60:00:e9:4a:2e inet addr:176.9.126.xx Bcast:176.9.126.95 Mask:255.255.255.192 UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:6623242 errors:0 dropped:35190 overruns:0 frame:0 TX packets:4668377 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:2484375973 (2.4 GB) TX bytes:1715146205 (1.7 GB) Interrupt:17 Memory:fe50-fe52 One thing I notice here is that you have an ip address on eth0, which I assume is bridged with br0? When I bridge eth0 to br0 using the following /etc/network/interfaces: = auto lo iface lo inet loopback auto br0 iface br0 inet dhcp bridge_ports eth0 # The primary network interface auto eth0 iface eth0 inet manual = I get the following ifconfig -a output: = br0 Link encap:Ethernet HWaddr fa:16:3e:59:27:16 inet addr:10.55.60.89 Bcast:10.55.60.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fe59:2716/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:223 errors:0 dropped:0 overruns:0 frame:0 TX packets:178 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:21627 (21.6 KB) TX bytes:19555 (19.5 KB) eth0 Link encap:Ethernet HWaddr fa:16:3e:59:27:16 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:259 errors:0 dropped:0 overruns:0 frame:0 TX packets:178 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:26871 (26.8 KB) TX bytes:19487 (19.4 KB) = What does your /etc/network/interfaces look like? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/bridge-utils/+bug/997978/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
Thanks, In order to check whether it is a qemu (perhaps virtio driver) bug or a bug in the kernel or network utilities on the host, would you be able to try setting up a container and checking it's networking? There are lighter weight ways of testing this, but the simplest way would be to: sudo apt-get install lxc # If having lxcbr0 bothers you, since you don't need it for this test, you # can set LXC_AUTO=false in /etc/default/lxc and do # sudo stop lxc; sudo start lxc. cat lxc.conf EOF lxc.network.type=veth lxc.network.link=br0 lxc.network.flags=up EOF sudo lxc-create -t ubuntu -f lxc.conf -n lxc1 sudo lxc-start -n lxc1 -d Then log into the container's console with sudo lxc-console -n lxc1 and, from there, periodically check the network status. If that also loses connectivity periodically, then we know the bug is happening below kvm. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to bridge-utils in Ubuntu. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/bridge-utils/+bug/997978/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 997978] Re: KVM images lose connectivity with bridged network
Thanks, In order to check whether it is a qemu (perhaps virtio driver) bug or a bug in the kernel or network utilities on the host, would you be able to try setting up a container and checking it's networking? There are lighter weight ways of testing this, but the simplest way would be to: sudo apt-get install lxc # If having lxcbr0 bothers you, since you don't need it for this test, you # can set LXC_AUTO=false in /etc/default/lxc and do # sudo stop lxc; sudo start lxc. cat lxc.conf EOF lxc.network.type=veth lxc.network.link=br0 lxc.network.flags=up EOF sudo lxc-create -t ubuntu -f lxc.conf -n lxc1 sudo lxc-start -n lxc1 -d Then log into the container's console with sudo lxc-console -n lxc1 and, from there, periodically check the network status. If that also loses connectivity periodically, then we know the bug is happening below kvm. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/bridge-utils/+bug/997978/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
