[Bug 673705] Re: running guests freeze when a guest is powered down
Marking this fix releases as the upstream commit fixing it is in oneiric. ** Changed in: libvirt (Ubuntu) Status: Triaged = Fix Released -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in Ubuntu. https://bugs.launchpad.net/bugs/673705 Title: running guests freeze when a guest is powered down To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/673705/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 673705] Re: running guests freeze when a guest is powered down
upstream fixed this in 0.9.0 with commit 5754dbd56d4738112a86776c09e810e32f7c3224 ** Bug watch added: Red Hat Bugzilla #609463 https://bugzilla.redhat.com/show_bug.cgi?id=609463 ** Also affects: libvirt (Fedora) via https://bugzilla.redhat.com/show_bug.cgi?id=609463 Importance: Unknown Status: Unknown -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in Ubuntu. https://bugs.launchpad.net/bugs/673705 Title: running guests freeze when a guest is powered down -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 673705] Re: running guests freeze when a guest is powered down
** Changed in: libvirt (Ubuntu) Status: Confirmed = In Progress ** Changed in: libvirt (Ubuntu) Assignee: (unassigned) = Serge Hallyn (serge-hallyn) -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in Ubuntu. https://bugs.launchpad.net/bugs/673705 Title: running guests freeze when a guest is powered down -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 673705] Re: running guests freeze when a guest is powered down
** Branch linked: lp:~serge-hallyn/ubuntu/natty/libvirt/fix-macaddr -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in Ubuntu. https://bugs.launchpad.net/bugs/673705 Title: running guests freeze when a guest is powered down -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 673705] Re: running guests freeze when a guest is powered down
** Branch linked: lp:~serge-hallyn/ubuntu/natty/libvirt/fix-maccaddr2 -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in Ubuntu. https://bugs.launchpad.net/bugs/673705 Title: running guests freeze when a guest is powered down -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 673705] Re: running guests freeze when a guest is powered down
** Tags added: server-nrs -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in Ubuntu. https://bugs.launchpad.net/bugs/673705 Title: running guests freeze when a guest is powered down -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 673705] Re: running guests freeze when a guest is powered down
This bug is probably a duplicate issue as this: https://bugs.launchpad.net/ubuntu/maverick/+source/qemu-kvm/+bug/584048 The problem is with Linux bridging. When adding or removing a MAC address (like for KVM, VirtualBox, or even LXC) then if the bridge changes its MAC, this symptom happens. See comment #60 at the URL above, and also see this: https://www.redhat.com/archives/libvir-list/2010-July/msg00450.html The workaround is to use a MAC address that starts with fe (or any high really high number) for your guests. This causes the kernel to default to the hardware MAC for the bridge. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/673705 Title: running guests freeze when a guest is powered down -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 673705] Re: running guests freeze when a guest is powered down
Quoting Derek Simkowiak (ubu...@cool-st.com): This bug is probably a duplicate issue as this: https://bugs.launchpad.net/ubuntu/maverick/+source/qemu-kvm/+bug/584048 The problem is with Linux bridging. When adding or removing a MAC address (like for KVM, VirtualBox, or even LXC) then if the bridge changes its MAC, this symptom happens. See comment #60 at the URL above, and also see this: https://www.redhat.com/archives/libvir-list/2010-July/msg00450.html The workaround is to use a MAC address that starts with fe (or any high really high number) for your guests. This causes the kernel to default to the hardware MAC for the bridge. Derek, thanks so much for this. You're almost 100% correct. I had even suspected that bug (in comment #7), but I failed to make the simple connection explaining this behavior. The proposed solution does *not* suffice, and in fact I just reproduced the bug in natty! The current solution counts on the bridge being associated with a physical NIC. But our default configuration uses a NAT bridge which starts with no physical devices attached. So it starts with a zero-ed out mac addr. Then every time you start a VM with a lower MAC address than the first VM's, you can see this pause. And if you shut down the VM with the lowest MACADDR, you'll again see the pause. ** Changed in: qemu-kvm (Ubuntu) Status: Incomplete = Confirmed ** Changed in: linux (Ubuntu) Status: Incomplete = Invalid ** Changed in: qemu-kvm (Ubuntu) Importance: Low = High -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/673705 Title: running guests freeze when a guest is powered down -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 673705] Re: running guests freeze when a guest is powered down
Hi Mathias, I've contacted upstream about this issue. For now, the simplest workaround would be to create a tap or veth NIC with a low macaddr and always keep it attached to your virbr0. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in Ubuntu. https://bugs.launchpad.net/bugs/673705 Title: running guests freeze when a guest is powered down -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 673705] Re: running guests freeze when a guest is powered down
** Package changed: qemu-kvm (Ubuntu) = libvirt (Ubuntu) ** Changed in: libvirt (Ubuntu) Assignee: Serge Hallyn (serge-hallyn) = (unassigned) -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to libvirt in Ubuntu. https://bugs.launchpad.net/bugs/673705 Title: running guests freeze when a guest is powered down -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 673705] Re: running guests freeze when a guest is powered down
Hi Mathias, This bug was reported a while ago and there hasn't been any activity in it recently. We were wondering if this is still an issue? Can you try with the latest development release of Ubuntu? ISO CD images are available from http://cdimage.ubuntu.com/releases/ . If it remains an issue, could you run the following command from a Terminal (Applications-Accessories-Terminal). It will automatically gather and attach updated debug information to this report. apport-collect -p linux 673705 Also, if you could test the latest upstream kernel available that would be great. It will allow additional upstream developers to examine the issue. Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Once you've tested the upstream kernel, please remove the 'needs-upstream-testing' tag. This can be done by clicking on the yellow pencil icon next to the tag located at the bottom of the bug description and deleting the 'needs-upstream-testing' text. Please let us know your results. Thanks in advance. [This is an automated message. Apologies if it has reached you inappropriately; please just reply to this message indicating so.] ** Tags added: needs-kernel-logs ** Tags added: needs-upstream-testing ** Tags added: kj-triage ** Changed in: linux (Ubuntu) Status: New = Incomplete -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. https://bugs.launchpad.net/bugs/673705 Title: running guests freeze when a guest is powered down -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 673705] Re: running guests freeze when a guest is powered down
Network manager is running. $ sudo iptables -nL Chain INPUT (policy ACCEPT) target prot opt source destination ACCEPT udp -- 0.0.0.0/00.0.0.0/0 udp dpt:53 ACCEPT tcp -- 0.0.0.0/00.0.0.0/0 tcp dpt:53 ACCEPT udp -- 0.0.0.0/00.0.0.0/0 udp dpt:67 ACCEPT tcp -- 0.0.0.0/00.0.0.0/0 tcp dpt:67 ACCEPT udp -- 0.0.0.0/00.0.0.0/0 udp dpt:53 ACCEPT tcp -- 0.0.0.0/00.0.0.0/0 tcp dpt:53 ACCEPT udp -- 0.0.0.0/00.0.0.0/0 udp dpt:67 ACCEPT tcp -- 0.0.0.0/00.0.0.0/0 tcp dpt:67 Chain FORWARD (policy ACCEPT) target prot opt source destination ACCEPT all -- 0.0.0.0/0192.168.233.0/24state RELATED,ESTABLISHED ACCEPT all -- 192.168.233.0/24 0.0.0.0/0 ACCEPT all -- 0.0.0.0/00.0.0.0/0 REJECT all -- 0.0.0.0/00.0.0.0/0 reject-with icmp-port-unreachable REJECT all -- 0.0.0.0/00.0.0.0/0 reject-with icmp-port-unreachable ACCEPT all -- 0.0.0.0/0192.168.122.0/24state RELATED,ESTABLISHED ACCEPT all -- 192.168.122.0/24 0.0.0.0/0 ACCEPT all -- 0.0.0.0/00.0.0.0/0 REJECT all -- 0.0.0.0/00.0.0.0/0 reject-with icmp-port-unreachable REJECT all -- 0.0.0.0/00.0.0.0/0 reject-with icmp-port-unreachable Chain OUTPUT (policy ACCEPT) target prot opt source destination $ sudo iptables -nL -t nat Chain PREROUTING (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination Chain POSTROUTING (policy ACCEPT) target prot opt source destination MASQUERADE tcp -- 192.168.233.0/24!192.168.233.0/24masq ports: 1024-65535 MASQUERADE udp -- 192.168.233.0/24!192.168.233.0/24masq ports: 1024-65535 MASQUERADE all -- 192.168.233.0/24!192.168.233.0/24 MASQUERADE tcp -- 192.168.122.0/24!192.168.122.0/24masq ports: 1024-65535 MASQUERADE udp -- 192.168.122.0/24!192.168.122.0/24masq ports: 1024-65535 MASQUERADE all -- 192.168.122.0/24!192.168.122.0/24 $ cat /etc/network/interfaces auto lo iface lo inet loopback Two guests: 179 was shutdown, 110 froze. Before: $ cat /proc/net/arp IP address HW type Flags HW addressMask Device 192.168.242.10x1 0x2 00:12:17:1a:50:47 *eth0 192.168.122.110 0x1 0x2 52:54:00:a2:4e:07 *virbr0 192.168.122.179 0x1 0x2 52:54:00:72:58:e3 *virbr0 While the freeze: $ cat /proc/net/arp IP address HW type Flags HW addressMask Device 192.168.242.10x1 0x2 00:12:17:1a:50:47 *eth0 192.168.122.110 0x1 0x2 52:54:00:a2:4e:07 *virbr0 192.168.122.179 0x1 0x2 52:54:00:72:58:e3 *virbr0 Just after the freeze stopped: $ cat /proc/net/arp IP address HW type Flags HW addressMask Device 192.168.242.10x1 0x2 00:12:17:1a:50:47 *eth0 192.168.122.110 0x1 0x2 52:54:00:a2:4e:07 *virbr0 192.168.122.179 0x1 0x2 52:54:00:72:58:e3 *virbr0 Some time after the freeze stop (guess: ~30 seconds): $ cat /proc/net/arp IP address HW type Flags HW addressMask Device 192.168.242.10x1 0x2 00:12:17:1a:50:47 *eth0 192.168.122.110 0x1 0x2 52:54:00:a2:4e:07 *virbr0 -- running guests freeze when a guest is powered down https://bugs.launchpad.net/bugs/673705 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 673705] Re: running guests freeze when a guest is powered down
Running https://bugs.launchpad.net/ubuntu/+source/qemu- kvm/+bug/616064/comments/33 recipe for 15 minutes didn't lead to a lock up. -- running guests freeze when a guest is powered down https://bugs.launchpad.net/bugs/673705 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 673705] Re: running guests freeze when a guest is powered down
Sorry, Mathiaz, that recipe tests for connections which just pause randomly. We need to test when a guest shuts down, or, in other words, when the veth device is removed from the bridge. So: 1. fire up a guest with libvirt. Monitor its network continuously (i.e. fire up a screen session over ssh doing while [ 1 ]; do echo -n .; sleep 5s; done and keep that open so you can see any pauses. 2. Get a usable ns_exec: git clone git://git.sr71.net/~hallyn/cr_tests.git cd cr_tests git checkout ns_exec make ns_exec cp ns_exec /bin/ 3. Create a veth tunnel sudo ip link add type veth 4. Open two root terminals to configure a network namespace for our test terminal 1: ip link add type veth terminal 2: /bin/ns_exec -cmn /bin/bash echo $$ # call this $pid henceforth terminal 1: ifconfig veth0 0.0.0.0 up brctl addif virbr0 veth0 ip link set veth1 netns $pid # use pid from above terminal 2: ifconfig veth1 up dhclient veth1 5. Now we want to emulate shutting down a libvirt guest. Let's try several ways: A. From the host root shell, just remove veth0 from the bridge: brctl delif virbr0 veth0 B. Shut down the veth interfaces. Try veth0 and veth1 on separate runs (ifconfig veth0 down). C. Just exit the child shell. D. Shut down the child shell, and then remove the veth interfaces altogether, by doing: ip link del veth0 After each test please remove the veth devices: ip link del veth0 Just to make sure that the commands in step 4 (referencing veth0/veth1) stay correct. -- running guests freeze when a guest is powered down https://bugs.launchpad.net/bugs/673705 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 673705] Re: running guests freeze when a guest is powered down
I assume you've not made any customizations to /etc/default/qemu-kvm or /etc/init/qemu-kvm.conf? I'll try to reproduce this tonight. -- running guests freeze when a guest is powered down https://bugs.launchpad.net/bugs/673705 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 673705] Re: running guests freeze when a guest is powered down
It seems that the network is actually frozen when another guest is shutdown. Sharing a screen session between an ssh connection and a console session (via virt-viewer) shows that only the ssh connection is frozen. The running guest is still running correctly as noticed with the console connection. -- running guests freeze when a guest is powered down https://bugs.launchpad.net/bugs/673705 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 673705] Re: running guests freeze when a guest is powered down
** Attachment added: qemu-kvm default file https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/673705/+attachment/1729601/+files/qemu-kvm -- running guests freeze when a guest is powered down https://bugs.launchpad.net/bugs/673705 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 673705] Re: running guests freeze when a guest is powered down
** Attachment added: qemu-kvm upstart job https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/673705/+attachment/1729602/+files/qemu-kvm.conf -- running guests freeze when a guest is powered down https://bugs.launchpad.net/bugs/673705 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 673705] Re: running guests freeze when a guest is powered down
This sounds most like comments #2 and #16 in bug 584048. It can't be bug #579892, because that patch is in maverick. Do you have wicd or network-manager running? Can you give the output of: iptables -L? cat /etc/network/interfaces cat /proc/net/arp (both before and during a network freeze) Also, could you follow the recipe at https://bugs.launchpad.net/ubuntu/+source/qemu- kvm/+bug/616064/comments/33 ? That should determine whether this is a kernel or libvirt/qemu bug. thanks, -serge ** Changed in: qemu-kvm (Ubuntu) Status: New = Incomplete ** Changed in: qemu-kvm (Ubuntu) Importance: Undecided = Medium ** Changed in: qemu-kvm (Ubuntu) Assignee: (unassigned) = Serge Hallyn (serge-hallyn) ** Changed in: qemu-kvm (Ubuntu) Importance: Medium = Low -- running guests freeze when a guest is powered down https://bugs.launchpad.net/bugs/673705 You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to qemu-kvm in ubuntu. -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
