Re: [Lxc-users] lxc container messing with hosts networking
On Sat, May 14, 2011 at 1:34 AM, Serge Hallyn serge.hal...@canonical.comwrote: Immediately I assume that your problem is that the mac addresses assigned to your containers are lower than that of the bridge. The bridge always takes on the lowest mac of any of its nics, so it'll change its mac address, which will temporarily drop the host's network connection if it's also part of that bridge. If I'm not mistaken you can specify a macaddr for your container using lxc.network.hwaddr=$x where x is a mac address - just make sure it's higher than the host's. Your diagnosis looks right to me, all the symptoms match with a mac addr change, and I've seen the br0 mac address change between the 2 ifaces on some tests. However, forcing kvm to assign the host a macaddr of 00:... and even seting that ip to the bridge in the hosts network/interfaces did not seem to solve the problem, I haven't seen the bridge changing mac addr anymore, but the 20-30 second hang is still there, so I don't know what to think, I've find few webs in the internet detailing the problem and solving it like that, but apparently it does not work for me. So, with a bridge with the lower macaddr I still experiment network hangs when starting containers, and even worse, when using the network from the container it hangs as well with a simple apt-get update on my ssh link laptop-host-guest. Somehow each time the container uses the network the host networking hangs, I suspect the guest has no problem with it though I can't see it because my ssh connection goes through the host and hangs there. Apart from that, I discovered that lxc.network.hwaddr= sets the internal macaddr of the container, but not the in-host generated interface that gets added to the bridge. Thanks Arkaitz -- Achieve unprecedented app performance and reliability What every C/C++ and Fortran developer should know. Learn how Intel has extended the reach of its next-generation tools to help boost performance applications - inlcuding clusters. http://p.sf.net/sfu/intel-dev2devmay___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxc container messing with hosts networking
I've setup a web server and do requests each 5 seconds from my laptop. Then start tcpdump in the host machine and after a while I do lxc-start. Inspecting later with wireshark, it looks like once the lxc guest finishes DHCP negotiation and setups the local IP address(10.0.2.17) any request to the host IP(10.0.2.15) is identified by the system as Unicast to another host and it sends the packet again trying to forward it, previous to the lxc guest dhcp it used to identify them as Unicast to us and answered them. The hosts br0 doesn't change the MAC at all as I can see it the same through ifconfig br0 in the kvm console window, besides, I'm setting the hosts eth mac address to very low so that it does not trigger any bridge mac update. Any hints? -- Arkaitz -- Achieve unprecedented app performance and reliability What every C/C++ and Fortran developer should know. Learn how Intel has extended the reach of its next-generation tools to help boost performance applications - inlcuding clusters. http://p.sf.net/sfu/intel-dev2devmay___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [PATCH] ignore non-lxc configuration line
Quoting David Serrano (dserra...@gmail.com): On Sat, May 14, 2011 at 00:15, Serge Hallyn serge.hal...@canonical.com wrote: I'm curious, whatcha got in mind? I don't think you have to have something in mind to implement this. Just that old motto Be lenient in what you accept :). So if I type 'lcx.' instead of 'lxc.', as I often do, it'll silently ignore it? No, that's a bad idea. In any case I wasn't (until now) doubting Daniel's motivations, rather I was pretty sure he had something neat in mind. -serge -- Achieve unprecedented app performance and reliability What every C/C++ and Fortran developer should know. Learn how Intel has extended the reach of its next-generation tools to help boost performance applications - inlcuding clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxc container messing with hosts networking
Quoting arkai...@gmail.com (arkai...@gmail.com): I've setup a web server and do requests each 5 seconds from my laptop. Then start tcpdump in the host machine and after a while I do lxc-start. Inspecting later with wireshark, it looks like once the lxc guest finishes DHCP negotiation and setups the local IP address(10.0.2.17) any request to the host IP(10.0.2.15) is identified by the system as Unicast to another host and it sends the packet again trying to forward it, previous to the lxc guest dhcp it used to identify them as Unicast to us and answered them. The hosts br0 doesn't change the MAC at all as I can see it the same through ifconfig br0 in the kvm console window, besides, I'm setting the hosts eth mac address to very low so that it does not trigger any bridge mac update. Any hints? Make sure stp is on on the bridge inside your kvm guest. If that doesn't work, I'll just have to try and reproduce, but you'll probably need someone more network-savvy than me to look into it. I'll set up a test environment later this weekend. -serge -- Achieve unprecedented app performance and reliability What every C/C++ and Fortran developer should know. Learn how Intel has extended the reach of its next-generation tools to help boost performance applications - inlcuding clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] LVM in LXC
Hi, That's still doable, just a bit more work. Take a look at ls -l /dev/lxc (or whatever is the vg you're looking at). It has symlinks to the real devices. When you look at the link targets, you can find their maj:min. For me, serge@sergelap:~$ ls -l /dev/lxc total 0 lrwxrwxrwx 1 root root 7 2011-05-13 17:26 build1 - ../dm-1 lrwxrwxrwx 1 root root 7 2011-05-13 17:26 delme - ../dm-4 lrwxrwxrwx 1 root root 7 2011-05-13 17:26 nattylvm - ../dm-0 serge@sergelap:~$ ls -l /dev/dm-1 brw-rw 1 root disk 252, 1 2011-05-13 17:26 /dev/dm-1 So if I only wanted /dev/lxc/build1 to be available to container nattylvm, then in it's config I would keep the existing lxc.cgroup.devices entries, and add lxc.cgroup.devices.allow = b 252:1 rwm To actually give the container access to the vg so it can create LVM devices, I'm afraid I don't know enough about how lvcreate to be sure. But here's my guess (based on a quick read of strace -f lvcreate output): Use a different physical partition for each container's pv, and give the container full access to that partition. Then pvscan/pvcreate will have access to the full drive, and all metadata is on there. vgscan/vgcreate and lvscan/lvcreate likewise I believe will then be able to create vgs and lvs on that partition. That's what I was basically trying to do (and doesn't work this way as far as I can see). Currently I'm granting access to specific /dev/dm-* files to the container. For example: /dev/dm-2 is the partition/logical volume of vm0 with maj:min 252:2. So I set lxc.cgroup.devices.allow = b 252:2 rwm. In the container I create a vg on /dev/dm-2 (works so far) with name vg-vm0. Then I create a logical volume on vg-vm0 in the container. This pseudo-fails as the container doesn't have the rights to create any /dev/dm-* (or else an container could just create /dev/dm-n and access data on other logical volumes). On the host system the corresponding /dev/dm-7 of the new container lv has been created and I grant access to create the device node to the container: lxc.cgroup.devices.allow = b 252:7 rwm. vm0 is now able to create the device node and access the new lv. So either users have to contact me each time they want to create a new logical volume in their vm (so I can enable device node access) or they can create arbitrary /dev/dm-* nodes and access data from other users. Regards, Benjamin signature.asc Description: Digital signature -- Achieve unprecedented app performance and reliability What every C/C++ and Fortran developer should know. Learn how Intel has extended the reach of its next-generation tools to help boost performance applications - inlcuding clusters. http://p.sf.net/sfu/intel-dev2devmay___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxc container messing with hosts networking
On Sat, May 14, 2011 at 2:39 PM, Serge Hallyn serge.hal...@canonical.comwrote: Make sure stp is on on the bridge inside your kvm guest. If that doesn't work, I'll just have to try and reproduce, but you'll probably need someone more network-savvy than me to look into it. I'll set up a test environment later this weekend. -serge Tried enabling stp but nothing improved. I'm trying to come up with a script that automates the env setup, will send it later on. Thanks Arkaitz -- Achieve unprecedented app performance and reliability What every C/C++ and Fortran developer should know. Learn how Intel has extended the reach of its next-generation tools to help boost performance applications - inlcuding clusters. http://p.sf.net/sfu/intel-dev2devmay___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] LVM in LXC
Quoting Benjamin Kiessling (mittages...@l.unchti.me): Hi, That's still doable, just a bit more work. Take a look at ls -l /dev/lxc (or whatever is the vg you're looking at). It has symlinks to the real devices. When you look at the link targets, you can find their maj:min. For me, serge@sergelap:~$ ls -l /dev/lxc total 0 lrwxrwxrwx 1 root root 7 2011-05-13 17:26 build1 - ../dm-1 lrwxrwxrwx 1 root root 7 2011-05-13 17:26 delme - ../dm-4 lrwxrwxrwx 1 root root 7 2011-05-13 17:26 nattylvm - ../dm-0 serge@sergelap:~$ ls -l /dev/dm-1 brw-rw 1 root disk 252, 1 2011-05-13 17:26 /dev/dm-1 So if I only wanted /dev/lxc/build1 to be available to container nattylvm, then in it's config I would keep the existing lxc.cgroup.devices entries, and add lxc.cgroup.devices.allow = b 252:1 rwm To actually give the container access to the vg so it can create LVM devices, I'm afraid I don't know enough about how lvcreate to be sure. But here's my guess (based on a quick read of strace -f lvcreate output): Use a different physical partition for each container's pv, and give the container full access to that partition. Then pvscan/pvcreate will have access to the full drive, and all metadata is on there. vgscan/vgcreate and lvscan/lvcreate likewise I believe will then be able to create vgs and lvs on that partition. That's what I was basically trying to do (and doesn't work this way as far as I can see). Currently I'm granting access to specific /dev/dm-* files to the container. For example: /dev/dm-2 is the partition/logical volume of vm0 with maj:min 252:2. So I set lxc.cgroup.devices.allow = b 252:2 rwm. In the container I create a vg on /dev/dm-2 (works so far) with name vg-vm0. Then I create a logical volume on vg-vm0 in the container. This pseudo-fails as the container doesn't have the rights to create any /dev/dm-* (or else an container could just create /dev/dm-n and access data on other logical volumes). On the host system the corresponding /dev/dm-7 of the new container lv has been created and I grant access to create the device node to the container: lxc.cgroup.devices.allow = b 252:7 rwm. vm0 is now able to create the device node and access the new lv. So either users have to contact me each time they want to create a new logical volume in their vm (so I can enable device node access) or they can create arbitrary /dev/dm-* nodes and access data from other users. Ah yeah. Of course. I wonder if there is a not-too-hacky way that we could prealloc certain dm-N ranges to containers, and get those to be used at lvcreate. -serge -- Achieve unprecedented app performance and reliability What every C/C++ and Fortran developer should know. Learn how Intel has extended the reach of its next-generation tools to help boost performance applications - inlcuding clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxc container messing with hosts networking
Quoting arkai...@gmail.com (arkai...@gmail.com): On Sat, May 14, 2011 at 2:39 PM, Serge Hallyn serge.hal...@canonical.comwrote: Make sure stp is on on the bridge inside your kvm guest. If that doesn't work, I'll just have to try and reproduce, but you'll probably need someone more network-savvy than me to look into it. I'll set up a test environment later this weekend. -serge Tried enabling stp but nothing improved. I'm trying to come up with a script that automates the env setup, will send it later on. Hm, I just did this on natty (natty host, natty kvm VM, with a natty container inside that) and could actually not reproduce your problem. Just a normal bridge on the kvm VM: root@lxc-natty-amd64:~# brctl show bridge name bridge id STP enabled interfaces br0 8000.001636dd34bc no eth0 And the lxc container was created with a minimal normal config: lxc.network.type=veth lxc.network.link=br0 lxc.network.flags=up So I guess I may have to try to reproduce on debian. -serge -- Achieve unprecedented app performance and reliability What every C/C++ and Fortran developer should know. Learn how Intel has extended the reach of its next-generation tools to help boost performance applications - inlcuding clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxc container messing with hosts networking
Hi, Hm, I just did this on natty (natty host, natty kvm VM, with a natty container inside that) and could actually not reproduce your problem. Just a normal bridge on the kvm VM: root@lxc-natty-amd64:~# brctl show bridge name bridge id STP enabled interfaces br0 8000.001636dd34bc no eth0 And the lxc container was created with a minimal normal config: lxc.network.type=veth lxc.network.link=br0 lxc.network.flags=up So I guess I may have to try to reproduce on debian. Weird, I doubt is debian only, it has to be something from my setup. Check http://pastebin.com/zZXWmCF8 , I created this script 98% automated that will setup my env so that you see if there is something wrong. I'm running this on Ubuntu 10.10 Thanks Arkaitz -- Achieve unprecedented app performance and reliability What every C/C++ and Fortran developer should know. Learn how Intel has extended the reach of its next-generation tools to help boost performance applications - inlcuding clusters. http://p.sf.net/sfu/intel-dev2devmay___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
