Re: [Lxc-users] LVM in LXC
Hi, That's still doable, just a bit more work. Take a look at ls -l /dev/lxc (or whatever is the vg you're looking at). It has symlinks to the real devices. When you look at the link targets, you can find their maj:min. For me, serge@sergelap:~$ ls -l /dev/lxc total 0 lrwxrwxrwx 1 root root 7 2011-05-13 17:26 build1 - ../dm-1 lrwxrwxrwx 1 root root 7 2011-05-13 17:26 delme - ../dm-4 lrwxrwxrwx 1 root root 7 2011-05-13 17:26 nattylvm - ../dm-0 serge@sergelap:~$ ls -l /dev/dm-1 brw-rw 1 root disk 252, 1 2011-05-13 17:26 /dev/dm-1 So if I only wanted /dev/lxc/build1 to be available to container nattylvm, then in it's config I would keep the existing lxc.cgroup.devices entries, and add lxc.cgroup.devices.allow = b 252:1 rwm To actually give the container access to the vg so it can create LVM devices, I'm afraid I don't know enough about how lvcreate to be sure. But here's my guess (based on a quick read of strace -f lvcreate output): Use a different physical partition for each container's pv, and give the container full access to that partition. Then pvscan/pvcreate will have access to the full drive, and all metadata is on there. vgscan/vgcreate and lvscan/lvcreate likewise I believe will then be able to create vgs and lvs on that partition. That's what I was basically trying to do (and doesn't work this way as far as I can see). Currently I'm granting access to specific /dev/dm-* files to the container. For example: /dev/dm-2 is the partition/logical volume of vm0 with maj:min 252:2. So I set lxc.cgroup.devices.allow = b 252:2 rwm. In the container I create a vg on /dev/dm-2 (works so far) with name vg-vm0. Then I create a logical volume on vg-vm0 in the container. This pseudo-fails as the container doesn't have the rights to create any /dev/dm-* (or else an container could just create /dev/dm-n and access data on other logical volumes). On the host system the corresponding /dev/dm-7 of the new container lv has been created and I grant access to create the device node to the container: lxc.cgroup.devices.allow = b 252:7 rwm. vm0 is now able to create the device node and access the new lv. So either users have to contact me each time they want to create a new logical volume in their vm (so I can enable device node access) or they can create arbitrary /dev/dm-* nodes and access data from other users. Regards, Benjamin signature.asc Description: Digital signature -- Achieve unprecedented app performance and reliability What every C/C++ and Fortran developer should know. Learn how Intel has extended the reach of its next-generation tools to help boost performance applications - inlcuding clusters. http://p.sf.net/sfu/intel-dev2devmay___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] LVM in LXC
Quoting Benjamin Kiessling (mittages...@l.unchti.me): Hi, That's still doable, just a bit more work. Take a look at ls -l /dev/lxc (or whatever is the vg you're looking at). It has symlinks to the real devices. When you look at the link targets, you can find their maj:min. For me, serge@sergelap:~$ ls -l /dev/lxc total 0 lrwxrwxrwx 1 root root 7 2011-05-13 17:26 build1 - ../dm-1 lrwxrwxrwx 1 root root 7 2011-05-13 17:26 delme - ../dm-4 lrwxrwxrwx 1 root root 7 2011-05-13 17:26 nattylvm - ../dm-0 serge@sergelap:~$ ls -l /dev/dm-1 brw-rw 1 root disk 252, 1 2011-05-13 17:26 /dev/dm-1 So if I only wanted /dev/lxc/build1 to be available to container nattylvm, then in it's config I would keep the existing lxc.cgroup.devices entries, and add lxc.cgroup.devices.allow = b 252:1 rwm To actually give the container access to the vg so it can create LVM devices, I'm afraid I don't know enough about how lvcreate to be sure. But here's my guess (based on a quick read of strace -f lvcreate output): Use a different physical partition for each container's pv, and give the container full access to that partition. Then pvscan/pvcreate will have access to the full drive, and all metadata is on there. vgscan/vgcreate and lvscan/lvcreate likewise I believe will then be able to create vgs and lvs on that partition. That's what I was basically trying to do (and doesn't work this way as far as I can see). Currently I'm granting access to specific /dev/dm-* files to the container. For example: /dev/dm-2 is the partition/logical volume of vm0 with maj:min 252:2. So I set lxc.cgroup.devices.allow = b 252:2 rwm. In the container I create a vg on /dev/dm-2 (works so far) with name vg-vm0. Then I create a logical volume on vg-vm0 in the container. This pseudo-fails as the container doesn't have the rights to create any /dev/dm-* (or else an container could just create /dev/dm-n and access data on other logical volumes). On the host system the corresponding /dev/dm-7 of the new container lv has been created and I grant access to create the device node to the container: lxc.cgroup.devices.allow = b 252:7 rwm. vm0 is now able to create the device node and access the new lv. So either users have to contact me each time they want to create a new logical volume in their vm (so I can enable device node access) or they can create arbitrary /dev/dm-* nodes and access data from other users. Ah yeah. Of course. I wonder if there is a not-too-hacky way that we could prealloc certain dm-N ranges to containers, and get those to be used at lvcreate. -serge -- Achieve unprecedented app performance and reliability What every C/C++ and Fortran developer should know. Learn how Intel has extended the reach of its next-generation tools to help boost performance applications - inlcuding clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] LVM in LXC
On 05/13/2011 12:13 AM, Benjamin Kiessling wrote: Hi, under Debian (and in general I think) LVM requires udev to work at all which makes it unusable in a container environment. Has anybody tried to get it working in a container? You can use udev inside a container. It is not optimal because that trigger events everywhere but it is possible. My setup consists of a logical volume that's mapped in the container which the container user should be able to subdivide into partitions (i.e. in the end I'd have a chain like pg-vg-lv-pg-vg-lv or LVM on an logical volume if that's more clear). Is there another way to achieve this kind of setup? I thought about letting users just partition the raw logical volume like any other hard disk but this doesn't seem to be supported by the kernel. Maybe Serge can help you on that. Thanks -- Daniel -- Achieve unprecedented app performance and reliability What every C/C++ and Fortran developer should know. Learn how Intel has extended the reach of its next-generation tools to help boost performance applications - inlcuding clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] LVM in LXC
Quoting Daniel Lezcano (dlezc...@fr.ibm.com): On 05/13/2011 12:13 AM, Benjamin Kiessling wrote: Hi, under Debian (and in general I think) LVM requires udev to work at all which makes it unusable in a container environment. Has anybody tried to get it working in a container? You can use udev inside a container. It is not optimal because that trigger events everywhere but it is possible. What is your host? Which OS/release and which kernel version? My setup consists of a logical volume that's mapped in the container which the container user should be able to subdivide into partitions (i.e. in the end I'd have a chain like pg-vg-lv-pg-vg-lv or LVM on an logical volume if that's more clear). Is there another way to achieve this kind of setup? I thought about letting users just partition the raw logical volume like any other hard disk but this doesn't seem to be supported by the kernel. Maybe Serge can help you on that. It works fine for me. I've got a natty host with natty guest (itself backed on an lvm partition :). I did apt-get install lvm2, powered down, edit /var/lib/lvmtest/config and deleted all lxc.cgroup.devices lines, started the container back up, and all my lvm partitions appeared under /dev/lxc/. -serge -- Achieve unprecedented app performance and reliability What every C/C++ and Fortran developer should know. Learn how Intel has extended the reach of its next-generation tools to help boost performance applications - inlcuding clusters. http://p.sf.net/sfu/intel-dev2devmay ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
[Lxc-users] LVM in LXC
Hi, under Debian (and in general I think) LVM requires udev to work at all which makes it unusable in a container environment. Has anybody tried to get it working in a container? My setup consists of a logical volume that's mapped in the container which the container user should be able to subdivide into partitions (i.e. in the end I'd have a chain like pg-vg-lv-pg-vg-lv or LVM on an logical volume if that's more clear). Is there another way to achieve this kind of setup? I thought about letting users just partition the raw logical volume like any other hard disk but this doesn't seem to be supported by the kernel. Best Regards, Benjamin Kiessling signature.asc Description: Digital signature -- Achieve unprecedented app performance and reliability What every C/C++ and Fortran developer should know. Learn how Intel has extended the reach of its next-generation tools to help boost performance applications - inlcuding clusters. http://p.sf.net/sfu/intel-dev2devmay___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
