Thanks much.. This helps RegardsMohan From: Fajar A. Nugraha <l...@fajar.net> To: LXC users mailing-list <lxc-users@lists.linuxcontainers.org> Sent: Friday, March 6, 2015 12:30 PM Subject: Re: [lxc-users] nested containers This is on ubuntu 14.10, lxc and lxcfs from ppa:ubuntu-lxc/daily
Test memory limit on the parent container, by writing data to /run/shm (a tmpfs mount): @host # lxc-cgroup -n v memory.use_hierarchy 1 @host # lxc-cgroup -n v memory.limit_in_bytes 1G @host # lxc-cgroup -n v memory.limit_in_bytes 1073741824 @host # lxc-attach -n v -- mount | grep shm none on /run/shm type tmpfs (rw,nosuid,nodev,relatime) @host # lxc-cgroup -n v memory.usage_in_bytes 12881920 @host # lxc-attach -n v -- dd if=/dev/zero of=/run/shm/testfile dd: writing to ‘/run/shm/testfile’: Cannot allocate memory 2076473+0 records in 2076472+0 records out 1063153664 bytes (1.1 GB) copied, 2.18223 s, 487 MB/s @host # lxc-cgroup -n v memory.usage_in_bytes 1073561600 See how it's limited to around 1GB? Now delete the file before running more test to free up memory. @host # lxc-attach -n v -- rm /run/shm/testfile @host # lxc-cgroup -n v memory.usage_in_bytes 10219520 Now test the limit on the nested container. We test with memory limit 2G. It should max out at 1G, since the parent only has that much limit. @host # lxc-attach -n v root@v:~# lxc-cgroup -n nv memory.limit_in_bytes 2G root@v:~# lxc-cgroup -n nv memory.limit_in_bytes 2147483648 root@v:~# lxc-cgroup -n nv memory.usage_in_bytes 7045120 root@v:~# lxc-attach -n nv -- dd if=/dev/zero of=/run/shm/testfile dd: writing to ‘/run/shm/testfile’: Cannot allocate memory 2080265+0 records in 2080264+0 records out 1065095168 bytes (1.1 GB) copied, 2.96393 s, 359 MB/s root@v:~# lxc-cgroup -n nv memory.usage_in_bytes lxc_container: lxc_cgroup.c: main: 113 failed to retrieve value of 'memory.usage_in_bytes' for '/var/lib/lxc:nv' root@v:~# lxc-ls -f Killed See how the testfile on the nested container is also limited at around 1GB? Note that at this time the container "v" (the parent) can't do anything (it can't even run "lxc-ls"), because it wants to allocate more memory, and all available memory has been used by the child container "nv". Since we can't run anything on the parent container due to memory limit, kill the nested container root@v:~# lxc-stop -k -n nv lxc-stop: commands.c: lxc_cmd_stop: 615 failed to stop 'nv': Operation not permitted root@v:~# lxc-stop -k -n nv nv is not running root@v:~# lxc-ls -f NAME STATE IPV4 IPV6 GROUPS AUTOSTART -------------------------------------------- nv STOPPED - - - NO So in short, the nested limit works. To make it operable, make sure that the parent container ALWAYS have some free memory to work with. -- Fajar On Fri, Mar 6, 2015 at 1:19 PM, Mohan G <mohan...@yahoo.com> wrote: > Hi Folks, > Let me explain my problem and then you can suggest me some way of over > coming this. > I wan to be able to run different protocol clients to use my file system > mounted on the host. But i want to be able to limit their memory resource to > 10Gb in total. Also, be able to set memory.sw limit so that each can have a > softy limit of 5G. But if only one container is runing, it can use the > entire 10G. I am able to use memory.use_hierarchy in cgroups to acheive the > same. But how do i acheive this with container. > Will nesting of containers help. my plan is to create a parent container and > set limits to this and expect the nested conatiners to inherit these > limit, and then i will set soft limits on these children container. But when > i start nested containers, i don;t see any entrry under /sys/fs/groups for > the children container. > > Regards > Mohan > > > ________________________________ > From: Fajar A. Nugraha <l...@fajar.net> > To: LXC users mailing-list <lxc-users@lists.linuxcontainers.org> > Sent: Wednesday, March 4, 2015 2:44 PM > Subject: Re: [lxc-users] nested containers > > On Wed, Mar 4, 2015 at 12:15 PM, Mohan G <mohan...@yahoo.com> wrote: > > > >> Hi, >> Is there anyway we can have nested containers/cgroups. One parent >> container >> forming the basis for children containers. i.e subset of parent container. > > > Yes. > > On parent container config (in ubuntu), add this: > lxc.aa_profile=lxc-container-default-with-nesting > > And then on that container, you can create containers > > utopic ~ # lxc-ls -f --running > NAME STATE IPV4 IPV6 GROUPS AUTOSTART > ----------------------------------------------------------------- > v RUNNING 10.0.3.1, 192.168.124.173 - - NO > > utopic ~ # lxc-attach -n v > > root@v:~# > > root@v:~# cat /proc/1/cgroup > 12:name=systemd:/lxc/v > 11:perf_event:/lxc/v > 10:net_prio:/lxc/v > 9:net_cls:/lxc/v > 8:memory:/lxc/v > 7:hugetlb:/lxc/v > 6:freezer:/lxc/v > 5:devices:/lxc/v > 4:cpuset:/lxc/v > 3:cpuacct:/lxc/v > 2:cpu:/lxc/v > 1:blkio:/lxc/v > > root@v:~# lxc-create -t download -n nv -- -d ubuntu -r vivid -a amd64 > Using image from local cache > Unpacking the rootfs > > --- > You just created an Ubuntu container (release=vivid, arch=amd64, > variant=default) > > To enable sshd, run: apt-get install openssh-server > > For security reason, container images ship without user accounts > and without a root password. > > Use lxc-attach or chroot directly into the rootfs to set a root password > or create user accounts. > > root@v:~# lxc-start -n nv > > root@v:~# lxc-ls -f --running > NAME STATE IPV4 IPV6 GROUPS AUTOSTART > -------------------------------------------------- > nv RUNNING 10.0.3.249 - - NO > > > > Now run a process inside the nested container > > root@v:~# lxc-attach -n nv -- cat /proc/1/cgroup > 12:name=systemd:/lxc/v/lxc/nv > 11:perf_event:/lxc/v/lxc/nv > 10:net_prio:/lxc/v/lxc/nv > 9:net_cls:/lxc/v/lxc/nv > 8:memory:/lxc/v/lxc/nv > 7:hugetlb:/lxc/v/lxc/nv > 6:freezer:/lxc/v/lxc/nv > 5:devices:/lxc/v/lxc/nv > 4:cpuset:/lxc/v/lxc/nv > 3:cpuacct:/lxc/v/lxc/nv > 2:cpu:/lxc/v/lxc/nv > 1:blkio:/lxc/v/lxc/nv > > Note how the cgroup is nested > > -- > Fajar > _______________________________________________ > lxc-users mailing list > lxc-users@lists.linuxcontainers.org > http://lists.linuxcontainers.org/listinfo/lxc-users > > > _______________________________________________ > lxc-users mailing list > lxc-users@lists.linuxcontainers.org > http://lists.linuxcontainers.org/listinfo/lxc-users _______________________________________________ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users