[Lxc-users] Mounting the rootfs as read-only in Ubuntu
Hi,
I know there have been a few threads on this, such as
http://comments.gmane.org/gmane.linux.kernel.containers.lxc.general/2167
http://comments.gmane.org/gmane.linux.kernel.containers.lxc.general/1673
but none have been able to solve my problem so far.
My problem: In Ubuntu 11.10, I'm using the default lxc-ubuntu template
script to create a rootfs + setup container config. As soon as this script
finishes, I open the config file and change lxc.rootfs to point to a
directory that is empty.
I then modify the fstab so that the rootfs that was generated by
debootstrap is mounted into this empty directory as read-only.
Now when I try to start the container. I get a warning the /lib/init/fstab
is mounted as read-only and Ubuntu fails to start (I read something about
lxcguest trying to replace this file with a empty version).
So then modify the fstab a second time to mount a RW mount onto
/lib/init/fstab, and the warning disappears. But Ubuntu still does not
start.
Note that if I mount the filesystem as RW, then it starts up correctly.
Here are my config/fstab/log files
*config:*
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0
lxc.network.ipv4 = 0.0.0.0/24
lxc.utsname = amd64
lxc.tty = 4
lxc.pts = 1024
lxc.rootfs = /var/lib/lxc/xxx/rootfs
lxc.mount = /var/lib/lxc/xxx/fstab
lxc.arch = amd64
lxc.cgroup.devices.deny = a
# Allow any mknod (but not using the node)
lxc.cgroup.devices.allow = c *:* m
lxc.cgroup.devices.allow = b *:* m
# /dev/null and zero
lxc.cgroup.devices.allow = c 1:3 rwm
lxc.cgroup.devices.allow = c 1:5 rwm
# consoles
lxc.cgroup.devices.allow = c 5:1 rwm
lxc.cgroup.devices.allow = c 5:0 rwm
#lxc.cgroup.devices.allow = c 4:0 rwm
#lxc.cgroup.devices.allow = c 4:1 rwm
# /dev/{,u}random
lxc.cgroup.devices.allow = c 1:9 rwm
lxc.cgroup.devices.allow = c 1:8 rwm
lxc.cgroup.devices.allow = c 136:* rwm
lxc.cgroup.devices.allow = c 5:2 rwm
# rtc
lxc.cgroup.devices.allow = c 254:0 rwm
#fuse
lxc.cgroup.devices.allow = c 10:229 rwm
*fstab:*
#/home/martin/rootfs contains the files downloaded by debootstrap
#/var/lib/lxc/xxx/rootfs is an empty directory
/home/martin/rootfs /var/lib/lxc/xxx/rootfs
none ro,bind 0 0
#/home/martin/init was copied from /home/martin/rootfs/init. It is mounted
as rw so that lxcguest can do its thing with /lib/init/fstab
/home/martin/init /var/lib/lxc/xxx/rootfs/lib/init/
nonerw,bind 0 0
proc/var/lib/lxc/xxx/rootfs/procproc
nodev,noexec,nosuid 0 0
sysfs /var/lib/lxc/xxx/rootfs/sys sysfs
defaults 0 0
*Log file on startup*
lxc-start 1327948980.704 DEBUGlxc_conf - allocated pty
'/dev/pts/1' (4/5)
lxc-start 1327948980.704 DEBUGlxc_conf - allocated pty
'/dev/pts/2' (6/7)
lxc-start 1327948980.704 DEBUGlxc_conf - allocated pty
'/dev/pts/3' (8/9)
lxc-start 1327948980.704 DEBUGlxc_conf - allocated pty
'/dev/pts/4' (10/11)
lxc-start 1327948980.704 INFO lxc_conf - tty's configured
lxc-start 1327948980.704 DEBUGlxc_console - using '/dev/tty' as
console
lxc-start 1327948980.704 DEBUGlxc_start - sigchild handler set
lxc-start 1327948980.704 INFO lxc_start - 'xxx' is initialized
lxc-start 1327948980.708 DEBUGlxc_conf - instanciated veth
'vetha5XM5V/vethaTEtvU', index is '10'
lxc-start 1327948980.709 DEBUGlxc_cgroup - checking '/' (rootfs)
lxc-start 1327948980.709 DEBUGlxc_cgroup - checking '/sys' (sysfs)
lxc-start 1327948980.709 DEBUGlxc_cgroup - checking '/proc' (proc)
lxc-start 1327948980.709 DEBUGlxc_cgroup - checking '/dev'
(devtmpfs)
lxc-start 1327948980.709 DEBUGlxc_cgroup - checking '/dev/pts'
(devpts)
lxc-start 1327948980.709 DEBUGlxc_cgroup - checking '/run' (tmpfs)
lxc-start 1327948980.709 DEBUGlxc_cgroup - checking
'/sys/fs/fuse/connections' (fusectl)
lxc-start 1327948980.709 DEBUGlxc_cgroup - checking '/' (ext4)
lxc-start 1327948980.709 DEBUGlxc_cgroup - checking
'/sys/fs/cgroup' (tmpfs)
lxc-start 1327948980.709 DEBUGlxc_cgroup - checking
'/sys/fs/cgroup/cpuset' (cgroup)
lxc-start 1327948980.709 INFO lxc_cgroup - found cgroup mounted
at '/sys/fs/cgroup/cpuset'
lxc-start 1327948980.709 DEBUGlxc_cgroup - cgroup
/sys/fs/cgroup/cpuset has flags 0x2
lxc-start 1327948980.709 INFO lxc_cgroup - created cgroup
'/sys/fs/cgroup/cpuset/xxx'
lxc-start 1327948980.709 DEBUGlxc_cgroup - checking
'/sys/fs/cgroup/cpu' (cgroup)
lxc-start 1327948980.709 INFO lxc_cgroup - found cgroup mounted
at '/sys/fs/cgroup/cpu'
lxc-start 1327948980.709 DEBUGlxc_cgroup - cgroup
/sys/fs/cgroup/cpu has flags 0x2
lxc-start 1327948980.709 INFO lxc_cgroup - created cgroup
'/sys/fs/cgroup/cpu/xxx'
lxc-start 1327948980.709 DEBUGlxc_cgroup - checking
'/sys/fs/cgroup/cpuacct' (cgroup)
lxc-start 1327948980.709 INFO
Re: [Lxc-users] Doesn't auditd work on an LXC instance?
You need to give us way more information than that... What is the error message? What distribution of Linux is this for? Martin On Thu, Jan 19, 2012 at 11:00 AM, David Kang dk...@isi.edu wrote: Hi, I'm trying to run auditd on an LXC instance. First of all, I cannot make kauditd start. And $ service auditd start always fails. Does it mean auditd does not work on an LXC instance? I'll appreciate your help. David. -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users -- Open source radio in the cloud. Get yours now! --- http://airtime.pro Martin Konecny Software Developer, Sourcefabric martin.kone...@sourcefabric.org 720 Bathurst St. Suite 203 M5S 2R4, Toronto, ON, Canada +1 (416) 892-8420 (Cell) Skype: martin.konecny15 http://www.sourcefabric.org http://www.twitter.com/Sourcefabric -- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] Doesn't auditd work on an LXC instance?
On Thu, Jan 19, 2012 at 11:00 AM, David Kang dk...@isi.edu wrote: Hi, I'm trying to run auditd on an LXC instance. First of all, I cannot make kauditd start. And $ service auditd start always fails. Does it mean auditd does not work on an LXC instance? I'll appreciate your help. Sorry meant to respond to this earlier. Auditd won't work on an lxc instance, because you can't open that netlink socket in a non-init network namespace. -serge -- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2 ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] PostgreSQL - sh: cannot create /dev/null: Permission denied - LXC Issue?
Just wanted to fill everyone in. I edited the permissions of
/lxc/PE1800-db0/dev/null as suggested, restarted the container and all seems to
be well now. I also changed permissions of /dev/null on my other containers
(and my template), as I assume this is the correct setup.
Thanks for your help,
Pat
- Original Message -
From: Christoph Willing cwill...@users.sourceforge.net
To: lxc-users lxc-users@lists.sourceforge.net
Sent: Saturday, January 28, 2012 7:58:07 AM
Subject: Re: [Lxc-users] PostgreSQL - sh: cannot create /dev/null:
Permission denied - LXC Issue?
On 25/01/2012, at 3:54 AM, Patrick Kevin McCaffrey wrote:
On 24/01/2012, at 7:27 AM, Patrick Kevin McCaffrey wrote:
...
I'm working on a database container, but when I run:
/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data
to set up my database, it errors out with:
sh: cannot create /dev/null: Permission denied
fgets failure: Success
The program postgres is needed by initdb but was not found in the
same directory as /usr/local/pgsql/bin/initdb.
Check your installation.
Try adding this to config file:
lxc.cgroup.devices.allow = c 1:3 rwm
##This line is already in my config file. The entire config file is
further below.
This allows the container to read/write/mknod
character device major 1, minor 3
tmike@tmike-Inspiron-1464:~/lxc/oneiric$ ls -l /dev/null
crw-rw-rw- 1 root root 1, 3 2012-01-13 13:45 /dev/null
This should take care of the
sh: cannot create /dev/null: Permission denied
## Again, sorry for the how long it took me to reply. Getting this
system setup is sort of a side project, and other things have gotten
in the way of me working on it lately. Anyway, When I ls -l on /
dev/nul, I get the following:
crw-r--r-- 1 root root 1, 3 Jan 16 23:24 null
What are the permissions on /dev/null before you start the container?
i.e. what is output of ls -l /lxc/PE1800-db0/rootfs/dev/null ?
Probably 644 - if so, fix the permissions for that location, then try
running the container again.
Something else you could perhaps try - just temporarily till you find
the real cause of the problem - is to comment out the config line:
lxc.cgroup.devices.deny = a
chris
Here is the output of ls -l before starting the containter:
pat@PowerEdge1800:/lxc$ ls -l /lxc/PE1800-db0/rootfs/dev/null
crw-r--r-- 1 root root 1, 3 Jan 16 23:24 /lxc/PE1800-db0/rootfs/dev/
null
Thats not good - it should look like:
chris@v1:~$ ls -l /var/lib/lxc/v1video/rootfs/dev/null
crw-rw-rw- 1 root root 1, 3 Jul 20 2011 /var/lib/lxc/v1video/rootfs/
dev/null
You can change yours with:
sudo chmod go+w /lxc/PE1800-db0/rootfs/dev/null
Now run the container.
I haven't tried removing lxc.cgroup.devices.deny = a yet, but it
seems like that's my next step.
That won't help if the initial permissions are not fixed (as suggested
above) and won't be needed when the permissions are fixed.
Of course this doesn't explain how the permissions became wrong in the
first place ...
chris
I'm just lost as to where I've gone wrong.
-Pat
It looks like I definitely don't have write support, if I understand
that output correctly. Here is my entire config file for this
container:
-
lxc.utsname = PE1800-db0
lxc.tty = 4
lxc.pts = 1024
lxc.rootfs = /lxc/PE1800-db0/rootfs
lxc.cgroup.devices.deny = a
lxc.network.type = veth
lxc.network.link = br0
#lxc.network.veth.pair =
lxc.network.ipv4 = 192.168.80.4
# /dev/null and zero
lxc.cgroup.devices.allow = c 1:3 rwm
lxc.cgroup.devices.allow = c 1:5 rwm
# consoles
lxc.cgroup.devices.allow = c 5:1 rwm
lxc.cgroup.devices.allow = c 5:0 rwm
lxc.cgroup.devices.allow = c 4:0 rwm
lxc.cgroup.devices.allow = c 4:1 rwm
# /dev/{,u}random
lxc.cgroup.devices.allow = c 1:9 rwm
lxc.cgroup.devices.allow = c 1:8 rwm
lxc.cgroup.devices.allow = c 136:* rwm
lxc.cgroup.devices.allow = c 5:2 rwm
# rtc
lxc.cgroup.devices.allow = c 254:0 rwm
# mounts point
lxc.mount.entry=proc /lxc/PE1800-db0//rootfs/proc proc
nodev,noexec,nosuid 0 0
lxc.mount.entry=sysfs /lxc/PE1800-db0//rootfs/sys sysfs defaults 0 0
--
I still get exactly the same output when running /usr/local/pgsql/
bin/initdb -D /usr/local/pgsql/data:
sh: cannot create /dev/null: Permission denied
fgets failure: Success
The program postgres is needed by initdb but was not found in the
same directory as /usr/local/pgsql/bin/initdb.
Check your installation.
Anyone?
- Original Message -
From: Guido Jäkel g.jae...@dnb.de
To: Patrick Kevin McCaffrey p...@uwm.edu, lxc-users
lxc-users@lists.sourceforge.net
Sent: Tuesday, December 20, 2011 2:06:49 AM
Subject: Re: [Lxc-users] PostgreSQL - sh: cannot create /dev/null:
Permission denied - LXC Issue?
Dear Patrick,
As I understand /dev/null isn't writable in your container. That's
definitely a wrong
