Re: [lxc-users] not allowed to change kernel parameters inside container
My applications are very complex and involved many applications in the traditional sense. It is a nightmare to install them. My application runs on Centos but I prefer to use Ubuntu as LXC host. I found that rsynching a container over the WAN is the only perfect way to deploy. The issue that kills me is why I can change some kernel parameters, but not for example net.core.rmem_max = 67108864 net.core.wmem_max = 33554432 net.core.rmem_default = 31457280 net.core.wmem_default = 31457280 Any idea? On Mon, May 27, 2019 at 2:57 AM Jäkel, Guido wrote: > Dear Michael, > > > For me, the single point of using LXC is to be able to redeploy > a complex > > app from host to host in a few minutes. I use > one-host->one-Container. So > > what is the issue of giving all power to the containers? > > I don't understand yet, why you want to use Containers, LXC or Dockers at > all: You need to have full access to the host and it hardware at low level > and don't want to use any isolation or virtualization aspects at all. If > you just want to redeploy a complex setup within minutes, you may just need > to use a prepared backup of your hosts, or an layered setup with an > read-only image and an writeable layer for the changes. > > Guido > > ___ > lxc-users mailing list > lxc-users@lists.linuxcontainers.org > http://lists.linuxcontainers.org/listinfo/lxc-users > ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] not allowed to change kernel parameters inside container
I probably missed it, but which release are you using on the host? And what's the output of prlimit -p 1 ? On Mon, May 27, 2019, 1:52 PM Saint Michael wrote: > My applications are very complex and involved many applications in the > traditional sense. It is a nightmare to install them. > My application runs on Centos but I prefer to use Ubuntu as LXC host. > I found that rsynching a container over the WAN is the only perfect way to > deploy. > The issue that kills me is why I can change some kernel parameters, but > not for example > net.core.rmem_max = 67108864 > net.core.wmem_max = 33554432 > net.core.rmem_default = 31457280 > net.core.wmem_default = 31457280 > Any idea? > > > > > > On Mon, May 27, 2019 at 2:57 AM Jäkel, Guido wrote: > >> Dear Michael, >> >> > For me, the single point of using LXC is to be able to redeploy >> a complex >> > app from host to host in a few minutes. I use >> one-host->one-Container. So >> > what is the issue of giving all power to the containers? >> >> I don't understand yet, why you want to use Containers, LXC or Dockers at >> all: You need to have full access to the host and it hardware at low level >> and don't want to use any isolation or virtualization aspects at all. If >> you just want to redeploy a complex setup within minutes, you may just need >> to use a prepared backup of your hosts, or an layered setup with an >> read-only image and an writeable layer for the changes. >> >> Guido >> >> ___ >> lxc-users mailing list >> lxc-users@lists.linuxcontainers.org >> http://lists.linuxcontainers.org/listinfo/lxc-users >> > ___ > lxc-users mailing list > lxc-users@lists.linuxcontainers.org > http://lists.linuxcontainers.org/listinfo/lxc-users > ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] not allowed to change kernel parameters inside container
cat /etc/*release* DISTRIB_ID=Ubuntu DISTRIB_RELEASE=18.04 DISTRIB_CODENAME=bionic DISTRIB_DESCRIPTION="Ubuntu 18.04.2 LTS" NAME="Ubuntu" VERSION="18.04.2 LTS (Bionic Beaver)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 18.04.2 LTS" VERSION_ID="18.04" HOME_URL="https://www.ubuntu.com/"; SUPPORT_URL="https://help.ubuntu.com/"; BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"; PRIVACY_POLICY_URL=" https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"; VERSION_CODENAME=bionic UBUNTU_CODENAME=bionic prlimit -p 1 RESOURCE DESCRIPTION SOFT HARD UNITS AS address space limitunlimited unlimited bytes CORE max core file size 0 unlimited bytes CPUCPU time unlimited unlimited seconds DATA max data size unlimited unlimited bytes FSIZE max file size unlimited unlimited bytes LOCKS max number of file locks held unlimited unlimited locks MEMLOCKmax locked-in-memory address space 16777216 16777216 bytes MSGQUEUE max bytes in POSIX mqueues819200819200 bytes NICE max nice prio allowed to raise 0 0 NOFILE max number of open files 1048576 1048576 files NPROC max number of processes 503249503249 processes RSSmax resident set size unlimited unlimited bytes RTPRIO max real-time priority 0 0 RTTIME timeout for real-time tasksunlimited unlimited microsecs SIGPENDING max number of pending signals 503249503249 signals STACK max stack size 8388608 unlimited bytes On Mon, May 27, 2019 at 8:06 AM Kees Bos wrote: > I probably missed it, but which release are you using on the host? > > And what's the output of > prlimit -p 1 > ? > > On Mon, May 27, 2019, 1:52 PM Saint Michael wrote: > >> My applications are very complex and involved many applications in the >> traditional sense. It is a nightmare to install them. >> My application runs on Centos but I prefer to use Ubuntu as LXC host. >> I found that rsynching a container over the WAN is the only perfect way >> to deploy. >> The issue that kills me is why I can change some kernel parameters, but >> not for example >> net.core.rmem_max = 67108864 >> net.core.wmem_max = 33554432 >> net.core.rmem_default = 31457280 >> net.core.wmem_default = 31457280 >> Any idea? >> >> >> >> >> >> On Mon, May 27, 2019 at 2:57 AM Jäkel, Guido wrote: >> >>> Dear Michael, >>> >>> > For me, the single point of using LXC is to be able to >>> redeploy a complex >>> > app from host to host in a few minutes. I use >>> one-host->one-Container. So >>> > what is the issue of giving all power to the containers? >>> >>> I don't understand yet, why you want to use Containers, LXC or Dockers >>> at all: You need to have full access to the host and it hardware at low >>> level and don't want to use any isolation or virtualization aspects at all. >>> If you just want to redeploy a complex setup within minutes, you may just >>> need to use a prepared backup of your hosts, or an layered setup with an >>> read-only image and an writeable layer for the changes. >>> >>> Guido >>> >>> ___ >>> lxc-users mailing list >>> lxc-users@lists.linuxcontainers.org >>> http://lists.linuxcontainers.org/listinfo/lxc-users >>> >> ___ >> lxc-users mailing list >> lxc-users@lists.linuxcontainers.org >> http://lists.linuxcontainers.org/listinfo/lxc-users >> > ___ > lxc-users mailing list > lxc-users@lists.linuxcontainers.org > http://lists.linuxcontainers.org/listinfo/lxc-users > ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] not allowed to change kernel parameters inside container
Because * your Container is not started as a privileged one? * you let bind-mount /sys readonly? Guido >-Original Message- >From: lxc-users [mailto:lxc-users-boun...@lists.linuxcontainers.org] On Behalf >Of Saint Michael >Sent: Monday, May 27, 2019 1:49 PM >To: LXC users mailing-list >Subject: Re: [lxc-users] not allowed to change kernel parameters inside >container > >The issue that kills me is why I can change some kernel parameters, but not >for example >[...] > >Any idea? > ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] not allowed to change kernel parameters inside container
I thought I did start the containers as privileged:
lxc.include = /usr/share/lxc/config/ubuntu.common.conf
lxc.mount.auto=
lxc.mount.auto=proc:rw sys:rw cgroup:rw
lxc.apparmor.profile=unconfined
lxc.tty.max = 10
lxc.pty.max = 1024
lxc.cgroup.devices.allow = c 1:3 rwm
lxc.cgroup.devices.allow = c 1:5 rwm
lxc.cgroup.devices.allow = c 5:1 rwm
lxc.cgroup.devices.allow = c 5:0 rwm
lxc.cgroup.devices.allow = c 4:0 rwm
lxc.cgroup.devices.allow = c 4:1 rwm
lxc.cgroup.devices.allow = c 1:9 rwm
lxc.cgroup.devices.allow = c 1:8 rwm
lxc.cgroup.devices.allow = c 136:* rwm
lxc.cgroup.devices.allow = c 5:2 rwm
lxc.cgroup.devices.allow = c 254:0 rwm
lxc.cgroup.devices.allow = c 10:137 rwm # loop-control
lxc.cgroup.devices.allow = b 7:* rwm# loop*
lxc.cgroup.devices.allow = c 10:229 rwm #fuse
lxc.cgroup.devices.allow = c 10:200 rwm #docker
lxc.cgroup.devices.allow= a
lxc.cap.drop=
lxc.cgroup.devices.deny=
lxc.autodev= 1
lxc.hook.autodev = sh -c 'mknod ${LXC_ROOTFS_MOUNT}/dev/fuse c 10 229'
On Mon, May 27, 2019 at 9:03 AM Jäkel, Guido wrote:
> Because
>
> * your Container is not started as a privileged one?
> * you let bind-mount /sys readonly?
>
> Guido
>
> >-Original Message-
> >From: lxc-users [mailto:lxc-users-boun...@lists.linuxcontainers.org] On
> Behalf Of Saint Michael
> >Sent: Monday, May 27, 2019 1:49 PM
> >To: LXC users mailing-list
> >Subject: Re: [lxc-users] not allowed to change kernel parameters inside
> container
> >
> >The issue that kills me is why I can change some kernel parameters, but
> not for example
> >[...]
> >
> >Any idea?
> >
> ___
> lxc-users mailing list
> lxc-users@lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users
>
___
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] not allowed to change kernel parameters inside container
On Mon, May 27, 2019 at 8:11 PM Saint Michael wrote:
> I thought I did start the containers as privileged:
>
> lxc.include = /usr/share/lxc/config/ubuntu.common.conf
> lxc.mount.auto=
> lxc.mount.auto=proc:rw sys:rw cgroup:rw
> lxc.apparmor.profile=unconfined
> lxc.tty.max = 10
> lxc.pty.max = 1024
> lxc.cgroup.devices.allow = c 1:3 rwm
> lxc.cgroup.devices.allow = c 1:5 rwm
> lxc.cgroup.devices.allow = c 5:1 rwm
> lxc.cgroup.devices.allow = c 5:0 rwm
> lxc.cgroup.devices.allow = c 4:0 rwm
> lxc.cgroup.devices.allow = c 4:1 rwm
> lxc.cgroup.devices.allow = c 1:9 rwm
> lxc.cgroup.devices.allow = c 1:8 rwm
> lxc.cgroup.devices.allow = c 136:* rwm
> lxc.cgroup.devices.allow = c 5:2 rwm
> lxc.cgroup.devices.allow = c 254:0 rwm
> lxc.cgroup.devices.allow = c 10:137 rwm # loop-control
> lxc.cgroup.devices.allow = b 7:* rwm# loop*
> lxc.cgroup.devices.allow = c 10:229 rwm #fuse
> lxc.cgroup.devices.allow = c 10:200 rwm #docker
> lxc.cgroup.devices.allow= a
> lxc.cap.drop=
> lxc.cgroup.devices.deny=
> lxc.autodev= 1
> lxc.hook.autodev = sh -c 'mknod ${LXC_ROOTFS_MOUNT}/dev/fuse c 10 229'
>
Following Stephane's suggestion works on my test vm. You didn't do so, thus
it didn't work.
###
# Distribution configuration
lxc.include = /usr/share/lxc/config/common.conf
lxc.arch = x86_64
# Container specific configuration
lxc.rootfs.path = dir:/var/lib/lxc/c7-ul/rootfs
lxc.uts.name = c7-ul
lxc.net.0.type = none
lxc.mount.auto=
lxc.mount.auto=proc:rw sys:rw cgroup:rw
lxc.apparmor.profile=unconfined
###
###
c7-ul / # sysctl --system
* Applying /usr/lib/sysctl.d/00-system.conf ...
* Applying /usr/lib/sysctl.d/10-default-yama-scope.conf ...
kernel.yama.ptrace_scope = 0
* Applying /usr/lib/sysctl.d/50-default.conf ...
kernel.sysrq = 16
kernel.core_uses_pid = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.promote_secondaries = 1
net.ipv4.conf.all.promote_secondaries = 1
fs.protected_hardlinks = 1
fs.protected_symlinks = 1
* Applying /etc/sysctl.d/99-sysctl.conf ...
* Applying /etc/sysctl.d/net.conf ...
net.core.rmem_max = 67108864
net.core.wmem_max = 33554432
net.core.rmem_default = 31457280
net.core.wmem_default = 31457280
* Applying /etc/sysctl.conf ...
c7-ul / # cat /proc/sys/net/core/rmem_max
67108864
###
Of course as warned earlier, host networking brings along some quirks. For
instance:
- host and container can't have services run on the same port (e.g. if you
want sshd on both host and container, you need to change the listening port
for one of them)
- do not configure networking on the container (ONBOOT=no should be enough
on your container's eth confi)
- absolutely do not run "reboot", "init 6", or "poweroff" on the container.
At the very least, it will cause hosts's eth0 to go down. "reboot -f" on
the container should work nicely though.
--
Fajar
___
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] not allowed to change kernel parameters inside container
This
"host and container can't have services run on the same port (e.g. if you
want sshd on both host and container, you need to change the listening port
for one of them)"
is untrue.
each container in my case has a different IP address, the host has another
one, and I run SSHD inside each container just fine.
On Mon, May 27, 2019 at 10:00 PM Fajar A. Nugraha wrote:
> On Mon, May 27, 2019 at 8:11 PM Saint Michael wrote:
>
>> I thought I did start the containers as privileged:
>>
>> lxc.include = /usr/share/lxc/config/ubuntu.common.conf
>> lxc.mount.auto=
>> lxc.mount.auto=proc:rw sys:rw cgroup:rw
>> lxc.apparmor.profile=unconfined
>> lxc.tty.max = 10
>> lxc.pty.max = 1024
>> lxc.cgroup.devices.allow = c 1:3 rwm
>> lxc.cgroup.devices.allow = c 1:5 rwm
>> lxc.cgroup.devices.allow = c 5:1 rwm
>> lxc.cgroup.devices.allow = c 5:0 rwm
>> lxc.cgroup.devices.allow = c 4:0 rwm
>> lxc.cgroup.devices.allow = c 4:1 rwm
>> lxc.cgroup.devices.allow = c 1:9 rwm
>> lxc.cgroup.devices.allow = c 1:8 rwm
>> lxc.cgroup.devices.allow = c 136:* rwm
>> lxc.cgroup.devices.allow = c 5:2 rwm
>> lxc.cgroup.devices.allow = c 254:0 rwm
>> lxc.cgroup.devices.allow = c 10:137 rwm # loop-control
>> lxc.cgroup.devices.allow = b 7:* rwm# loop*
>> lxc.cgroup.devices.allow = c 10:229 rwm #fuse
>> lxc.cgroup.devices.allow = c 10:200 rwm #docker
>> lxc.cgroup.devices.allow= a
>> lxc.cap.drop=
>> lxc.cgroup.devices.deny=
>> lxc.autodev= 1
>> lxc.hook.autodev = sh -c 'mknod ${LXC_ROOTFS_MOUNT}/dev/fuse c 10 229'
>>
>
>
> Following Stephane's suggestion works on my test vm. You didn't do so,
> thus it didn't work.
>
> ###
> # Distribution configuration
> lxc.include = /usr/share/lxc/config/common.conf
> lxc.arch = x86_64
>
> # Container specific configuration
> lxc.rootfs.path = dir:/var/lib/lxc/c7-ul/rootfs
> lxc.uts.name = c7-ul
>
> lxc.net.0.type = none
> lxc.mount.auto=
> lxc.mount.auto=proc:rw sys:rw cgroup:rw
> lxc.apparmor.profile=unconfined
> ###
>
> ###
> c7-ul / # sysctl --system
> * Applying /usr/lib/sysctl.d/00-system.conf ...
> * Applying /usr/lib/sysctl.d/10-default-yama-scope.conf ...
> kernel.yama.ptrace_scope = 0
> * Applying /usr/lib/sysctl.d/50-default.conf ...
> kernel.sysrq = 16
> kernel.core_uses_pid = 1
> net.ipv4.conf.default.rp_filter = 1
> net.ipv4.conf.all.rp_filter = 1
> net.ipv4.conf.default.accept_source_route = 0
> net.ipv4.conf.all.accept_source_route = 0
> net.ipv4.conf.default.promote_secondaries = 1
> net.ipv4.conf.all.promote_secondaries = 1
> fs.protected_hardlinks = 1
> fs.protected_symlinks = 1
> * Applying /etc/sysctl.d/99-sysctl.conf ...
> * Applying /etc/sysctl.d/net.conf ...
> net.core.rmem_max = 67108864
> net.core.wmem_max = 33554432
> net.core.rmem_default = 31457280
> net.core.wmem_default = 31457280
> * Applying /etc/sysctl.conf ...
>
> c7-ul / # cat /proc/sys/net/core/rmem_max
> 67108864
> ###
>
>
> Of course as warned earlier, host networking brings along some quirks. For
> instance:
> - host and container can't have services run on the same port (e.g. if you
> want sshd on both host and container, you need to change the listening port
> for one of them)
> - do not configure networking on the container (ONBOOT=no should be enough
> on your container's eth confi)
> - absolutely do not run "reboot", "init 6", or "poweroff" on the
> container. At the very least, it will cause hosts's eth0 to go down.
> "reboot -f" on the container should work nicely though.
>
> --
> Fajar
> ___
> lxc-users mailing list
> lxc-users@lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users
>
___
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] not allowed to change kernel parameters inside container
On Tue, May 28, 2019 at 12:39 PM Saint Michael wrote: > This > "host and container can't have services run on the same port (e.g. if you > want sshd on both host and container, you need to change the listening port > for one of them)" > is untrue. > each container in my case has a different IP address, the host has another > one, and I run SSHD inside each container just fine. > > That is indeed the case for normal container setup. However you repeatedly said you want to be able to set net.core.rmem_max (and friends) from inside the container, which requires a not-normal setup. If you want to be able to do that from inside the container, you need the container to share host networking (lxc.net.0.type = none). It comes with its own consequences, thus the warnings above. If you want to keep having separate ip for the host and container, then you can't set net.core.rmem_max from inside the container. However, as someone point out earlier, you can simply setup passwordless ssh, and have container set it using ssh to the host during boot time. -- Fajar ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
