On 2018-05-31 7:00 AM, Jakob Hasse wrote:
Hello,
Make sure to cc meta-virtualization on questions like this, since
that is where you'll get more eyes that are running docker
all the time.
I ran into trouble running docker on our target.
1. When I want to start docker, I first have to re-mount cgroups:
root@target:~# cgroups-umount
root@target:~# cgroups-mount
Otherwise docker would produce an error:
ERRO[0002] Failed to built-in GetDriver graph btrfs /var/lib/docker
2. When I then start dockerd, it complains about a missing nat table:
root@target:~# dockerd
INFO[0000] libcontainerd: new containerd process, pid: 929
WARN[0000] containerd: low RLIMIT_NOFILE changing to max current=1024
max=4096
INFO[0001] [graphdriver] using prior storage driver: overlay2
INFO[0001] Graph migration to content-addressability took 0.00 seconds
WARN[0001] Your kernel does not support cgroup memory limit
WARN[0001] Unable to find cpu cgroup in mounts
WARN[0001] Unable to find blkio cgroup in mounts
WARN[0001] Unable to find cpuset cgroup in mounts
WARN[0001] mountpoint for pids not found
INFO[0001] Loading containers: start.
WARN[0001] Running modprobe nf_nat failed with message: `modprobe:
WARNING: Module nf_nat not found in directory
/lib/modules/4.9.81-dey+g2c6ae4c`, error: exit status 1
WARN[0001] Running modprobe xt_conntrack failed with message: `modprobe:
WARNING: Module xt_conntrack not found in directory
/lib/modules/4.9.81-dey+g2c6ae4c`, error: exit status 1
Error starting daemon: Error initializing network controller: error
obtaining controller instance: failed to create NAT chain: iptables
failed: iptables --wait -t nat -N DOCKER: iptables v1.6.1: can't
initialize iptables table `nat': Table does not exist (do you need to
insmod?)
Perhaps iptables or your kernel needs to be upgraded.
(exit status 3)
Our configuration is as suggested here:
https://wiki.yoctoproject.org/wiki/TipsAndTricks/DockerOnImage, except
I've never seen that wiki page before (or at least I don't remember
seeing it), so I can't confirm or deny the validity of the content :)
that I don't include the system systemd stuff (it lets my build fail)
If systemd is breaking your build, make sure to log a bugzilla against
oe-core
and connman (using NetworkManager).
Furthermore, I added the following lines to the kernel bbappend file:
# remove old defconfig
SRC_URI_remove = " defconfig"
# replace with new defconfig
SRC_URI_append = " file://defconfig"
KERNEL_FEATURES_append = " features/cgroups/cgroups.scc "
I also added a lot of configurations manually to the defconfig (mostly
via menuconfig) to enable NAT:
CONFIG_CGROUP_DEVICE=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_NETFILTER=y
CONFIG_NF_CONNTRACK=y
CONFIG_NF_TABLES=y
CONFIG_NF_NAT=y
CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=y
CONFIG_NETFILTER_XT_MATCH_COMMENT=y
CONFIG_NETFILTER_XT_MATCH_HL=y
CONFIG_NETFILTER_XT_MATCH_IPRANGE=y
CONFIG_NETFILTER_XT_MATCH_LIMIT=y
CONFIG_NETFILTER_XT_MATCH_MULTIPORT=y
CONFIG_NETFILTER_XT_MATCH_RECENT=y
CONFIG_IP_VS=y
CONFIG_NF_TABLES_IPV4=y
CONFIG_IP_NF_IPTABLES=y
CONFIG_IP_NF_NAT=y
CONFIG_IP_NF_FILTER=y
CONFIG_IP_NF_MANGLE=y
CONFIG_IP6_NF_IPTABLES=y
CONFIG_IP6_NF_FILTER=y
CONFIG_IP6_NF_MANGLE=y
CONFIG_BTRFS_FS=y
CONFIG_OVERLAY_FS=y
Apart from that, I added virtualization and aufs as DISTRO_FEATURE in
local.conf and also enabled it in menuconfig.
But I still keep getting the above mentioned iptables error when trying
to start docker. All this hassle makes me suspicious, especially as I'm
quite sure that I once had docker running already with an image on our
target and it wasn't that hard. So maybe it's just a misconfiguration
and I need to add something in local.conf or the kernel recipe? Is
systemd necessary? Or am I missing some life-or-death kernel
configuration? It would also be nice if I could avoid the cgroup
re-mounting before starting docker.
What release branch are you using ?
I'm running docker from meta-virt every day, as are many others,
but you have several differences in your configuration.
- most use systemd as the init manager, I know that I do. That
is going to impact how cgroups is set up on your 'host' image.
You shouldn't need to touch cgroups at all if systemd is used,
since it is correct out of the box.
- You are using a different kernel and kernel configuration.
linux-yocto + the configuration fragments in the layer are what
is routinely tested. Are you using linux-yocto, or something
different ? If it is different, all you can do is run the various
checks to make sure that the docker prereqs are in place.
The errors you see in dockerd tells me that the options you are
turning on, are not making it into the final kernel that is
running on target.
Cheers,
Bruce
Thanks for every answer!
All the Best,
Jakob
--
_______________________________________________
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
