Bug#930684: pbuilder: creation of build env fails when run inside Docker container

2019-06-19 Thread Thorsten Glaser
Tobias Junghans dixit:

>> But with {cow|p}builder --login --save-after-login you can
>> upgrade the base to buster inside pbuilder.
>
>It's not about the pbuilder environment itself but the Docker container used 
>for invoking pbuilder/debootstrap. Using a stretch-based Docker container and 
>debootstrapping Buster works fine.

Ah okay. Yes, that’s supported, a difference of only one version,
as long as the host has (at least) the stretch kernel.

As for the other things, the debootstrap people might look at it.

bye,
//mirabilos
-- 
15:41⎜ Somebody write a testsuite for helloworld :-)



Bug#930684: pbuilder: creation of build env fails when run inside Docker container

2019-06-19 Thread Tobias Junghans
Am Dienstag, 18. Juni 2019, 21:10:53 CEST schrieb Thorsten Glaser:
> Tobias Junghans dixit:
> >I tried to upgrade my Docker-based pbuilder containers from stretch to
> 
> Erm… why do you use chroots inside of chroots? That’s… tricky.

Simple because we use Gitlab Runners with the builtin Docker Executor 
(https://docs.gitlab.com/runner/executors/docker.html) for running all kinds 
of jobs in a generic manner. Depending on the project individual pre-built 
Docker images (specified in the CI config) providing the desired build tools 
and toolchains are used. The jobs for building Debian packages use a Docker 
container with Debian and pbuilder installed.

This used to work for years but for Buster-based containers (i.e. pbuilder and 
debootstrap from Buster) it doesn't any longer.


> >mount: failed to read mtab: No such file or directory
> 
> This might be a container issue.

The mounts are fine before running pbuilder/debootstrap. Afterwards proc is 
not mounted even in the Docker container. The state of the container and its 
mounts shouldn't change when running debootstrap. I also saw Docker/container-
related changes between 1.0.89 (stretch) and buster (1.0.114) which likely 
cause the misbehaviour:

https://salsa.debian.org/installer-team/debootstrap/commit/
5a0f16664066b24c42c074643a4ca178890d7af7
https://salsa.debian.org/installer-team/debootstrap/commit/
0962af1527a1ba0e996a0b442b159b4dbf164988
https://salsa.debian.org/installer-team/debootstrap/commit/
1e7549c57c0f15816c89c4f243051785ca383be9

> >mount: /proc: mount(2) system call failed: Too many levels of symbolic
> >links.
> Check if /proc outside of pbuilder but inside the container is right.

/proc in the container is fine, see first output of "cat /proc/mounts
" in by original bug report.
 
> But with {cow|p}builder --login --save-after-login you can
> upgrade the base to buster inside pbuilder.

It's not about the pbuilder environment itself but the Docker container used 
for invoking pbuilder/debootstrap. Using a stretch-based Docker container and 
debootstrapping Buster works fine.

Thanks and best regards

Tobias



Bug#930684: pbuilder: creation of build env fails when run inside Docker container

2019-06-18 Thread Thorsten Glaser
Tobias Junghans dixit:

>I tried to upgrade my Docker-based pbuilder containers from stretch to

Erm… why do you use chroots inside of chroots? That’s… tricky.

>mount: failed to read mtab: No such file or directory

This might be a container issue.

>mount: /proc: mount(2) system call failed: Too many levels of symbolic links.

Check if /proc outside of pbuilder but inside the container is right.

There also might be a /dev/shm vs. /run/shm issue. I recently had a
failure with these (one’s a mountpoint, the other a symbolic link to
it, either way works but it’s got to be consistent inside and out‐
side of the chroot) with schroot.

>the stretch-based pbuilder container and use it to build packages for buster.

Don’t.

But with {cow|p}builder --login --save-after-login you can
upgrade the base to buster inside pbuilder.

bye,
//mirabilos
-- 
FWIW, I'm quite impressed with mksh interactively. I thought it was much
*much* more bare bones. But it turns out it beats the living hell out of
ksh93 in that respect. I'd even consider it for my daily use if I hadn't
wasted half my life on my zsh setup. :-) -- Frank Terbeck in #!/bin/mksh



Processed: Re: Bug#930684: pbuilder: creation of build env fails when run inside Docker container

2019-06-18 Thread Debian Bug Tracking System
Processing control commands:

> reassign -1 debootstrap 1.0.114
Bug #930684 [pbuilder] pbuilder: creation of build env fails when run inside 
Docker container
Bug reassigned from package 'pbuilder' to 'debootstrap'.
No longer marked as found in versions pbuilder/0.230.4.
Ignoring request to alter fixed versions of bug #930684 to the same values 
previously set
Bug #930684 [debootstrap] pbuilder: creation of build env fails when run inside 
Docker container
Marked as found in versions debootstrap/1.0.114.
> tag -1 - upstream
Bug #930684 [debootstrap] pbuilder: creation of build env fails when run inside 
Docker container
Removed tag(s) upstream.

-- 
930684: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=930684
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems



Re: Bug#930684: pbuilder: creation of build env fails when run inside Docker container

2019-06-18 Thread Mattia Rizzolo
Control: reassign -1 debootstrap 1.0.114
Control: tag -1 - upstream

On Tue, Jun 18, 2019 at 01:49:27PM +, Tobias Junghans wrote:
> I tried to upgrade my Docker-based pbuilder containers from stretch to
> buster. However it appears that pbuilder and/or debootstrap do not work
> properly inside Docker containers any longer due to issues with mounting
> special filesystems such as proc and devpts.

From your log it seems like it's debootstrap that is actually failing.

I don't use docker and I don't really want to figure out how to try it,
so I'll just bounce the ball to the debootstrap maintainers :)

> The issue can be reproduced easily in a Debian Buster based container:
> 
> # docker run --privileged -it debian:buster /bin/bash
> 
> root@d81f634fe4a0:/# cat /proc/mounts
> overlay / overlay 
> rw,relatime,lowerdir=/var/lib/docker/overlay2/l/TPOD4JNRBNCTMXNHYCY5XVRBQ3:/var/lib/docker/overlay2/l/TSD62UVCIJQ2LJ4XTUHKTVEK77,upperdir=/var/lib/docker/overlay2/aa29cac2d0ebecfb12fdd71a9952845140052615f2bd746c4336daa8d7a4d533/diff,workdir=/var/lib/docker/overlay2/aa29cac2d0ebecfb12fdd71a9952845140052615f2bd746c4336daa8d7a4d533/work
>  0 0
> proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
> tmpfs /dev tmpfs rw,nosuid,size=65536k,mode=755 0 0
> devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666 
> 0 0
> sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
> tmpfs /sys/fs/cgroup tmpfs rw,nosuid,nodev,noexec,relatime,mode=755 0 0
> cgroup /sys/fs/cgroup/systemd cgroup 
> rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd
>  0 0
> cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 
> 0 0
> cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 
> 0 0
> cgroup /sys/fs/cgroup/net_cls,net_prio cgroup 
> rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0
> cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0
> cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0
> cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
> cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0
> cgroup /sys/fs/cgroup/cpu,cpuacct cgroup 
> rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0
> cgroup /sys/fs/cgroup/perf_event cgroup 
> rw,nosuid,nodev,noexec,relatime,perf_event 0 0
> mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0
> /dev/sdb /etc/resolv.conf ext4 rw,noatime,nodiratime,commit=300,data=ordered 
> 0 0
> /dev/sdb /etc/hostname ext4 rw,noatime,nodiratime,commit=300,data=ordered 0 0
> /dev/sdb /etc/hosts ext4 rw,noatime,nodiratime,commit=300,data=ordered 0 0
> shm /dev/shm tmpfs rw,nosuid,nodev,noexec,relatime,size=65536k 0 0
> devpts /dev/console devpts 
> rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666 0 0
> 
> root@d81f634fe4a0:/# apt-get update && apt-get -y --no-install-recommends 
> install pbuilder
> 
> [...]
> 
> root@d81f634fe4a0:/# pbuilder create --distribution buster
> W: /root/.pbuilderrc does not exist
> W: cgroups are not available on the host, not using them.
> I: Distribution is buster.
> I: Current time: Tue Jun 18 13:27:34 UTC 2019
> I: pbuilder-time-stamp: 1560864454
> I: Building the build environment
> I: running debootstrap
> /usr/sbin/debootstrap
> I: Retrieving InRelease 
> I: Checking Release signature
> I: Valid Release signature (key id 16E90B3FDF65EDE3AA7F323C04EE7237B7D453EC)
> I: Retrieving Packages 
> I: Validating Packages 
> I: Resolving dependencies of required packages...
> I: Resolving dependencies of base packages...
> I: Checking component main on http://deb.debian.org/debian...
> I: Retrieving libacl1 2.2.53-4
> I: Validating libacl1 2.2.53-4
> 
> [...]
> 
> W: Failure trying to run: chroot "/var/cache/pbuilder/build/489" mount -t 
> proc proc /proc
> W: See /var/cache/pbuilder/build/489/debootstrap/debootstrap.log for details
> 
> [...]
> 
> Setting up aptitude (0.8.11-7) ...
> update-alternatives: using /usr/bin/aptitude-curses to provide 
> /usr/bin/aptitude (aptitude) in auto mode
> Processing triggers for libc-bin (2.28-10) ...
> I: Copying back the cached apt archive contents
> I: new cache content 'aptitude-common_0.8.11-7_all.deb' added
> I: new cache content 'libboost-iostreams1.67.0_1.67.0-13_amd64.deb' added
> I: new cache content 'aptitude_0.8.11-7_amd64.deb' added
> I: new cache content 'libsqlite3-0_3.27.2-3_amd64.deb' added
> I: new cache content 'libxapian30_1.4.11-1_amd64.deb' added
> I: new cache content 'libcwidget3v5_0.5.17-11_amd64.deb' added
> I: new cache content 'libboost-system1.67.0_1.67.0-13_amd64.deb' added
> I: new cache content 'libsigc++-2.0-0v5_2.10.1-2_amd64.deb' added
> mount: failed to read mtab: No such file or directory
> mount: failed to read mtab: No such file or directory
> I: unmounting dev/pts filesystem
> I: unmounting dev/shm filesystem
> I: unmounting proc filesystem
> I: unmounting sys filesystem
> I: creating