Hi Josch. Thanks for replying. Notes inline
Johannes Schauer Marin Rodrigues <jo...@debian.org> writes:
> Quoting Dima Kogan (2023-03-08 00:46:18)
>> Package: mmdebstrap
>> Version: 1.3.1-2
>
> where is this version from? Debian stable has 0.7.5 and testing is at
> 1.3.3.
I run sid, manually updating periodically. I guess I last updated at
1.3.1-2. The breakage doesn't appear to be version-dependent, although I
see different behaviors. I tried it on 3 machines:
- My workstation (amd64, sid, mmdebstrap=1.3.3-6.1). Works fine. It has
this:
dima@fatty:~$ id
uid=60017(dima) gid=60017(dima)
groups=60017(dima),4(adm),20(dialout),24(cdrom),25(floppy),29(audio),30(dip),33(www-data),44(video),46(plugdev),108(netdev),110(lpadmin),112(xxxxx),113(scanner),119(bluetooth),131(sbuild),1002(yumsters),1003(mock),1004(pub)
dima@fatty:~$ cat /etc/subuid
systemd-timesync:100000:65536
systemd-network:165536:65536
systemd-resolve:231072:65536
AAAAAAAA:296608:65536
messagebus:362144:65536
avahi:427680:65536
uuidd:493216:65536
Debian-exim:558752:65536
statd:624288:65536
avahi-autoipd:689824:65536
colord:755360:65536
dnsmasq:820896:65536
geoclue:886432:65536
rtkit:951968:65536
pulse:1017504:65536
sshd:1083040:65536
sbuild:1148576:65536
saned:1214112:65536
usbmux:1279648:65536
hplip:1345184:65536
Debian-gdm:1410720:65536
dima:1476256:65536
_apt:1541792:65536
BBBBBBB:1607328:65536
pub:1672864:65536
bitlbee:1738400:65536
testman:1803936:65536
CCCCC:1869472:65536
mysql:1935008:65536
tftp:2000544:65536
DDDDDD:2066080:65536
EEEEEE:2131616:65536
FFFFFFFF:2197152:65536
GGGGGGGGG:2262688:65536
HHHHHHHH:2328224:65536
IIIII:2393760:65536
JJJJJJJ:2459296:65536
KKKKKK:2524832:65536
LLLLL:2590368:65536
MMMMM:2655904:65536
NNNNNNN:2721440:65536
OOOOOOO:2786976:65536
PPPPPP:2852512:65536
QQQQQ:2918048:65536
RRRR:2983584:65536
SSSS:3049120:65536
TTTTTTTT:3114656:65536
UUUUU:3180192:65536
VVVV:3245728:65536
WWW:3311264:65536
XXXXXXXX:3376800:65536
- My laptop (amd64, sid, mmdebstrap=1.3.3-6.1; same as the workstation).
Does NOT work fine:
dima@shorty:~$ mmdebstrap bookworm /tmp/tst.tar.gz
http://deb.debian.org/debian
I: automatically chosen mode: unshare
I: chroot architecture amd64 is equal to the host's architecture
I: finding correct signed-by value...
done
I: automatically chosen format: tar
I: using /tmp/mmdebstrap.VLwVKQsx19 as tempdir
W: no entry in /etc/subuid for dima
E: invalid idmap
The user id situation is different:
dima@shorty:~$ id
uid=1000(dima) gid=1000(dima)
groups=1000(dima),4(adm),5(tty),6(disk),7(lp),12(man),20(dialout),24(cdrom),25(floppy),27(sudo),29(audio),44(video),50(staff),100(users),107(netdev),111(nvram),112(fuse),120(stapdev),125(postgres),129(davfs2),132(motion),137(systemd-journal),140(sbuild),145(input)
dima@shorty:~$ cat /etc/subuid
bitlbee:100000:65536
stunnel4:165536:65536
sbuild:231072:65536
iodine:296608:65536
systemd-timesync:362144:65536
systemd-network:427680:65536
systemd-resolve:493216:65536
pulse:624288:65536
AAA:689824:65536
debian-tor:755360:65536
_apt:820896:65536
pub:886432:65536
BBB:558752:65536
- The server (amd64, Ubuntu 20.04, mmdebstrap=0.4.1-6). Also does not
work fine:
kogan@cadredev:~$ mmdebstrap bookworm /tmp/tst.tar.gz
http://deb.debian.org/debian
I: automatically chosen mode: unshare
I: chroot architecture amd64 is equal to the host's architecture
I: using /tmp/mmdebstrap.42KXYMZJtF as tempdir
newuidmap: uid range [1-2) -> [100000-100001) not allowed
E: newuidmap 2086656 0 60017 1 1 100000 1 failed:
E: child had a non-zero exit status: 1
E: chown failed
kogan@cadredev:~$ id
uid=60017(kogan) gid=1000(AAA) groups=1000(AAA),10773(perf),22373(BBB)
kogan@cadredev:~$ cat /etc/subuid
ssa:100000:65536
This is clearly running a much older mmdebstrap. It's also a more
complex beast regarding users, since it's a shared server with LDAP.
This is the machine I was complaining about originally; I had
forgotten that it's an old distro when making the report; sorry. But
it looks like similar failures are happening on other boxes too.
In all cases the "unshare" mode was selected. I'm guessing I need to add
an entry for my user to /etc/subuid? How is this managed? I've never
heard of this file before today, and I've certainly never added anything
to it. Why am I listed in it on one machine, but not on the other two?
>> but I don't understand the
>> problem, and would like ask here. For a little while now I've been using
>> mmdebstrap to create bookworm tarballs. This works very nicely. As a
>> non-root user I would do this:
>>
>> mmdebstrap \
>> bookworm \
>> image.tar.gz \
>> http://deb.debian.org/debian
>
> Note, that by giving the mirror address manually, the resulting chroot will
> *not* include stable update and security mirror entries in
> /etc/apt/sources.list. Do you really not want those? With your invocation you
> imitate the kind of chroot that debootstrap would produce which is missing out
> on stable updates and security fixes (see #543819 and #762222). This is due to
> debootstrap being limited to a single mirror but since mmdebstrap can handle
> multiple mirrors, adding stable updates and security mirrors is the default if
> you create a stable chroot.
The above is a simplified command to demo the bug. Usually I give it
multiple APT repos and other stuff. The lack of backport and security
updates doesn't matter in this case: this is hopefully going to an
exciting faraway place with no internet access. I'll make an
announcement once it's a done deal.
>> Today I tried this on a different machine. It's also running Debian, but
>> something is different about it, because this happens:
>>
>> mmdebstrap \
>> bookworm \
>> image.tar.gz \
>> http://deb.debian.org/debian
>>
>> I: automatically chosen mode: unshare
>> I: chroot architecture amd64 is equal to the host's architecture
>> I: finding correct signed-by value...
>> done
>> I: automatically chosen format: tar
>> I: using /tmp/mmdebstrap.hu5TsS_2_C as tempdir
>> newuidmap: write to uid_map failed: Operation not permitted
>> E: newuidmap 1474581 0 60017 1 1 1476256 1 failed:
I was confused. This is the Ubuntu 20.04 box with LDAP and other stuff.
> This looks very odd. What does it say in your /etc/subuid?
See above.
>> E: child had a non-zero exit status: 1
>> E: chown failed
>>
>> I'm reading the "newuidmap" manpage, but the issue isn't clear to me. In
>> an attempt to debug, I did this on the working machine:
>>
>> strace -f -o /tmp/stlog mmdebstrap ....
>>
>> Doing that makes it fail with that error! So adding strace to a working
>> mmdebstrap invocation causes this error too.
>
> This is not a problem of mmdebstrap but a problem with unsharing the user
> namespace and the involved permissions. Try running this on both machines with
> and without "strace -f" in front (the -f is very important):
>
> unshare --user --map-auto whoami
>
> It should print "nobody" without "strace -f" in front.
Sorta.
On the workstation (sid; mmdebstrap works):
dima@fatty:~$ unshare --user --map-auto whoami
nobody
dima@fatty:~$ strace -f -o /dev/null unshare --user --map-auto whoami
newuidmap: write to uid_map failed: Operation not permitted
On the laptop (sid; mmdebstrap doesn't work):
dima@shorty:~$ unshare --user --map-auto whoami
unshare: no line matching user "dima" in /etc/subuid: Success
dima@shorty:~$ strace -f -o /dev/null unshare --user --map-auto whoami
unshare: no line matching user "dima" in /etc/subuid: Success
On the old Ubuntu LDAP-enabled server it doesn't work at all:
unshare: unrecognized option '--map-auto'
This suggests once again that I should add an entry to /etc/subuid for
my user. Yes?
>> If I just run the failing "newuidmap" command all by itself in the
>> shell, it consistently produces that error. This makes me think that
>> when mmdebstrap is working for me, it's somehow not actually running
>> newuidmap. I don't know why.
>>
>> In all cases I see this:
>>
>> I: automatically chosen mode: unshare
>>
>> The mmdebstrap manpage talks about this option, but it's still not clear
>> to me. Can you please comment? Is the above supposed to work? If so, any
>> idea why it would fail on some machines and not others? Does it make
>> sense that strace breaks it?
>
> I have never tried or seen this breaking with strace (you are the first). But
> it makes sense that strace cannot do this because the process becomes owned by
> another user so of course the user running strace should not have permissions
> to continue observing it unless you are root (which your outer user is not).
That makes sense, I suppose. I don't actually care if it works in strace
or not; I was just trying to use strace to figure out why it wasn't
working right. Helmut's notes about strace are good to know too. Thanks
for looking into it.
> Now the problem is twofold:
>
> 1) improve the check that mmdebstrap does to make sure that unshare
> mode works before automatically choosing it
>
> 2) figure out why the newuidmap call fails on your machine
>
> Figuring out 2) will produce a better solution for 1).
>
> As explained above, you can run "unshare --user --map-auto" to test this issue
> without mmdebstrap in the loop. The unshare command will also run newuidmap
> and
> should fail in the same way as when newuidmap is called by mmdebstrap.
>
> If you don't mind installing or already have lxc installed, there is a bit
> more
> sophisticated lxc-usernsexec command in the mmdebstrap man page that you could
> also try because it also calls newuidmap and should fail in the same way.
Tell me what specific experiments to run, and I'll run them.
