Re: Debian and our frenemies of containers and userland repos
On Mon, Oct 21 2019, Enrico Weigelt wrote: > On 05.10.19 03:31, Paul Wise wrote: >> On Fri, Oct 4, 2019 at 10:49 PM Enrico Weigelt wrote: >>> On 24.07.19 08:17, Marc Haber wrote: >>> Do we have a build technology that uses containers instead of chroots yet? >>> >>> Something like docker-buildpackage ? >> >> AFAICT, docker-buildpackage doesn't exist > > I'm pretty sure it does exist, since I wrote it :p > > https://github.com/metux/docker-buildpackage I should also mention https://packages.debian.org/buster/debocker - though I haven't used it. John > > > --mtx
Re: Debian and our frenemies of containers and userland repos
On Mon, 2019-10-21 at 18:58 +0200, Enrico Weigelt wrote: > I'm pretty sure it does exist, since I wrote it :p > > https://github.com/metux/docker-buildpackage I couldn't find it in Debian so I incorrectly assumed, sorry. In case you want to get it into Debian: https://mentors.debian.net/intro-maintainers -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: Debian and our frenemies of containers and userland repos
On 05.10.19 19:03, Bernd Zeimetz wrote: > For that, developers also need or want the > latest shiniest software - which is something a distribution can't provide. It can, but that needs different workflows and higher grade of automation. (and of course wouldn't be so well tested) Actually, I for python world I already did something: fully automatic import and debianziation for pypi-packages. Yet experimental and part of another tool (which I'm using for building customer specific backport- and extra repos): https://github.com/metux/deb-pkg/tree/wip/pypi > I'm wondering if there is something Debian can do to be even more > successful in the container world. You could use dck-buildpackage --create-baseimage to do that. Feel free to create some target configs, and I'll be happy to add them. --mtx -- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287
Re: Debian and our frenemies of containers and userland repos
On 07.10.19 13:17, Shengjing Zhu wrote:
> Why not have a repository for it, like dockerhub. So this becomes
> "pull latest build env", which saves lots of time("re-bootstrap" is
> still slow nowadays).
No idea how sbuild works these days (turned away from it aeons ago, as
I've found it too complicated to set up), but dck-buildpackage can do
both. It can try to pull an existing image for the given target, but
will bootstrap it when there isn't one.
IMHO, the best idea is treating images as nothing as a cache, and
having the build machinery bootstrap automatically when needed.
One thing on my 2do list for dck-buildpackage is keeping cache images
for dependency sets (eg. if the same package is rebuilt many times,
during development) - installing dependencies can eat up a lot of time.
(for now, this can be achieved manually, by configuring a target with
dependencies already installed - but I don't like manual things :o)
BTW: one important point w/ dck-buildpackage for me was being able
to specifiy what's in the image. I really prefer to have it really
minimal.
--mtx
--
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
i...@metux.net -- +49-151-27565287
Re: Debian and our frenemies of containers and userland repos
On 05.10.19 18:25, Bernd Zeimetz wrote: Hi, > Having something that works with git-buildpackage would be really nice, :) > though. Even better if it would allow to use the k8s API to build things... Patches are always welcomed :) There're some problems to be solved for remote hosts (IMHO, k8s only on local node doesn't make so much sense ;-)): dck-buildpackage currently mounts some host directories (eg. local apt repo and reflink'ed copy of the source tree) into the container. While one could put docker nodes into a shared filesystem, that probably wouldn't be so nice w/ k8s ... --mtx -- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287
Re: Debian and our frenemies of containers and userland repos
On 05.10.19 03:31, Paul Wise wrote: > On Fri, Oct 4, 2019 at 10:49 PM Enrico Weigelt wrote: >> On 24.07.19 08:17, Marc Haber wrote: >> >>> Do we have a build technology that uses containers instead of chroots >>> yet? >> >> Something like docker-buildpackage ? > > AFAICT, docker-buildpackage doesn't exist I'm pretty sure it does exist, since I wrote it :p https://github.com/metux/docker-buildpackage --mtx -- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287
InstallBootstrap (was: Re: Debian and our frenemies of containers and userland repos)
Hi, Quoting Jonas Smedegaard (2019-10-08 10:30:45) > Quoting Paul Wise (2019-10-08 02:35:14) > > On Mon, 2019-10-07 at 11:29 +0100, Simon McVittie wrote: > > > I think "re-bootstrap, don't upgrade" is an equally good principle for > > > autopkgtest and sbuild? Both will be equally susceptible to accumulating > > > cruft during upgrades that wouldn't have been there in a fresh > > > debootstrap, > > > which is undesired if you want the invariant that you are > > > (building|testing) > > > in "today's" minimal environment. > > debootstrap uses a fair bit more time and resources than apt upgrade, > > so I think that both are needed. > This seems a good place to mention mmdebstrap as a speedier alternative to > deboostrap. while I certainly welcome use of my software, let me also use this opportunity for a word of caution. One of the core reasons why I created mmdebstrap was to create a proof-of-concept for the idea that it should at some point be possible to create a chroot of Debian (or a derivative) without all the magic instructions which currently live in debootstrap and cdebootstrap. Instead, all necessary information should live in the packages themselves using the same component-centric way of thinking we apply to other engineering decisions in the distribution. An extended rationale can be found on the following wiki page and in the bugs it links to in the first section: https://wiki.debian.org/Teams/Dpkg/Spec/InstallBootstrap But as of today, the problem is not solved. So that mmdebstrap works today (and has been working in stable, testing and unstable for the past year) is unfortunately more like happenstance than due to some ingenuity on the part of mmdebstrap. Thanks! cheers, josch signature.asc Description: signature
Re: Debian and our frenemies of containers and userland repos
Quoting Paul Wise (2019-10-08 02:35:14) > On Mon, 2019-10-07 at 11:29 +0100, Simon McVittie wrote: > > > I think "re-bootstrap, don't upgrade" is an equally good principle for > > autopkgtest and sbuild? Both will be equally susceptible to accumulating > > cruft during upgrades that wouldn't have been there in a fresh debootstrap, > > which is undesired if you want the invariant that you are (building|testing) > > in "today's" minimal environment. > > debootstrap uses a fair bit more time and resources than apt upgrade, > so I think that both are needed. This seems a good place to mention mmdebstrap as a speedier alternative to deboostrap. - Jonas -- * Jonas Smedegaard - idealist & Internet-arkitekt * Tlf.: +45 40843136 Website: http://dr.jones.dk/ [x] quote me freely [ ] ask before reusing [ ] keep private signature.asc Description: signature
"adt-virt-*" abstraction (was Re: Debian and our frenemies of containers and userland repos)
fwiw, src:piuparts also includes a (now) derived implementation of "adt-virt-*"... -- cheers, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C signature.asc Description: PGP signature
Re: Debian and our frenemies of containers and userland repos
On Mon, 2019-10-07 at 11:29 +0100, Simon McVittie wrote: > I think "re-bootstrap, don't upgrade" is an equally good principle for > autopkgtest and sbuild? Both will be equally susceptible to accumulating > cruft during upgrades that wouldn't have been there in a fresh debootstrap, > which is undesired if you want the invariant that you are (building|testing) > in "today's" minimal environment. debootstrap uses a fair bit more time and resources than apt upgrade, so I think that both are needed. I don't want to be rebuilding sbuild/pbuilder chroots before each package build, but an apt update and upgrade before each build starts installing build-dependencies is reasonable. At work we upgrade build containers interactively if needed, upgrade them automatically daily and rebuild them from scratch weekly, this feels like a reasonable compromise to me. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Re: Debian and our frenemies of containers and userland repos
Paul Wise writes ("Re: Debian and our frenemies of containers and userland
repos"):
> On Sun, Oct 6, 2019 at 7:23 PM Simon McVittie wrote:
> > FYI, this is because autopkgtest has an abstraction for multiple
> > container/virtualization mechanisms (lxc, lxd, qemu, schroot)
>
> It seems like this abstraction should be split out of the autopkgtest
> source and then depended on by autopkgtest.
I agree. I was the original designer of this interface and the
original author of autopkgtest. I put the virtualisation abstractions
in src:autopkgtest just because it was convenient, not for any
principled reason. I chose the name pattern "adt-virt-*". I put
"adt" in it to avoid it being too much of a namespace grab, not
because this interface was intended only for the use of autopktest.
I think the current autopkgtest maintainer is not so keen on this
abstraction. I would love it if we could split it out from the
autopkgtest package and I would volunteer to maintain it.
Martin, would such a move have your support ?
> Do any other such abstraction layers exist?
The closest I was aware of at the time was libvirt, which is not
really useable in the same way. I think the interface I designed and
which has subsequently been significantly improved, is a good one, and
we should continue to develop and enhance it.
Thanks,
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Debian and our frenemies of containers and userland repos
On 2019-10-07 13:43, Johannes Schauer wrote:
Quoting Philipp Kern (2019-10-07 13:21:36)
On 10/7/2019 1:17 PM, Shengjing Zhu wrote:
> On Mon, Oct 7, 2019 at 6:29 PM Simon McVittie wrote:
>> On Mon, 07 Oct 2019 at 07:22:53 +0200, Johannes Schauer wrote:
>>> Specifically, currently autopkgtest is limited to providing a read-only
layer
>>> for certain backends and its upstream has no intention of widening the
scope of
>>> the software [1]. This means that to upgrade an autopkgtest backend, the
user
>>> has to apply backend-specific actions
>> I think "re-bootstrap, don't upgrade" is an equally good principle
> Why not have a repository for it, like dockerhub. So this becomes
> "pull latest build env", which saves lots of time("re-bootstrap" is still
> slow nowadays).
In that case it'd probably be better to make bootstrapping faster
rather
than trusting random binaries on the internet. (Unless we grow an
"assemble an image from debs" service on, say, ftp-master.)
creating a working sbuild chroot takes 10 seconds on my system:
$ time mmdebstrap --variant=essential unstable debian-unstable.tar
[...]
mmdebstrap [...] 8.35s user 1.73s system 99% cpu 10.166 total
Do we need to spend engineering effort to become faster than that?
I suppose that also depends on deb caching/pipe bandwidth? But yes, I
find that totally fine.
Downloading "random binary from the internet" is less of a problem if
we can
create images which are bit-by-bit identical to checksums that we can
verify
through a trusted service. This is also already provided by mmdebstrap:
$ SOURCE_DATE_EPOCH=1570448177 mmdebstrap --variant=essential
unstable - | sha256sum
[...]
f40a3d2e9e168c3ec6270de1be79c522ce9f2381021c25072353bb3b5e1703d6 -
$ SOURCE_DATE_EPOCH=1570448177 mmdebstrap --variant=essential
unstable - | sha256sum
[...]
f40a3d2e9e168c3ec6270de1be79c522ce9f2381021c25072353bb3b5e1703d6 -
I think that's required, but not sufficient. That still depends on
someone actually verifying this fact and publishing their proofs.
Otherwise you need to do it yourself or risk getting a binary served to
your build process only if not checked interactively[0].
Lastly: yes, maybe "re-bootstrap, don't upgrade" is an equally good
principle.
It has the advantage of not accumulating cruft. The sbuild-createchroot
command
could gain an option which allows one to replace an existing chroot.
Currently,
it refuses to work on already existing chroots.
At work we always regenerate unless when you test multiple times locally
in quick succession. Assuming that the result is not totally wasteful
(e.g. by caching bootstrap debs locally) that seems like a good way to
get predictable local build environments.
Kind regards and thanks
Philipp Kern
[0] It seems to be a standard tool these days to serve exploits only
when the caller looks like the target environment. I.e. if you check the
script you pipe into bash from a browser it looks fine, if you curl it
into bash[1], it looks different.
[1] Yes, it seems like even that case can be identified.
Re: Debian and our frenemies of containers and userland repos
Hi,
shameless plug for mmdebstrap incoming.
Quoting Philipp Kern (2019-10-07 13:21:36)
> On 10/7/2019 1:17 PM, Shengjing Zhu wrote:
> > On Mon, Oct 7, 2019 at 6:29 PM Simon McVittie wrote:
> >> On Mon, 07 Oct 2019 at 07:22:53 +0200, Johannes Schauer wrote:
> >>> Specifically, currently autopkgtest is limited to providing a read-only
> >>> layer
> >>> for certain backends and its upstream has no intention of widening the
> >>> scope of
> >>> the software [1]. This means that to upgrade an autopkgtest backend, the
> >>> user
> >>> has to apply backend-specific actions
> >> I think "re-bootstrap, don't upgrade" is an equally good principle
> > Why not have a repository for it, like dockerhub. So this becomes
> > "pull latest build env", which saves lots of time("re-bootstrap" is still
> > slow nowadays).
> In that case it'd probably be better to make bootstrapping faster rather
> than trusting random binaries on the internet. (Unless we grow an
> "assemble an image from debs" service on, say, ftp-master.)
creating a working sbuild chroot takes 10 seconds on my system:
$ time mmdebstrap --variant=essential unstable debian-unstable.tar
[...]
mmdebstrap [...] 8.35s user 1.73s system 99% cpu 10.166 total
Do we need to spend engineering effort to become faster than that?
Downloading "random binary from the internet" is less of a problem if we can
create images which are bit-by-bit identical to checksums that we can verify
through a trusted service. This is also already provided by mmdebstrap:
$ SOURCE_DATE_EPOCH=1570448177 mmdebstrap --variant=essential unstable - |
sha256sum
[...]
f40a3d2e9e168c3ec6270de1be79c522ce9f2381021c25072353bb3b5e1703d6 -
$ SOURCE_DATE_EPOCH=1570448177 mmdebstrap --variant=essential unstable - |
sha256sum
[...]
f40a3d2e9e168c3ec6270de1be79c522ce9f2381021c25072353bb3b5e1703d6 -
Lastly: yes, maybe "re-bootstrap, don't upgrade" is an equally good principle.
It has the advantage of not accumulating cruft. The sbuild-createchroot command
could gain an option which allows one to replace an existing chroot. Currently,
it refuses to work on already existing chroots.
Thanks!
cheers, josch
signature.asc
Description: signature
Re: Debian and our frenemies of containers and userland repos
On Mon, Oct 7, 2019 at 7:21 PM Philipp Kern wrote: > In that case it'd probably be better to make bootstrapping faster rather > than trusting random binaries on the internet. (Unless we grow an > "assemble an image from debs" service on, say, ftp-master.) > We already have such, the cloud image service.(I think these images only include minbase variant, not buildd variant). https://cloud.debian.org/images/cloud/ -- Shengjing Zhu
Re: Debian and our frenemies of containers and userland repos
On 10/7/2019 1:17 PM, Shengjing Zhu wrote:
> On Mon, Oct 7, 2019 at 6:29 PM Simon McVittie wrote:
>>
>> On Mon, 07 Oct 2019 at 07:22:53 +0200, Johannes Schauer wrote:
>>> Specifically, currently autopkgtest is limited to providing a read-only
>>> layer
>>> for certain backends and its upstream has no intention of widening the
>>> scope of
>>> the software [1]. This means that to upgrade an autopkgtest backend, the
>>> user
>>> has to apply backend-specific actions
>>
>> I think "re-bootstrap, don't upgrade" is an equally good principle
>
> Why not have a repository for it, like dockerhub. So this becomes
> "pull latest build env", which saves lots of time("re-bootstrap" is
> still slow nowadays).
In that case it'd probably be better to make bootstrapping faster rather
than trusting random binaries on the internet. (Unless we grow an
"assemble an image from debs" service on, say, ftp-master.)
Kind regards
Philipp Kern
Re: Debian and our frenemies of containers and userland repos
On Mon, Oct 7, 2019 at 6:29 PM Simon McVittie wrote:
>
> On Mon, 07 Oct 2019 at 07:22:53 +0200, Johannes Schauer wrote:
> > Specifically, currently autopkgtest is limited to providing a read-only
> > layer
> > for certain backends and its upstream has no intention of widening the
> > scope of
> > the software [1]. This means that to upgrade an autopkgtest backend, the
> > user
> > has to apply backend-specific actions
>
> I think "re-bootstrap, don't upgrade" is an equally good principle
Why not have a repository for it, like dockerhub. So this becomes
"pull latest build env", which saves lots of time("re-bootstrap" is
still slow nowadays).
--
Shengjing Zhu
Re: Debian and our frenemies of containers and userland repos
On Mon, 07 Oct 2019 at 07:22:53 +0200, Johannes Schauer wrote: > Specifically, currently autopkgtest is limited to providing a read-only layer > for certain backends and its upstream has no intention of widening the scope > of > the software [1]. This means that to upgrade an autopkgtest backend, the user > has to apply backend-specific actions I think "re-bootstrap, don't upgrade" is an equally good principle for autopkgtest and sbuild? Both will be equally susceptible to accumulating cruft during upgrades that wouldn't have been there in a fresh debootstrap, which is undesired if you want the invariant that you are (building|testing) in "today's" minimal environment. smcv
Re: Debian and our frenemies of containers and userland repos
On Mon, Oct 7, 2019 at 1:23 PM Johannes Schauer wrote: > > Quoting Paul Wise (2019-10-07 03:38:55) > > On Sun, Oct 6, 2019 at 7:23 PM Simon McVittie wrote: > > > FYI, this is because autopkgtest has an abstraction for multiple > > > container/virtualization mechanisms (lxc, lxd, qemu, schroot) > > It seems like this abstraction should be split out of the autopkgtest > > source and then depended on by autopkgtest. Do any other such > > abstraction layers exist? > > I do not know of any other such abstraction layers but would warmly welcome > them not only for the purpose of including them in sbuild. > Not sure if containerd[1] and cri[2] are such abstraction layers. They are widely used in cloud/container area. For containerd, it supports both container and virtualization[3] backends. [1] https://containerd.io/ [2] https://github.com/kubernetes/community/blob/master/contributors/devel/sig-node/container-runtime-interface.md [3] https://katacontainers.io/ -- Shengjing Zhu
Re: Debian and our frenemies of containers and userland repos
Quoting Paul Wise (2019-10-07 03:38:55) > On Sun, Oct 6, 2019 at 7:23 PM Simon McVittie wrote: > > FYI, this is because autopkgtest has an abstraction for multiple > > container/virtualization mechanisms (lxc, lxd, qemu, schroot) > It seems like this abstraction should be split out of the autopkgtest > source and then depended on by autopkgtest. Do any other such > abstraction layers exist? I do not know of any other such abstraction layers but would warmly welcome them not only for the purpose of including them in sbuild. Specifically, currently autopkgtest is limited to providing a read-only layer for certain backends and its upstream has no intention of widening the scope of the software [1]. This means that to upgrade an autopkgtest backend, the user has to apply backend-specific actions which defeats the purpose of having a unified abstraction layer for multiple backends in the form they get used by sbuild. [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=886332 This is also one of the main reasons why the autopkgtest backend is still marked as experimental by sbuild as other contributors to this thread already noticed: sbuild-update will never be able to upgrade autopkgtest-based backends. :( Thanks! cheers, josch signature.asc Description: signature
Re: Debian and our frenemies of containers and userland repos
On Sun, Oct 6, 2019 at 7:23 PM Simon McVittie wrote: > FYI, this is because autopkgtest has an abstraction for multiple > container/virtualization mechanisms (lxc, lxd, qemu, schroot) It seems like this abstraction should be split out of the autopkgtest source and then depended on by autopkgtest. Do any other such abstraction layers exist? -- bye, pabs https://wiki.debian.org/PaulWise
Re: Debian and our frenemies of containers and userland repos
On Sat, 05 Oct 2019 at 12:41:55 -0400, Nicholas D Steeves wrote: > AFAIK sbuild has had this support for a while with --chroot-mode > autopkgtest, --autopkgtest-virt-server (lxc, lxd, or qemu), and > --autopkgtest-virt-server-opts='name of container goes here' will also > do the trick; however, it's still marked experimental. > > https://manpages.debian.org/unstable/sbuild/sbuild.1.en.html > > Yes, I also find it strange that one has to use arguments named > "autopkgtest" to build on LXC, but that seems to be what the fine manual > indicates how it works. FYI, this is because autopkgtest has an abstraction for multiple container/virtualization mechanisms (lxc, lxd, qemu, schroot), which some other tools (sbuild, reprotest, vectis) have reused as a way to avoid having to invent their own equivalent abstraction. sbuild doesn't know anything about lxc specifically, but it does know how to use an arbitrary autopkgtest-virt-* backend, and autopkgtest-virt-lxc is one of those. (There's an autopkgtest merge request open to add a docker backend, which I'll try to fix up enough to get it merged at some point.) smcv
Re: Debian and our frenemies of containers and userland repos
On 10/5/19 12:25, Bernd Zeimetz wrote: > > > On 10/5/19 3:31 AM, Paul Wise wrote: >> On Fri, Oct 4, 2019 at 10:49 PM Enrico Weigelt wrote: >>> On 24.07.19 08:17, Marc Haber wrote: >>> Do we have a build technology that uses containers instead of chroots yet? >>> >>> Something like docker-buildpackage ? >> >> AFAICT, docker-buildpackage doesn't exist but whalebuilder does. > > https://github.com/metux/docker-buildpackage > > > Having something that works with git-buildpackage would be really nice, > though. Even better if it would allow to use the k8s API to build things... > deb-build-pkg build -o ../ https://github.com/resnullius/deb-build-pkg It's a simple bash script. Maybe I should write better docs than the help already available with `deb-build-pkg --help`. It's not using k8s API but I don't see a reason why it would not be compatible. Building a Deployment yaml on-the-fly and you're ready. Just calling something different than the `run_docker()` function. With deb-build-pkg you don't even need to be in a Debian system to build stuff. -- __.https://wiki.debian.org/JoseLuisRivas ___'/___ rsa4096/D278F9C15E5461AA3C1E2FCD13EC43EEB9AC8C43
[What happened to CUT?] Re: Debian and our frenemies of containers and userland repos
Bernd Zeimetz writes: > I'm wondering if there is something Debian can do to be even more > successful in the container world. Like regular releases of a container > base image from testing. The amount of packages that needed for that is > limited, the number of RC bugs is probably low enough most of the time. > Maybe try resuscitating this project, possibly with a "for containers" focus, focusing on a small subset of the available packages? https://cut.debian.net On the topic of CUT, does anyone know what happened to it? Cheers, Nicholas signature.asc Description: PGP signature
Re: Debian and our frenemies of containers and userland repos
Hi, Quoting Nicholas D Steeves (2019-10-05 18:41:55) > Bernd Zeimetz writes: > > On 10/5/19 3:31 AM, Paul Wise wrote: > >> On Fri, Oct 4, 2019 at 10:49 PM Enrico Weigelt wrote: > >>> On 24.07.19 08:17, Marc Haber wrote: > Do we have a build technology that uses containers instead of chroots > yet? > AFAIK sbuild has had this support for a while with --chroot-mode > autopkgtest, --autopkgtest-virt-server (lxc, lxd, or qemu), and > --autopkgtest-virt-server-opts='name of container goes here' will also > do the trick; however, it's still marked experimental. > > https://manpages.debian.org/unstable/sbuild/sbuild.1.en.html > > Yes, I also find it strange that one has to use arguments named > "autopkgtest" to build on LXC, but that seems to be what the fine manual > indicates how it works. I also have no problem adding a docker backend to sbuild. There is a bug but since I don't use docker I cannot test it. If anybody else would like to use sbuild with docker you can help refine an existing proof-of-concept patch here: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=867176 Thanks! cheers, josch signature.asc Description: signature
Re: Debian and our frenemies of containers and userland repos
On Sat, Oct 05, 2019 at 07:03:06PM +0200, Bernd Zeimetz wrote: > I'm wondering if there is something Debian can do to be even more > successful in the container world. Like regular releases of a container > base image from testing. The amount of packages that needed for that is > limited, the number of RC bugs is probably low enough most of the time. do? -- cheers, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C signature.asc Description: PGP signature
Re: Debian and our frenemies of containers and userland repos
Hi, On 7/11/19 5:48 PM, Kyle Edwards wrote: > The demand is there. APT-style repositories aren't going away any time > soon. definitely not, I think the amount of various apt repositories will rise drastically. >From the container world most people are in the need of a stable base image for their containers. For that, developers also need or want the latest shiniest software - which is something a distribution can't provide. RedHat realized that and is providing their Universal Base Image now, which can be freely shared in container images. I'm wondering if there is something Debian can do to be even more successful in the container world. Like regular releases of a container base image from testing. The amount of packages that needed for that is limited, the number of RC bugs is probably low enough most of the time. Bernd -- Bernd ZeimetzDebian GNU/Linux Developer http://bzed.dehttp://www.debian.org GPG Fingerprint: ECA1 E3F2 8E11 2432 D485 DD95 EB36 171A 6FF9 435F
Re: Debian and our frenemies of containers and userland repos
Bernd Zeimetz writes: > On 10/5/19 3:31 AM, Paul Wise wrote: >> On Fri, Oct 4, 2019 at 10:49 PM Enrico Weigelt wrote: >>> On 24.07.19 08:17, Marc Haber wrote: >>> Do we have a build technology that uses containers instead of chroots yet? >>> AFAIK sbuild has had this support for a while with --chroot-mode autopkgtest, --autopkgtest-virt-server (lxc, lxd, or qemu), and --autopkgtest-virt-server-opts='name of container goes here' will also do the trick; however, it's still marked experimental. https://manpages.debian.org/unstable/sbuild/sbuild.1.en.html Yes, I also find it strange that one has to use arguments named "autopkgtest" to build on LXC, but that seems to be what the fine manual indicates how it works. >>> Something like docker-buildpackage ? >> >> AFAICT, docker-buildpackage doesn't exist but whalebuilder does. > > https://github.com/metux/docker-buildpackage > > > Having something that works with git-buildpackage would be really nice, > though. Even better if it would allow to use the k8s API to build things... > Sorry, I have no idea about the k8s API, or if plain LXC containers count as containers for the purposes of this thread. Sbuild arguments are definitely supported by git-buildpackage though, so maybe the above method will do the trick? Cheers, Nicholas signature.asc Description: PGP signature
Re: Debian and our frenemies of containers and userland repos
On 10/5/19 3:31 AM, Paul Wise wrote: > On Fri, Oct 4, 2019 at 10:49 PM Enrico Weigelt wrote: >> On 24.07.19 08:17, Marc Haber wrote: >> >>> Do we have a build technology that uses containers instead of chroots >>> yet? >> >> Something like docker-buildpackage ? > > AFAICT, docker-buildpackage doesn't exist but whalebuilder does. https://github.com/metux/docker-buildpackage Having something that works with git-buildpackage would be really nice, though. Even better if it would allow to use the k8s API to build things... -- Bernd ZeimetzDebian GNU/Linux Developer http://bzed.dehttp://www.debian.org GPG Fingerprint: ECA1 E3F2 8E11 2432 D485 DD95 EB36 171A 6FF9 435F
Re: Debian and our frenemies of containers and userland repos
On Fri, Oct 4, 2019 at 10:49 PM Enrico Weigelt wrote: > On 24.07.19 08:17, Marc Haber wrote: > > > Do we have a build technology that uses containers instead of chroots > > yet? > > Something like docker-buildpackage ? AFAICT, docker-buildpackage doesn't exist but whalebuilder does. For systemd-nspawn containers there is debspawn. -- bye, pabs https://wiki.debian.org/PaulWise
Re: Debian and our frenemies of containers and userland repos
On 10/4/19 10:49, Enrico Weigelt, metux IT consult wrote: > On 24.07.19 08:17, Marc Haber wrote: > >> Do we have a build technology that uses containers instead of chroots >> yet? > > Something like docker-buildpackage ? > For this purpose I built deb-build-pkg which is a pretty straight-forward way to build debian packages, it's a simple bash script that only requires bash and docker in your system in order to build the packages. https://github.com/resnullius/deb-build-pkg/blob/master/deb-build-pkg -- __.https://wiki.debian.org/JoseLuisRivas ___'/___ rsa4096/D278F9C15E5461AA3C1E2FCD13EC43EEB9AC8C43
Re: Debian and our frenemies of containers and userland repos
On 24.07.19 08:17, Marc Haber wrote: > Do we have a build technology that uses containers instead of chroots > yet? Something like docker-buildpackage ? --mtx -- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287
Re: Debian and our frenemies of containers and userland repos
Hi, Quoting Simon McVittie (2019-07-26 09:51:24) > schroot is also setuid root, and sbuild relies on this to set up the > build-dependencies anyway, so in principle schroot/sbuild ought to be > able to do something like this: > > - preparation step (as real root, in the chroot, with networking): > - install build-dependencies > - either run bubblewrap or reimplement it > - build step (as ordinary user, entering the chroot as a container, > with no networking): > - dpkg-buildpackage > - cleanup step (as real root, in the chroot): > - destroy session chroot, if used > > Doing that internally in schroot would require it to be actively > developed, but maybe it would be feasible to have code in sbuild that > wrapped bwrap (or even the combination of unshare(1) and setpriv(1)) > around (only) the actual build step? > > With the Debian Policy requirements around not writing to directories other > than /tmp, /var/tmp and the build directory, this would look something like: > > bwrap \ > --unshare-all \ > --ro-bind / / \# or --ro-bind /var/.../my-chroot / > --bind /tmp /tmp \ # or --tmpfs /tmp > --bind /var/tmp /var/tmp \ # or --tmpfs /var/tmp > --bind /build/hello-2.10-2 /build/hello-2.10-2 \ # or wherever the > build directory is kept > --setenv TMPDIR /tmp \ > --dev-bind /dev /dev \ # or --dev /dev for a minimal version > --proc /proc \ > --die-with-parent \ > --chdir /build/hello-2.10-2 \ > dpkg-buildpackage > > This would break any package that relies on being able to run setuid > executables (such as bwrap itself), and get privileges that way, during > its build - but perhaps that's desirable, because buildd operators > probably don't want setuid to be allowed anyway, in case it can be used > to escape the chroot? this is all very interesting! Thanks for writing it up! I will not be spending time on writing a backend using bubble wrap but I'll accept patches if anybody would like to do that work. This could easily be done by extending the current "sudo" chroot mode and wrapping the package build step itself by bubblewrap. Indeed it could probably be already done today by setting the BUILD_ENV_CMND config value to the bwrap line you posted above with the sudo chroot mode. Thanks! cheers, josch signature.asc Description: signature
Re: Debian and our frenemies of containers and userland repos
On Fri, 26 Jul 2019 at 07:34:58 +0200, Johannes Schauer wrote: > Sadly > this functionality requires a horribly complicated fork/syscall dance [1] > which > I also had to copy to mmdebstrap because no existing tool seemed to do it > already. bubblewrap might do the same dance, or at least a compatible dance? It won't be suitable for mmdebstrap and general full-system containers (there's only one uid and one gid in the container, with everything visible from the host system mapped to 'nobody') but it might be suitable for more restricted uses like building a Debian package. > But to unshare all the namespaces, even schroot would need #898446 to be > fixed. On systems that restrict user namespace creation (Debian, Arch Linux, RHEL), bubblewrap gets round this by being setuid root. To avoid reopening the same security vulnerabilities that would be available via unrestricted user namespace creation, it severely limits what it is willing to do/allow in user-supplied code in the container, in particular setting NO_NEW_PRIVS so that the setuid bit no longer works. schroot is also setuid root, and sbuild relies on this to set up the build-dependencies anyway, so in principle schroot/sbuild ought to be able to do something like this: - preparation step (as real root, in the chroot, with networking): - install build-dependencies - either run bubblewrap or reimplement it - build step (as ordinary user, entering the chroot as a container, with no networking): - dpkg-buildpackage - cleanup step (as real root, in the chroot): - destroy session chroot, if used Doing that internally in schroot would require it to be actively developed, but maybe it would be feasible to have code in sbuild that wrapped bwrap (or even the combination of unshare(1) and setpriv(1)) around (only) the actual build step? With the Debian Policy requirements around not writing to directories other than /tmp, /var/tmp and the build directory, this would look something like: bwrap \ --unshare-all \ --ro-bind / / \# or --ro-bind /var/.../my-chroot / --bind /tmp /tmp \ # or --tmpfs /tmp --bind /var/tmp /var/tmp \ # or --tmpfs /var/tmp --bind /build/hello-2.10-2 /build/hello-2.10-2 \ # or wherever the build directory is kept --setenv TMPDIR /tmp \ --dev-bind /dev /dev \ # or --dev /dev for a minimal version --proc /proc \ --die-with-parent \ --chdir /build/hello-2.10-2 \ dpkg-buildpackage This would break any package that relies on being able to run setuid executables (such as bwrap itself), and get privileges that way, during its build - but perhaps that's desirable, because buildd operators probably don't want setuid to be allowed anyway, in case it can be used to escape the chroot? smcv
Re: Debian and our frenemies of containers and userland repos
Hi, Quoting Paul Wise (2019-07-26 04:31:29) > > But you'd have to ask somebody who is more knowledgable about the security > > implications of such a change. There certainly is a reason why #898446 is > > still > > open. > > > > Furthermore, since buildds currently use the schroot backend, I guess that > > buildd admins already took all necessary precautions to secure their systems > > against arbitrary code running as part of the package build process. I do > > not > > know what benefit the "unshare" backend would have for buildds. > I think my mental model of what the "unshare" backend does was > incorrect. I didn't think it needed #898446 to be closed. I assumed it > was just like schroot except with the addition of moving all processes > run within the chroot into a separate network/process/mount/etc > namespace with no access to the host namespaces. Your initial intuition was correct. It is like a very primitive schroot with just enough functionality so that sbuild can build packages with it. It lacks all the advanced features that schroot has like configuration file management and session management and it is baked directly into sbuild so you cannot use it without sbuild. But feel free to steal the code for your own project! Sadly this functionality requires a horribly complicated fork/syscall dance [1] which I also had to copy to mmdebstrap because no existing tool seemed to do it already. [1] https://sources.debian.org/src/mmdebstrap/0.4.1-6/mmdebstrap/#L292 > The primary advantage of this would be to isolate the build chroot from the > network. Perhaps schroot is the component that should be adding support for > separate network/process/mount/etc namespaces? Yes, it should. There is this bug for it which I openend and I only wrote the "unshare" backend for sbuild when it became clear that schroot would not add this functionality any time soon: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=802849 But to unshare all the namespaces, even schroot would need #898446 to be fixed. Thanks! cheers, josch signature.asc Description: signature
Re: Debian and our frenemies of containers and userland repos
On Thu, Jul 25, 2019 at 2:18 PM Johannes Schauer wrote: > But you'd have to ask somebody who is more knowledgable about the security > implications of such a change. There certainly is a reason why #898446 is > still > open. > > Furthermore, since buildds currently use the schroot backend, I guess that > buildd admins already took all necessary precautions to secure their systems > against arbitrary code running as part of the package build process. I do not > know what benefit the "unshare" backend would have for buildds. I think my mental model of what the "unshare" backend does was incorrect. I didn't think it needed #898446 to be closed. I assumed it was just like schroot except with the addition of moving all processes run within the chroot into a separate network/process/mount/etc namespace with no access to the host namespaces. The primary advantage of this would be to isolate the build chroot from the network. Perhaps schroot is the component that should be adding support for separate network/process/mount/etc namespaces? -- bye, pabs https://wiki.debian.org/PaulWise
Re: Debian and our frenemies of containers and userland repos
Hi, Quoting Paul Wise (2019-07-25 05:38:49) > On Wed, Jul 24, 2019 at 2:42 PM Johannes Schauer wrote: > > Or using the built-in "unshare" backend which uses linux user namespaces: > > > > $ sbuild --chroot-mode=unshare --chroot=debian-unstable.tar > > Do you think it is feasible to use this on some of or the majority of > Debian buildds (most run stretch right now)? There would need to be > exceptions but most packages should build correctly in this scenario. > The main exception would be the Debian installer, IIRC it needs > network to the local apt mirror during the build process. it would certainly be possible. But you'd have to ask somebody who is more knowledgable about the security implications of such a change. There certainly is a reason why #898446 is still open. Furthermore, since buildds currently use the schroot backend, I guess that buildd admins already took all necessary precautions to secure their systems against arbitrary code running as part of the package build process. I do not know what benefit the "unshare" backend would have for buildds. Indeed I rather made the "unshare" backend for personal use. With it, I can manage all my chroots in my home directory without needing sudo to create or update them. In the light of #898446 this might give me a false sense of security but since I'm only ever processing Debian chroots, that should be fine. Using sudo always scares me a bit. When using the schroot backend (which is suid root) I sometimes have funny effects like some mounts still being there and not unmounted after I cancelled a build or some unpacked chroots still lingering around somewhere. Or processes run as root still running somewhere. I then have to use "sudo" to clean this up and I would like to avoid that. With the "unshare" backend, there can be no lingering mounts or processes and I rather "rm -r" in my home directory than somewhere in /. When using "unshare" there will not be any lingering processes because everything is inside its own process group and there will not be any lingering mounts. Or bugs like #833525 in principle should not happen anymore because the unshared user does not even have access to my home directory and thus cannot delete anything that it didn't create itself. Thanks! cheers, josch signature.asc Description: signature
Re: Debian and our frenemies of containers and userland repos
On Wed, Jul 24, 2019 at 2:42 PM Johannes Schauer wrote: > Or using the built-in "unshare" backend which uses linux user namespaces: > > $ sbuild --chroot-mode=unshare --chroot=debian-unstable.tar Do you think it is feasible to use this on some of or the majority of Debian buildds (most run stretch right now)? There would need to be exceptions but most packages should build correctly in this scenario. The main exception would be the Debian installer, IIRC it needs network to the local apt mirror during the build process. -- bye, pabs https://wiki.debian.org/PaulWise
Re: Debian and our frenemies of containers and userland repos
Quoting Marc Haber (2019-07-24 08:17:19) > On Mon, 22 Jul 2019 12:38:36 +0200, "Enrico Weigelt, metux IT consult" > wrote: > >Containerization is a valid approach for some kind of workloads > >(eg. specific inhouse applications) that can be easily isolated from > >the rest. But it comes with the price of huge redundancies (depending > >on how huge some application stacks are). And unless everybody wants > >to go back of maintaining everything on his own, we still need distros. > > Compared to a full VM, a container _is_ smaller. I am not sure whether > the difference is as huge in times where we have kernel same-page > merging though. You can create really small Debian chroots with mmdebstrap. In contrast to debootstrap, you can create chroots with just all Essential:yes packages and their dependencies (debootstrap cannot do less than Priority:required): $ mmdebstrap --variant=essential unstable debian-unstable.tar and you can use the dpkg path-excluded feature to remove lots of stuff you might not need inside a container: $ mmdebstrap --variant=essential \ --dpkgopt='path-exclude=/usr/share/man/*' \ --dpkgopt='path-include=/usr/share/man/man[1-9]/*' \ --dpkgopt='path-exclude=/usr/share/locale/*' \ --dpkgopt='path-include=/usr/share/locale/locale.alias' \ --dpkgopt='path-exclude=/usr/share/doc/*' \ --dpkgopt='path-include=/usr/share/doc/*/copyright' \ --dpkgopt='path-include=/usr/share/doc/*/changelog.Debian.*' \ unstable debian-unstable.tar or even with less than the Essential:yes package set but busybox instead: $ mmdebstrap --variant=custom \ --include=dpkg,busybox,libc-bin,base-files,base-passwd,debianutils \ --setup-hook='mkdir -p "\$1/bin"' \ --setup-hook='for p in awk cat chmod chown cp diff echo env grep less ln mkdir mount rm rmdir sed sh sleep sort touch uname; do ln -s busybox "\$1/bin/\$p"; done' \ --setup-hook='echo root:x:0:0:root:/root:/bin/sh > "\$1/etc/passwd"' \ --setup-hook='printf "root:x:0:\nmail:x:8:\nutmp:x:43:\n" > "\$1/etc/group"' \ unstable debian-unstable.tar Thanks! cheers, josch signature.asc Description: signature
Re: Debian and our frenemies of containers and userland repos
Quoting Marc Haber (2019-07-24 08:17:19) > Do we have a build technology that uses containers instead of chroots yet? Either using any container-based autopkgtest backend (like lxc or lxd): $ sbuild --chroot-mode=autopkgtest --autopkgtest-virt-server=lxc Or using the built-in "unshare" backend which uses linux user namespaces: $ sbuild --chroot-mode=unshare --chroot=debian-unstable.tar The latter allows one to either directly specify a chroot tarball with the --chroot argument or will look inside ~/.cache/sbuild for a fitting chroot tarball. If you also build your chroot tarballs using a tool that doesn't require superuser privileges like mmdebstrap (or debootstrap with the patch from #829134) then you can essentially build arbitrary packages inside arbitrary chroots without ever becoming root or touching anything outside your home directory, given that you at some point did "sysctl -w kernel.unprivileged_userns_clone=1" until #898446 is fixed. Thanks! cheers, josch signature.asc Description: signature
Re: Debian and our frenemies of containers and userland repos
Marc Haber writes: > Compared to a full VM, a container _is_ smaller. I am not sure whether > the difference is as huge in times where we have kernel same-page > merging though. > > Do we have a build technology that uses containers instead of chroots > yet? I haven't used it so far, but at least "whalebuilder" exists. The gitlab-ci used on salsa.d.o also uses Docker containers; people also build packages using this (mostly for testing though, see for example [1]). Ansgar [1] https://salsa.debian.org/salsa-ci-team/pipeline
Re: Debian and our frenemies of containers and userland repos
On Mon, 22 Jul 2019 12:38:36 +0200, "Enrico Weigelt, metux IT consult" wrote: >Containerization is a valid approach for some kind of workloads >(eg. specific inhouse applications) that can be easily isolated from >the rest. But it comes with the price of huge redundancies (depending >on how huge some application stacks are). And unless everybody wants >to go back of maintaining everything on his own, we still need distros. Compared to a full VM, a container _is_ smaller. I am not sure whether the difference is as huge in times where we have kernel same-page merging though. Do we have a build technology that uses containers instead of chroots yet? Greetings Marc -- -- !! No courtesy copies, please !! - Marc Haber | " Questions are the | Mailadresse im Header Mannheim, Germany | Beginning of Wisdom " | Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834
Re: Debian and our frenemies of containers and userland repos
Hi all, On 22.07.19 12:38, Enrico Weigelt, metux IT consult wrote: > COOS: just yet another special purpose distro, in that case for > docker hosts. neither the first, nor the last one to come. > Yocto: just yet another compile-yourself distro, focused on embeedded, > that happens to be hyped by certain corporations. > (for small/embedded devices, I'd really recommend ptxdist). > Alpine: yet another distro, optimized for running in small containers Just a shame it seems the default for everyone and their cat to use it as a base image. Recent article re Python container images: https://pythonspeed.com/articles/base-image-python-docker-images/ > Containerization is a valid approach for some kind of workloads > (eg. specific inhouse applications) that can be easily isolated from > the rest. But it comes with the price of huge redundancies (depending > on how huge some application stacks are). And unless everybody wants > to go back of maintaining everything on his own, we still need distros. > > If different applications need to deeply interact (eg. various plugin > stuff, applications calling each other, etc), containerization doesn't > help much. (eg: how can you have a pure texlive in one container and > extra things like fonts, document classes, etc, in separate ones ? :o) > > The whole point about containerization isn't about packaging and > deployment of individual applications - instead it's about automatizing > the rollout of fully-configured installations. Good points! Best Michael signature.asc Description: OpenPGP digital signature
Re: Debian and our frenemies of containers and userland repos
On 11.07.19 17:25, Yao Wei wrote: Hi, It can be a "solid base" of container images and barebone systems, but the days are numbered as operating systems as free and focused on its mission (like Google COOS, Yocto, Alpine etc.) is evolving steady. Could it be a disaster for us? And more importantly, do users care? I don't think so. COOS: just yet another special purpose distro, in that case for docker hosts. neither the first, nor the last one to come. Yocto: just yet another compile-yourself distro, focused on embeedded, that happens to be hyped by certain corporations. (for small/embedded devices, I'd really recommend ptxdist). Alpine: yet another distro, optimized for running in small containers BTW: the idea of building small payload/application-specific containers/chroot's is anything but new. I've done it somewhere in the 90th. But nowadays, these so-called "small" containers tend to be bigger than whole machines of the 90th. Containerization is a valid approach for some kind of workloads (eg. specific inhouse applications) that can be easily isolated from the rest. But it comes with the price of huge redundancies (depending on how huge some application stacks are). And unless everybody wants to go back of maintaining everything on his own, we still need distros. If different applications need to deeply interact (eg. various plugin stuff, applications calling each other, etc), containerization doesn't help much. (eg: how can you have a pure texlive in one container and extra things like fonts, document classes, etc, in separate ones ? :o) The whole point about containerization isn't about packaging and deployment of individual applications - instead it's about automatizing the rollout of fully-configured installations. One thing seems to be right: folks who always have been hostile towards the whole concept of distros now have a better excuse. --mtx -- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287
Re: Debian and our frenemies of containers and userland repos
On Thu, 2019-07-11 at 23:25 +0800, Yao Wei wrote: > Hi, > > Following to Mo Zhou's thread of Conda and Debian, it reminds me > that, > could Debian reduced into a "proof of concept" as an operating system > with collection of apps and things composed of completely free > software, > as more and more software repositories are moving away from the free > software repositories like Debian, and userland repositories and app > containers becomes more prominent and easier to access. I feel the > fear > when I was in a flatpak session in DebConf17. > > It can be a "solid base" of container images and barebone systems, > but > the days are numbered as operating systems as free and focused on its > mission (like Google COOS, Yocto, Alpine etc.) is evolving steady. > > Could it be a disaster for us? And more importantly, do users care? > > Best regards, > Yao Wei I think that there will always be some demand for APT-style packages as opposed to containers. I know that I personally am a lot more likely to use software if I can get it from apt-get than if I have to download a Docker image. I have also seen this first-hand from other people. Since we launched our Ubuntu CMake APT repository earlier this year, it has become massively popular. It is regularly getting hundreds of downloads per week, both from people inside our company and from external users. I suspect it is only going to get bigger over time, especially with our next release cycle. I get emails about it almost every day. The demand is there. APT-style repositories aren't going away any time soon. Kyle
Debian and our frenemies of containers and userland repos
Hi, Following to Mo Zhou's thread of Conda and Debian, it reminds me that, could Debian reduced into a "proof of concept" as an operating system with collection of apps and things composed of completely free software, as more and more software repositories are moving away from the free software repositories like Debian, and userland repositories and app containers becomes more prominent and easier to access. I feel the fear when I was in a flatpak session in DebConf17. It can be a "solid base" of container images and barebone systems, but the days are numbered as operating systems as free and focused on its mission (like Google COOS, Yocto, Alpine etc.) is evolving steady. Could it be a disaster for us? And more importantly, do users care? Best regards, Yao Wei signature.asc Description: PGP signature
