Re: Debian and our frenemies of containers and userland repos

2019-10-28 Thread John Goerzen
On Mon, Oct 21 2019, Enrico Weigelt wrote: > On 05.10.19 03:31, Paul Wise wrote: >> On Fri, Oct 4, 2019 at 10:49 PM Enrico Weigelt wrote: >>> On 24.07.19 08:17, Marc Haber wrote: >>> Do we have a build technology that uses containers instead of chroots yet? >>> >>> Something like

Re: Debian and our frenemies of containers and userland repos

2019-10-21 Thread Paul Wise
On Mon, 2019-10-21 at 18:58 +0200, Enrico Weigelt wrote: > I'm pretty sure it does exist, since I wrote it :p > > https://github.com/metux/docker-buildpackage I couldn't find it in Debian so I incorrectly assumed, sorry. In case you want to get it into Debian:

Re: Debian and our frenemies of containers and userland repos

2019-10-21 Thread Enrico Weigelt, metux IT consult
On 05.10.19 19:03, Bernd Zeimetz wrote: > For that, developers also need or want the > latest shiniest software - which is something a distribution can't provide. It can, but that needs different workflows and higher grade of automation. (and of course wouldn't be so well tested) Actually, I

Re: Debian and our frenemies of containers and userland repos

2019-10-21 Thread Enrico Weigelt, metux IT consult
On 07.10.19 13:17, Shengjing Zhu wrote: > Why not have a repository for it, like dockerhub. So this becomes > "pull latest build env", which saves lots of time("re-bootstrap" is > still slow nowadays). No idea how sbuild works these days (turned away from it aeons ago, as I've found it too

Re: Debian and our frenemies of containers and userland repos

2019-10-21 Thread Enrico Weigelt, metux IT consult
On 05.10.19 18:25, Bernd Zeimetz wrote: Hi, > Having something that works with git-buildpackage would be really nice, :) > though. Even better if it would allow to use the k8s API to build things... Patches are always welcomed :) There're some problems to be solved for remote hosts (IMHO,

Re: Debian and our frenemies of containers and userland repos

2019-10-21 Thread Enrico Weigelt, metux IT consult
On 05.10.19 03:31, Paul Wise wrote: > On Fri, Oct 4, 2019 at 10:49 PM Enrico Weigelt wrote: >> On 24.07.19 08:17, Marc Haber wrote: >> >>> Do we have a build technology that uses containers instead of chroots >>> yet? >> >> Something like docker-buildpackage ? > > AFAICT, docker-buildpackage

InstallBootstrap (was: Re: Debian and our frenemies of containers and userland repos)

2019-10-08 Thread Johannes Schauer
Hi, Quoting Jonas Smedegaard (2019-10-08 10:30:45) > Quoting Paul Wise (2019-10-08 02:35:14) > > On Mon, 2019-10-07 at 11:29 +0100, Simon McVittie wrote: > > > I think "re-bootstrap, don't upgrade" is an equally good principle for > > > autopkgtest and sbuild? Both will be equally susceptible to

Re: Debian and our frenemies of containers and userland repos

2019-10-08 Thread Jonas Smedegaard
Quoting Paul Wise (2019-10-08 02:35:14) > On Mon, 2019-10-07 at 11:29 +0100, Simon McVittie wrote: > > > I think "re-bootstrap, don't upgrade" is an equally good principle for > > autopkgtest and sbuild? Both will be equally susceptible to accumulating > > cruft during upgrades that wouldn't have

"adt-virt-*" abstraction (was Re: Debian and our frenemies of containers and userland repos)

2019-10-08 Thread Holger Levsen
fwiw, src:piuparts also includes a (now) derived implementation of "adt-virt-*"... -- cheers, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09

Re: Debian and our frenemies of containers and userland repos

2019-10-07 Thread Paul Wise
On Mon, 2019-10-07 at 11:29 +0100, Simon McVittie wrote: > I think "re-bootstrap, don't upgrade" is an equally good principle for > autopkgtest and sbuild? Both will be equally susceptible to accumulating > cruft during upgrades that wouldn't have been there in a fresh debootstrap, > which is

Re: Debian and our frenemies of containers and userland repos

2019-10-07 Thread Ian Jackson
Paul Wise writes ("Re: Debian and our frenemies of containers and userland repos"): > On Sun, Oct 6, 2019 at 7:23 PM Simon McVittie wrote: > > FYI, this is because autopkgtest has an abstraction for multiple > > container/virtualization mechanisms (lxc, lxd, qemu, schroot) > > It seems like this

Re: Debian and our frenemies of containers and userland repos

2019-10-07 Thread Philipp Kern
On 2019-10-07 13:43, Johannes Schauer wrote: Quoting Philipp Kern (2019-10-07 13:21:36) On 10/7/2019 1:17 PM, Shengjing Zhu wrote: > On Mon, Oct 7, 2019 at 6:29 PM Simon McVittie wrote: >> On Mon, 07 Oct 2019 at 07:22:53 +0200, Johannes Schauer wrote: >>> Specifically, currently autopkgtest is

Re: Debian and our frenemies of containers and userland repos

2019-10-07 Thread Johannes Schauer
Hi, shameless plug for mmdebstrap incoming. Quoting Philipp Kern (2019-10-07 13:21:36) > On 10/7/2019 1:17 PM, Shengjing Zhu wrote: > > On Mon, Oct 7, 2019 at 6:29 PM Simon McVittie wrote: > >> On Mon, 07 Oct 2019 at 07:22:53 +0200, Johannes Schauer wrote: > >>> Specifically, currently

Re: Debian and our frenemies of containers and userland repos

2019-10-07 Thread Shengjing Zhu
On Mon, Oct 7, 2019 at 7:21 PM Philipp Kern wrote: > In that case it'd probably be better to make bootstrapping faster rather > than trusting random binaries on the internet. (Unless we grow an > "assemble an image from debs" service on, say, ftp-master.) > We already have such, the cloud image

Re: Debian and our frenemies of containers and userland repos

2019-10-07 Thread Philipp Kern
On 10/7/2019 1:17 PM, Shengjing Zhu wrote: > On Mon, Oct 7, 2019 at 6:29 PM Simon McVittie wrote: >> >> On Mon, 07 Oct 2019 at 07:22:53 +0200, Johannes Schauer wrote: >>> Specifically, currently autopkgtest is limited to providing a read-only >>> layer >>> for certain backends and its upstream

Re: Debian and our frenemies of containers and userland repos

2019-10-07 Thread Shengjing Zhu
On Mon, Oct 7, 2019 at 6:29 PM Simon McVittie wrote: > > On Mon, 07 Oct 2019 at 07:22:53 +0200, Johannes Schauer wrote: > > Specifically, currently autopkgtest is limited to providing a read-only > > layer > > for certain backends and its upstream has no intention of widening the > > scope of >

Re: Debian and our frenemies of containers and userland repos

2019-10-07 Thread Simon McVittie
On Mon, 07 Oct 2019 at 07:22:53 +0200, Johannes Schauer wrote: > Specifically, currently autopkgtest is limited to providing a read-only layer > for certain backends and its upstream has no intention of widening the scope > of > the software [1]. This means that to upgrade an autopkgtest backend,

Re: Debian and our frenemies of containers and userland repos

2019-10-07 Thread Shengjing Zhu
On Mon, Oct 7, 2019 at 1:23 PM Johannes Schauer wrote: > > Quoting Paul Wise (2019-10-07 03:38:55) > > On Sun, Oct 6, 2019 at 7:23 PM Simon McVittie wrote: > > > FYI, this is because autopkgtest has an abstraction for multiple > > > container/virtualization mechanisms (lxc, lxd, qemu, schroot) >

Re: Debian and our frenemies of containers and userland repos

2019-10-06 Thread Johannes Schauer
Quoting Paul Wise (2019-10-07 03:38:55) > On Sun, Oct 6, 2019 at 7:23 PM Simon McVittie wrote: > > FYI, this is because autopkgtest has an abstraction for multiple > > container/virtualization mechanisms (lxc, lxd, qemu, schroot) > It seems like this abstraction should be split out of the

Re: Debian and our frenemies of containers and userland repos

2019-10-06 Thread Paul Wise
On Sun, Oct 6, 2019 at 7:23 PM Simon McVittie wrote: > FYI, this is because autopkgtest has an abstraction for multiple > container/virtualization mechanisms (lxc, lxd, qemu, schroot) It seems like this abstraction should be split out of the autopkgtest source and then depended on by

Re: Debian and our frenemies of containers and userland repos

2019-10-06 Thread Simon McVittie
On Sat, 05 Oct 2019 at 12:41:55 -0400, Nicholas D Steeves wrote: > AFAIK sbuild has had this support for a while with --chroot-mode > autopkgtest, --autopkgtest-virt-server (lxc, lxd, or qemu), and > --autopkgtest-virt-server-opts='name of container goes here' will also > do the trick; however,

Re: Debian and our frenemies of containers and userland repos

2019-10-05 Thread Jose-Luis Rivas
On 10/5/19 12:25, Bernd Zeimetz wrote: > > > On 10/5/19 3:31 AM, Paul Wise wrote: >> On Fri, Oct 4, 2019 at 10:49 PM Enrico Weigelt wrote: >>> On 24.07.19 08:17, Marc Haber wrote: >>> Do we have a build technology that uses containers instead of chroots yet? >>> >>> Something like

[What happened to CUT?] Re: Debian and our frenemies of containers and userland repos

2019-10-05 Thread Nicholas D Steeves
Bernd Zeimetz writes: > I'm wondering if there is something Debian can do to be even more > successful in the container world. Like regular releases of a container > base image from testing. The amount of packages that needed for that is > limited, the number of RC bugs is probably low enough

Re: Debian and our frenemies of containers and userland repos

2019-10-05 Thread Johannes Schauer
Hi, Quoting Nicholas D Steeves (2019-10-05 18:41:55) > Bernd Zeimetz writes: > > On 10/5/19 3:31 AM, Paul Wise wrote: > >> On Fri, Oct 4, 2019 at 10:49 PM Enrico Weigelt wrote: > >>> On 24.07.19 08:17, Marc Haber wrote: > Do we have a build technology that uses containers instead of chroots

Re: Debian and our frenemies of containers and userland repos

2019-10-05 Thread Holger Levsen
On Sat, Oct 05, 2019 at 07:03:06PM +0200, Bernd Zeimetz wrote: > I'm wondering if there is something Debian can do to be even more > successful in the container world. Like regular releases of a container > base image from testing. The amount of packages that needed for that is > limited, the

Re: Debian and our frenemies of containers and userland repos

2019-10-05 Thread Bernd Zeimetz
Hi, On 7/11/19 5:48 PM, Kyle Edwards wrote: > The demand is there. APT-style repositories aren't going away any time > soon. definitely not, I think the amount of various apt repositories will rise drastically. >From the container world most people are in the need of a stable base image for

Re: Debian and our frenemies of containers and userland repos

2019-10-05 Thread Nicholas D Steeves
Bernd Zeimetz writes: > On 10/5/19 3:31 AM, Paul Wise wrote: >> On Fri, Oct 4, 2019 at 10:49 PM Enrico Weigelt wrote: >>> On 24.07.19 08:17, Marc Haber wrote: >>> Do we have a build technology that uses containers instead of chroots yet? >>> AFAIK sbuild has had this support for a

Re: Debian and our frenemies of containers and userland repos

2019-10-05 Thread Bernd Zeimetz
On 10/5/19 3:31 AM, Paul Wise wrote: > On Fri, Oct 4, 2019 at 10:49 PM Enrico Weigelt wrote: >> On 24.07.19 08:17, Marc Haber wrote: >> >>> Do we have a build technology that uses containers instead of chroots >>> yet? >> >> Something like docker-buildpackage ? > > AFAICT, docker-buildpackage

Re: Debian and our frenemies of containers and userland repos

2019-10-04 Thread Paul Wise
On Fri, Oct 4, 2019 at 10:49 PM Enrico Weigelt wrote: > On 24.07.19 08:17, Marc Haber wrote: > > > Do we have a build technology that uses containers instead of chroots > > yet? > > Something like docker-buildpackage ? AFAICT, docker-buildpackage doesn't exist but whalebuilder does. For

Re: Debian and our frenemies of containers and userland repos

2019-10-04 Thread Jose-Luis Rivas
On 10/4/19 10:49, Enrico Weigelt, metux IT consult wrote: > On 24.07.19 08:17, Marc Haber wrote: > >> Do we have a build technology that uses containers instead of chroots >> yet? > > Something like docker-buildpackage ? > For this purpose I built deb-build-pkg which is a pretty

Re: Debian and our frenemies of containers and userland repos

2019-10-04 Thread Enrico Weigelt, metux IT consult
On 24.07.19 08:17, Marc Haber wrote: > Do we have a build technology that uses containers instead of chroots > yet? Something like docker-buildpackage ? --mtx -- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287

Re: Debian and our frenemies of containers and userland repos

2019-07-26 Thread Johannes Schauer
Hi, Quoting Simon McVittie (2019-07-26 09:51:24) > schroot is also setuid root, and sbuild relies on this to set up the > build-dependencies anyway, so in principle schroot/sbuild ought to be > able to do something like this: > > - preparation step (as real root, in the chroot, with networking):

Re: Debian and our frenemies of containers and userland repos

2019-07-26 Thread Simon McVittie
On Fri, 26 Jul 2019 at 07:34:58 +0200, Johannes Schauer wrote: > Sadly > this functionality requires a horribly complicated fork/syscall dance [1] > which > I also had to copy to mmdebstrap because no existing tool seemed to do it > already. bubblewrap might do the same dance, or at least a

Re: Debian and our frenemies of containers and userland repos

2019-07-25 Thread Johannes Schauer
Hi, Quoting Paul Wise (2019-07-26 04:31:29) > > But you'd have to ask somebody who is more knowledgable about the security > > implications of such a change. There certainly is a reason why #898446 is > > still > > open. > > > > Furthermore, since buildds currently use the schroot backend, I

Re: Debian and our frenemies of containers and userland repos

2019-07-25 Thread Paul Wise
On Thu, Jul 25, 2019 at 2:18 PM Johannes Schauer wrote: > But you'd have to ask somebody who is more knowledgable about the security > implications of such a change. There certainly is a reason why #898446 is > still > open. > > Furthermore, since buildds currently use the schroot backend, I

Re: Debian and our frenemies of containers and userland repos

2019-07-25 Thread Johannes Schauer
Hi, Quoting Paul Wise (2019-07-25 05:38:49) > On Wed, Jul 24, 2019 at 2:42 PM Johannes Schauer wrote: > > Or using the built-in "unshare" backend which uses linux user namespaces: > > > > $ sbuild --chroot-mode=unshare --chroot=debian-unstable.tar > > Do you think it is feasible to use this

Re: Debian and our frenemies of containers and userland repos

2019-07-24 Thread Paul Wise
On Wed, Jul 24, 2019 at 2:42 PM Johannes Schauer wrote: > Or using the built-in "unshare" backend which uses linux user namespaces: > > $ sbuild --chroot-mode=unshare --chroot=debian-unstable.tar Do you think it is feasible to use this on some of or the majority of Debian buildds (most run

Re: Debian and our frenemies of containers and userland repos

2019-07-24 Thread Johannes Schauer
Quoting Marc Haber (2019-07-24 08:17:19) > On Mon, 22 Jul 2019 12:38:36 +0200, "Enrico Weigelt, metux IT consult" > wrote: > >Containerization is a valid approach for some kind of workloads > >(eg. specific inhouse applications) that can be easily isolated from > >the rest. But it comes with the

Re: Debian and our frenemies of containers and userland repos

2019-07-24 Thread Johannes Schauer
Quoting Marc Haber (2019-07-24 08:17:19) > Do we have a build technology that uses containers instead of chroots yet? Either using any container-based autopkgtest backend (like lxc or lxd): $ sbuild --chroot-mode=autopkgtest --autopkgtest-virt-server=lxc Or using the built-in "unshare"

Re: Debian and our frenemies of containers and userland repos

2019-07-24 Thread Ansgar
Marc Haber writes: > Compared to a full VM, a container _is_ smaller. I am not sure whether > the difference is as huge in times where we have kernel same-page > merging though. > > Do we have a build technology that uses containers instead of chroots > yet? I haven't used it so far, but at least

Re: Debian and our frenemies of containers and userland repos

2019-07-24 Thread Marc Haber
On Mon, 22 Jul 2019 12:38:36 +0200, "Enrico Weigelt, metux IT consult" wrote: >Containerization is a valid approach for some kind of workloads >(eg. specific inhouse applications) that can be easily isolated from >the rest. But it comes with the price of huge redundancies (depending >on how huge

Re: Debian and our frenemies of containers and userland repos

2019-07-23 Thread Michael Kesper
Hi all, On 22.07.19 12:38, Enrico Weigelt, metux IT consult wrote: > COOS:   just yet another special purpose distro, in that case for >     docker hosts. neither the first, nor the last one to come. > Yocto:  just yet another compile-yourself distro, focused on embeedded, >     that

Re: Debian and our frenemies of containers and userland repos

2019-07-22 Thread Enrico Weigelt, metux IT consult
On 11.07.19 17:25, Yao Wei wrote: Hi, It can be a "solid base" of container images and barebone systems, but the days are numbered as operating systems as free and focused on its mission (like Google COOS, Yocto, Alpine etc.) is evolving steady. Could it be a disaster for us? And more

Re: Debian and our frenemies of containers and userland repos

2019-07-11 Thread Kyle Edwards
On Thu, 2019-07-11 at 23:25 +0800, Yao Wei wrote: > Hi, > > Following to Mo Zhou's thread of Conda and Debian, it reminds me > that, > could Debian reduced into a "proof of concept" as an operating system > with collection of apps and things composed of completely free > software, > as more and

Debian and our frenemies of containers and userland repos

2019-07-11 Thread Yao Wei
Hi, Following to Mo Zhou's thread of Conda and Debian, it reminds me that, could Debian reduced into a "proof of concept" as an operating system with collection of apps and things composed of completely free software, as more and more software repositories are moving away from the free software