Re: Fast `guix environment --container' switch

[Pierre Neidhardt]

Hi Ludo!

Just tried `guix run inkscape`: it starts up in a fingersnap!
Impressive!

I tried with IceCat and while it starts, fonts are broken and network
access seems to be missing.

Would you have an idea of what's left to be done?
Shall we send this script to a new bug and iterate from there?

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/


signature.asc
Description: PGP signature

Re: Fast `guix environment --container' switch

[Christopher Lemmer Webber]

This is very cool.  We need something like this!

I'm replying partly to make a note that here's where some of this
conversation is... but I'm going to try to write an email over the next
couple of weeks of how to lead the way for making users safe through
Guix.  This is a useful reference starting point.

 - Chris


Ludovic Courtès writes:

> Hi!
>
> Pierre Neidhardt  skribis:
>
>> `guix environment --container ... -- my-foo-program` is great but a bit
>> slow to start.
>>
>> Is there a way to speed this up?
>
> The attached program (based on an experiment from 2018¹ with exciting
> yet to date mythical prospects) picks a program from $PATH (typically
> from your profile) and runs it in a container.  As in:
>
>   guix run inkscape
>
> It has less work to do compared to ‘guix environment’ so it is faster.
>
> HTH!
>
> Ludo’.
>
> ¹ https://lists.gnu.org/archive/html/help-guix/2018-01/msg00117.html
>
> ;;; GNU Guix --- Functional package management for GNU
> ;;; Copyright © 2018, 2019, 2020 Ludovic Courtès 
> ;;;
> ;;; This file is part of GNU Guix.
> ;;;
> ;;; GNU Guix is free software; you can redistribute it and/or modify it
> ;;; under the terms of the GNU General Public License as published by
> ;;; the Free Software Foundation; either version 3 of the License, or (at
> ;;; your option) any later version.
> ;;;
> ;;; GNU Guix is distributed in the hope that it will be useful, but
> ;;; WITHOUT ANY WARRANTY; without even the implied warranty of
> ;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> ;;; GNU General Public License for more details.
> ;;;
> ;;; You should have received a copy of the GNU General Public License
> ;;; along with GNU Guix.  If not, see .
>
> (define-module (guix scripts run)
>   #:use-module (guix ui)
>   #:use-module (guix utils)
>   #:use-module (guix scripts)
>   #:use-module (guix store)
>   #:use-module (guix packages)
>   #:use-module (guix derivations)
>   #:use-module ((guix build utils) #:select (which mkdir-p))
>   #:use-module (gnu build linux-container)
>   #:use-module (gnu system file-systems)
>   #:use-module (gnu packages)
>   #:use-module (srfi srfi-1)
>   #:use-module (srfi srfi-9 gnu)
>   #:use-module (srfi srfi-11)
>   #:use-module (srfi srfi-26)
>   #:use-module (srfi srfi-37)
>   #:use-module (ice-9 match)
>   #:export (guix-run))
>
> (define %options
>   (list (option '(#\h "help") #f #f
> (lambda args
>   (show-help)
>   (exit 0)))
> (option '(#\V "version") #f #f
> (lambda args
>   (show-version-and-exit "guix run")
>
> (define (show-help)
>   (display (G_ "Usage: guix run PACKAGE COMMAND...
> Run COMMAND from PACKAGE in a container.\n"))
>   (newline)
>   (display (G_ "
>   -h, --help display this help and exit"))
>   (display (G_ "
>   -V, --version  display version information and exit"))
>   (newline)
>   (show-bug-report-information))
>
>
>
> (define (bind-mount-spec/ro item)
>   (and (file-exists? item)
>(file-system
>  (device item)
>  (mount-point item)
>  (type "none")
>  (flags '(bind-mount read-only))
>  (check? #f
>
> (define (bind-mount-spec/rw item)
>   (and (file-exists? item)
>(file-system
>  (inherit (bind-mount-spec/ro item))
>  (flags '(bind-mount)
>
> ;; Safe in which applications run.
> (define-immutable-record-type 
>   (safe namespaces mappings)
>   safe?
>   (namespaces  safe-namespaces)
>   (mappingssafe-mappings))
>
> (define (application-safe items)
>   "Return safe corresponding to the application whose dependencies are listed
> in ITEMS."
>   (define packages
> (map (compose (cut package-name->name+version <> #\-)
>   store-path-package-name)
>  items))
>
>   (define x11? (member "libx11" packages))
>   (define dbus? (member "dbus" packages))
>   (define alsa? (member "alsa-lib" packages))
>   (define pulseaudio? (member "pulseaudio" packages))
>
>   (define mappings
> (let-syntax ((if (syntax-rules ()
>((_ condition body)
> (if condition
> (or (and=> body list) '())
> '()
>  (ro (identifier-syntax bind-mount-spec/ro))
>  (rw (identifier-syntax bind-mount-spec/rw)))
>   `(,(rw "/var/run/nscd/socket")
> ,@(if x11? (rw (string-append (getenv "HOME") "/.Xauthority")))
> ,@(if x11? (rw "/tmp/.X11-unix"))
> ,@(if x11? (rw (string-append "/run/user/"
>   (number->string (getuid)
> ,@(if dbus? (ro "/etc/machine-id"))
> ,@(if alsa? (rw "/dev/snd"))
> ,@(if pulseaudio? (rw (string-append (getenv "HOME") "/.pulse"))
>
>   (define namespaces
> ;; X11 applications need to run in the same IPC namespace as
> ;; the server.
> (i

Re: Fast `guix environment --container' switch

[Pierre Neidhardt]

Hi Ludo,

thanks for the snippet, this looks very useful indeed!  I'll check it
out later and report.

Thanks!

-- 
Pierre Neidhardt
https://ambrevar.xyz/


signature.asc
Description: PGP signature

Re: Fast `guix environment --container' switch

[Ludovic Courtès]

Hi!

Pierre Neidhardt  skribis:

> `guix environment --container ... -- my-foo-program` is great but a bit
> slow to start.
>
> Is there a way to speed this up?

The attached program (based on an experiment from 2018¹ with exciting
yet to date mythical prospects) picks a program from $PATH (typically
from your profile) and runs it in a container.  As in:

  guix run inkscape

It has less work to do compared to ‘guix environment’ so it is faster.

HTH!

Ludo’.

¹ https://lists.gnu.org/archive/html/help-guix/2018-01/msg00117.html

;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2018, 2019, 2020 Ludovic Courtès 
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix.  If not, see .

(define-module (guix scripts run)
  #:use-module (guix ui)
  #:use-module (guix utils)
  #:use-module (guix scripts)
  #:use-module (guix store)
  #:use-module (guix packages)
  #:use-module (guix derivations)
  #:use-module ((guix build utils) #:select (which mkdir-p))
  #:use-module (gnu build linux-container)
  #:use-module (gnu system file-systems)
  #:use-module (gnu packages)
  #:use-module (srfi srfi-1)
  #:use-module (srfi srfi-9 gnu)
  #:use-module (srfi srfi-11)
  #:use-module (srfi srfi-26)
  #:use-module (srfi srfi-37)
  #:use-module (ice-9 match)
  #:export (guix-run))

(define %options
  (list (option '(#\h "help") #f #f
(lambda args
  (show-help)
  (exit 0)))
(option '(#\V "version") #f #f
(lambda args
  (show-version-and-exit "guix run")

(define (show-help)
  (display (G_ "Usage: guix run PACKAGE COMMAND...
Run COMMAND from PACKAGE in a container.\n"))
  (newline)
  (display (G_ "
  -h, --help display this help and exit"))
  (display (G_ "
  -V, --version  display version information and exit"))
  (newline)
  (show-bug-report-information))



(define (bind-mount-spec/ro item)
  (and (file-exists? item)
   (file-system
 (device item)
 (mount-point item)
 (type "none")
 (flags '(bind-mount read-only))
 (check? #f

(define (bind-mount-spec/rw item)
  (and (file-exists? item)
   (file-system
 (inherit (bind-mount-spec/ro item))
 (flags '(bind-mount)

;; Safe in which applications run.
(define-immutable-record-type 
  (safe namespaces mappings)
  safe?
  (namespaces  safe-namespaces)
  (mappingssafe-mappings))

(define (application-safe items)
  "Return safe corresponding to the application whose dependencies are listed
in ITEMS."
  (define packages
(map (compose (cut package-name->name+version <> #\-)
  store-path-package-name)
 items))

  (define x11? (member "libx11" packages))
  (define dbus? (member "dbus" packages))
  (define alsa? (member "alsa-lib" packages))
  (define pulseaudio? (member "pulseaudio" packages))

  (define mappings
(let-syntax ((if (syntax-rules ()
   ((_ condition body)
(if condition
(or (and=> body list) '())
'()
 (ro (identifier-syntax bind-mount-spec/ro))
 (rw (identifier-syntax bind-mount-spec/rw)))
  `(,(rw "/var/run/nscd/socket")
,@(if x11? (rw (string-append (getenv "HOME") "/.Xauthority")))
,@(if x11? (rw "/tmp/.X11-unix"))
,@(if x11? (rw (string-append "/run/user/"
  (number->string (getuid)
,@(if dbus? (ro "/etc/machine-id"))
,@(if alsa? (rw "/dev/snd"))
,@(if pulseaudio? (rw (string-append (getenv "HOME") "/.pulse"))

  (define namespaces
;; X11 applications need to run in the same IPC namespace as
;; the server.
(if x11?
(fold delq %namespaces '(ipc net))
%namespaces))

  (safe namespaces mappings))

(define %not-colon
  (char-set-complement (char-set #\:)))

(define (guix-run . args)
  (define (parse-options)
;; Return the alist of option values.  With this hack, the first
;; non-option argument is considered to be the beginning of the command.
(let-values (((args command) (span (cut string-prefix? "-" <>) args)))
  (args-fold* args %options
  (lambda (opt name arg result)
(leave (G_ "~A: unrecognized option~%") name))
   

Re: Fast `guix environment --container' switch

[zimoun]

On Wed, 09 Dec 2020 at 11:55, Pierre Neidhardt  wrote:
> `guix environment` incurs an overhead:
>
> --8<---cut here---start->8---
> time /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10/bin/hello
> Hello, world!
>
> real  0m0.002s
> user  0m0.002s
> sys   0m0.000s
> --8<---cut here---end--->8---
>
> --8<---cut here---start->8---
> $ time guix environment --ad-hoc hello -- hello
> Hello, world!
>
> real  0m0.921s
> user  0m1.003s
> sys   0m0.091s
> --8<---cut here---end--->8---
>
> It's possible to bypass this overhead by using --root:

It by-passes the computations of derivations and profiles.  Somehow,
using --root is using a precomputed profile. IIUC.

> So is it possible to use a similar trick to run something containerized
> "instantly", i.e.  with less than, say, 100ms overhead?

I see, you would like to be able to run a profile in container, right?
Somehow, it is similar to the wanted [1], IIUC.

1: 


Cheers,
simon

Re: Fast `guix environment --container' switch

[Pierre Neidhardt]

`guix environment` incurs an overhead:

--8<---cut here---start->8---
time /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10/bin/hello
Hello, world!

real0m0.002s
user0m0.002s
sys 0m0.000s
--8<---cut here---end--->8---

--8<---cut here---start->8---
$ time guix environment --ad-hoc hello -- hello
Hello, world!

real0m0.921s
user0m1.003s
sys 0m0.091s
--8<---cut here---end--->8---

It's possible to bypass this overhead by using --root:

--8<---cut here---start->8---
$ guix environment --ad-hoc hello --root=foo
[env]$ exit

$ source foo/etc/profile && time hello
Hello, world!

real0m0.003s
user0m0.003s
sys 0m0.000s
--8<---cut here---end--->8---

The above `source' is of course not containerized.

So is it possible to use a similar trick to run something containerized
"instantly", i.e.  with less than, say, 100ms overhead?

-- 
Pierre Neidhardt
https://ambrevar.xyz/


signature.asc
Description: PGP signature

Re: Fast `guix environment --container' switch

[zimoun]

Hi,

On Wed, 09 Dec 2020 at 11:13, Pierre Neidhardt  wrote:

> Maybe a misunderstanding, the question is not about --pure.  I'd like a
> `containerized hello' to start about as fast as non-containerized
> `hello', without the 1 s overhead.

Which 1s overhead?

The overhead between --pure and --container is 0.2s in both cases, which
is linux namespace creation and co.

Thereofore, on my machine, «“containerized hello” starts as fast as
“non-containerized hello”», with a 0.2s overhead.  So I am missing what
you are asking. :-)

I guess, in some case, this overhead is bigger.  That’s why I asked if
you have examples.


Cheers,
simon

Re: Fast `guix environment --container' switch

[Pierre Neidhardt]

Hi Simon,

Maybe a misunderstanding, the question is not about --pure.  I'd like a
`containerized hello' to start about as fast as non-containerized
`hello', without the 1 s overhead.

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/


signature.asc
Description: PGP signature

Re: Fast `guix environment --container' switch

[zimoun]

Hi Pierre,

On Wed, 09 Dec 2020 at 10:40, Pierre Neidhardt  wrote:

> `guix environment --container ... -- my-foo-program` is great but a bit
> slow to start.
>
> Is there a way to speed this up?

I get, cold cache:

--8<---cut here---start->8---
$ sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
$ time guix environment --container --ad-hoc hello -- hello
Hello, world!

real0m2.815s
user0m1.852s
sys 0m0.238s

$ sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
$ time guix environment --pure --ad-hoc hello -- hello
Hello, world!

real0m2.656s
user0m1.733s
sys 0m0.240s
--8<---cut here---end--->8---

and warm cache:

--8<---cut here---start->8---
$ time guix environment --container --ad-hoc hello -- hello
Hello, world!

real0m1.399s
user0m1.544s
sys 0m0.104s

$ time guix environment --pure --ad-hoc hello -- hello
Hello, world!

real0m1.250s
user0m1.492s
sys 0m0.094s
--8<---cut here---end--->8---


Do you have examples where the difference is significant?


All the best,
simon