Hi, Lars-Dominik Braun <l...@leibniz-psychology.org> skribis:
> That would have been my best guess too, but it does not seem to be the biggest > problem right now. Looking at the numbers again (both patches applied) with > the > attached manifest[1], I see that: > > ---snip--- > Local UNIX socket with and without --no-grafts: > N Min Max Median Avg Stddev > x 10 6.07 6.35 6.145 6.16 0.08232726 > + 10 17.47 17.89 17.545 17.602 0.14351152 > Difference at 99.0% confidence > 11.442 +/- 0.150576 > 185.747% +/- 4.07133% > > Local UNIX socket vs. guix://localhost transport: > N Min Max Median Avg Stddev > x 10 17.47 17.89 17.545 17.602 0.14351152 > + 10 17.43 18.1 17.61 17.642 0.20131788 > No difference proven at 99.0% confidence > > Local UNIX socket vs ssh://localhost transport: > N Min Max Median Avg Stddev > x 10 17.47 17.89 17.545 17.602 0.14351152 > + 10 33.46 35.27 34.315 34.359 0.53873205 > Difference at 99.0% confidence > 16.757 +/- 0.5074 > 95.1994% +/- 3.13957% > ---snap--- > > So I would conclude: > > 1) Grafting still takes a lot of time and needs more work Yes, I noticed that there’s still redundant work being done, so we may need global caching in addition to what 58bb833365db4e8934a386497d5b00a063cfd27d did. > 2) Linux optimizes localhost networking pretty well > 3) Our SSH transport is terribly slow Yes. The patch below is a noticeable improvement for me. On my laptop, GUIX_DAEMON_SOCKET=ssh://localhost ./pre-inst-env guix build libreoffice -n goes from 5.8s to 3.3s. It just does the same thing as we do for guix://. Could you check what results it gives you? > So, I guess it would make sense for me to look at the SSH transport itself > again and see if there are any other low-hanging fruit. Not sure how much I > can > help with profiling guile/guix itself. A different/better RPC protocol is > probably GSoC/v2.0-worthy? There’s a project to rewrite the daemon in Scheme, started by Caleb Ristvedt (reepa) as part of GSoC a few years ago. This could be an opportunity to add a new version of the protocol that would support pipelining. Thanks, Ludo’.
diff --git a/guix/ssh.scm b/guix/ssh.scm index 2d7ca7d01d..a9312b7c8c 100644 --- a/guix/ssh.scm +++ b/guix/ssh.scm @@ -98,7 +98,7 @@ actual key does not match." key type)))))))) (define* (open-ssh-session host #:key user port identity - host-key + host-key open-connection (compression %compression) (timeout 3600)) "Open an SSH session for HOST and return it. IDENTITY specifies the file @@ -110,6 +110,10 @@ When HOST-KEY is true, it must be a string like \"ssh-ed25519 AAAAC3Nz… r...@example.org\"; the server is authenticated and an error is raised if its host key is different from HOST-KEY. +When OPEN-CONNECTION is true, it must be a two-argument procedure; it is +passed HOST and PORT and must return a socket (a file port). When +OPEN-CONNECTION is false, Guile-SSH takes care of opening the connection. + Install TIMEOUT as the maximum time in seconds after which a read or write operation on a channel of the returned session is considered as failing. @@ -134,6 +138,13 @@ Throw an error on failure." ;; Honor ~/.ssh/config. (session-parse-config! session) + (when open-connection + (let* ((sock (open-connection host port))) + ;; Since 'session-set!' doesn't increase the revealed count of SOCK + ;; (as of Guile-SSH 0.12.0), do it ourselves. + (set-port-revealed! sock 1) + (session-set! session 'fd sock))) + (match (connect! session) ('ok (if host-key diff --git a/guix/store/ssh.scm b/guix/store/ssh.scm index 09c0832505..d07f0d7467 100644 --- a/guix/store/ssh.scm +++ b/guix/store/ssh.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2017 Ludovic Courtès <l...@gnu.org> +;;; Copyright © 2017, 2020 Ludovic Courtès <l...@gnu.org> ;;; ;;; This file is part of GNU Guix. ;;; @@ -19,6 +19,7 @@ (define-module (guix store ssh) #:use-module (guix ssh) #:use-module (web uri) + #:use-module (ssh session) #:export (connect-to-daemon)) ;;; Commentary: @@ -29,11 +30,27 @@ ;;; ;;; End: +(define (open-connection host port) + "Open a connection to HOST and PORT. Use the standard SSH port if PORT is +false." + (let* ((lst (getaddrinfo host + (if port (number->string port) "ssh") + (if port AI_NUMERICSERV 0))) + (addr (addrinfo:addr (car lst))) + (sock (socket (sockaddr:fam addr) SOCK_STREAM 0))) + ;; Setting this option makes a dramatic difference because it avoids the + ;; "ACK delay" on our RPC messages. + (setsockopt sock IPPROTO_TCP TCP_NODELAY 1) + + (connect sock addr) + sock)) + (define (connect-to-daemon uri) "Connect to the SSH daemon at URI, a URI object with the 'ssh' scheme." (remote-daemon-channel (open-ssh-session (uri-host uri) #:port (or (uri-port uri) 22) - #:user (uri-userinfo uri)))) + #:user (uri-userinfo uri) + #:open-connection open-connection))) ;;; ssh.scm ends here