retitle 36754 SSH connections to hydra-slave{1,2,3} fail during builds thanks
Hi, I've added Ludovic to the CC list, since he recently added hydra-slave{1,2,3} to Berlin. Marius wrote: > I tried building 5.2.2 'interactively' on Berlin, and got an SSH error: > > CC [M] net/openvswitch/vport-geneve.o > CC [M] net/openvswitch/vport-gre.o > LD [M] net/openvswitch/openvswitch.o > ;;; [2019/07/23 05:14:53.501502, 0] read_from_channel_port: [GSSH ERROR] > Error reading from the channel: #<input-output: channel (closed) 14c0e60> > Backtrace: > 16 (apply-smob/1 #<catch-closure b79640>) > In ice-9/boot-9.scm: > 705:2 15 (call-with-prompt _ _ #<procedure default-prompt-handle…>) > In ice-9/eval.scm: > 619:8 14 (_ #(#(#<directory (guile-user) bfb140>))) > In guix/ui.scm: > 1747:12 13 (run-guix-command _ . _) > In guix/scripts/offload.scm: > 781:22 12 (guix-offload . _) > In ice-9/boot-9.scm: > 829:9 11 (catch _ _ #<procedure 7f576678d910 at guix/ui.scm:703…> …) > 829:9 10 (catch _ _ #<procedure 7f576678d928 at guix/ui.scm:826…> …) > In guix/scripts/offload.scm: > 580:19 9 (process-request _ _ _ _ #:print-build-trace? _ # _ # _) > 531:6 8 (call-with-timeout _ _ _) > 361:2 7 (transfer-and-offload #<derivation /gnu/store/yfns7ga4…> …) > In ice-9/boot-9.scm: > 829:9 6 (catch _ _ #<procedure dbdab0 at guix/scripts/offload.…> …) > In guix/scripts/offload.scm: > 385:6 5 (_) > In guix/store.scm: > 1203:15 4 (_ #<store-connection 256.99 19a0ba0> _ _) > 692:11 3 (process-stderr #<store-connection 256.99 19a0ba0> _) > In guix/serialization.scm: > 87:11 2 (read-int _) > 73:12 1 (get-bytevector-n* #<input-output: channel (closed) 14…> …) > In unknown file: > 0 (get-bytevector-n #<input-output: channel (closed) 14c…> …) > > ERROR: In procedure get-bytevector-n: > Throw to key `guile-ssh-error' with args `("read_from_channel_port" "Error > reading from the channel" #<input-output: channel (closed) 14c0e60> #f)'. > guix build: error: build of > `/gnu/store/yfns7ga468vmv9jn72snk79b16p8mhfa-linux-libre-5.2.2.drv' failed > > real 637m24.906s > user 0m6.661s > sys 0m0.897s Thank you, this is helpful. > Unfortunately I failed to record which machine was used and don't know a > way to find out after the fact. I believe it was hydra-slave2, one of the three armhf machines that I host which were formerly part of hydra.gnu.org's build farm and were recently added to Berlin by Ludovic. I checked hydra-slave{1,2,3} for build log files corresponding to the derivation above, and found that all three of them have been attempted recently: hydra-slave2 attempted to build it on July 23 08:07 UTC. hydra-slave3 attempted to build it on July 22 16:40 UTC. hydra-slave1 attempted to build it on July 22 04:44 UTC. To be precise, each of those dates correspond to the end of the build attempt. All three build logs are truncated on the build machine as well, with no error message at the end. I now believe that these failures are related to the newly added armhf build slaves, and that they have nothing to do with the recent changes to our linux-libre packages. Well, except for the silence timeout that sometimes happens on slower machines while deblobbing linux-libre. That's a separate issue. Thanks, Mark