Hi, Mathieu Othacehe <othac...@gnu.org> skribis:
>> /gnu/store/qmzr030rzgikdxv3g9msqv0l8qp5j6y2-btrfs-raid-root-os.drv, >> which was marked as failed earlier today due to missing .drv. It’s a >> 4KiB file, and the cache-bypass-threshold is ‘guix publish’ is typically >> set to something much higher than that. So ‘guix publish’ won’t return >> 404 in that case. > > Yes but that derivation also depends on other derivations, for instance > qemu-minimal and if I try: > > mathieu@berlin ~$ guix build qemu-minimal > ... > /gnu/store/lwv2pl0m6dkf6bkzip755w5p71g5akq4-qemu-minimal-7.1.0 > > > and then, from my machine. > > curl https://ci.guix.gnu.org/lwv2pl0m6dkf6bkzip755w5p71g5akq4.narinfo > We're baking it: /lwv2pl0m6dkf6bkzip755w5p71g5akq4.narinfo Yes, but derivations (.drv) are depend only on “sources” (like *-guile-builder, *.patch, *.scm) and on other derivations, all of which are typically less than 1 MiB. Derivations don’t depend on derivation outputs like that of qemu-minimal above. > So any build that requires a heavy substitute, heavier than the cache > bypass threshold at least, will fail on the workers, as it would fail > locally. The main ‘guix publish’ instance on berlin has cache-bypass-threshold set to 100 MiB; I don’t think we have any source or .drv that is this big, or even a tenth of it. :-) The ‘guix publish’ instance spawned in (cuirass remote) doesn’t use caching at all, so there’s no bypass threshold. The only clue we have so far is a long chain of successful .drv substitutions followed by a failing one: --8<---------------cut here---------------start------------->8--- Downloading http://141.80.167.131/nar/zstd/9nm35401i8j42559iadi3iz3kmhmj7pr-guix-system-tests.drv... [K guix-system-tests.drv 1KiB 0B/s 00:00 [ ] 0.0% [K guix-system-tests.drv 1KiB 1.3MiB/s 00:00 [##################] 100.0% [K guix-system-tests.drv 1KiB 408KiB/s 00:00 [##################] 100.0% @ substituter-succeeded /gnu/store/9nm35401i8j42559iadi3iz3kmhmj7pr-guix-system-tests.drv fetching path `/gnu/store/9zf6inizcb09m136c44dj35fmzf2g3hs-guix-system-tests-modules.drv'... @ substituter-started /gnu/store/9zf6inizcb09m136c44dj35fmzf2g3hs-guix-system-tests-modules.drv substitute Downloading http://141.80.167.131/nar/zstd/9zf6inizcb09m136c44dj35fmzf2g3hs-guix-system-tests-modules.drv... [K guix-system-tests-modules.drv 543B 0B/s 00:00 [ ] 0.0% [K guix-system-tests-modules.drv 543B 851KiB/s 00:00 [##################] 100.0% [K guix-system-tests-modules.drv 543B 484KiB/s 00:00 [##################] 100.0% @ substituter-succeeded /gnu/store/9zf6inizcb09m136c44dj35fmzf2g3hs-guix-system-tests-modules.drv cannot build missing derivation ?/gnu/store/zq7idl0j51fdzqhhqm9ql90d0f2326k7-btrfs-root-on-subvolume-os.drv? --8<---------------cut here---------------end--------------->8--- (From <https://ci.guix.gnu.org/build/224849/log/raw>.) Plausible explanations that come to mind: 1. ‘guix publish’ returning 404, but not due to baking. Instead the .drv is simply not in store, hence 404. 2. Client timeout (‘guix publish’ fails to reply on time). 3. Cached 404 in nginx (though we’re not supposed to cache those I think?), or timeout in nginx (again due to ‘guix publish’ being too slow and ‘proxy_read_timeout’ is reached, currently 10s). We should see if we can get useful info from nginx/publish logs. Thanks, Ludo’.