Ludovic Courtès <l...@gnu.org> writes: > Hi, > > Christopher Baines <m...@cbaines.net> skribis: > >> When asked by the data service, it seems to take Guix around 3 minutes >> to compute cross derivations for all packages (to a single >> target). Here's a simple script that replicates this:
... > One idiom that defeats caching is: > > (define (make-me-a-package x y z) > (package > …)) > > Such a procedure returns a fresh package every time it’s called, > preventing caching from happening (because cache entries are compared > with ‘eq?’). That typically leads to lower hit rates. > > Anyway, lots of words to say that I don’t see anything immediately > obvious with cross-compilation, yet I wouldn’t be surprised if some of > these cache-defeating idioms were used because we’ve payed less > attention to this. I've got a feeling that performance has got worse since I looked at this originally, I've finally got around to having a further look. I spent some time looking at various metrics, but it was most useful to just write the cache keys of various types to files and have a read. The cross-base module was causing many issues, as all but one of the procedures there produced new package records each time. There is also make-rust-sysroot which showed up. I've sent some patches as #68266 to add memoization to avoid this, and that seems to speed things up. Looking at other things in the cache, I think there are some issues with file-append and local-file. The use of file-append in svn-fetch and local-file in the lower procedure in the python build system both bloat the cache for example, although I'm less sure about how to address these cases. One thing I am sure about though, is that these problems will come back. Maybe we could add some reporting in to Guix to look through the cache at the keys, lower them all and check for equivalence. That way it should be possible to automate saying that having [1] in the cache several thousand times is unhelpful. The data service could then run this reporting and store it. 1: #<file-append #<package subversion@1.14.2 gnu/packages/version-control.scm:2267 7f294d908840> "/bin/svn">
signature.asc
Description: PGP signature