On Mon, May 27, 2024 at 02:38:08PM +0300, Michael Tokarev wrote: > 27.05.2024 14:19, Thomas Huth wrote: > > On 27/05/2024 12.49, Michael Tokarev wrote: > > > Hi! > > > > > > Noticed today that a rebuild of basically the same tree (a few commits > > > apart) > > > in CI result in just 11% hit rate of ccache: > > > > > > https://gitlab.com/mjt0k/qemu/-/jobs/6947445337#L5054 > > > > For me, the results look better: > > > > https://gitlab.com/thuth/qemu/-/jobs/6918599017#L4954 > > Yeah, it's a bit better, but still not good enough. > I dunno how much changes the source had between the two runs. > It still had 11 cleanups, and the cache size is at the same level. > (It is an older ccache, too). > > > > while it should be near 100%. What's interesting in there is: > > > > > > 1) cache size is close to max cache size, > > > and more important, > > > 2) cleanups performed 78 > > > > > > so it has to remove old entries before it finished the build. > > > > Did you maybe switch between master and stable branches before that run? > > ... I guess that could have invalidated most of the cached files since > > we switched from CentOS 8 to 9 recently...? > > Nope, nothing else ran between the two and it was just a few > source-level commits (stable-8.2 pick ups), without changing > giltab/containers/etc configuration. > > I increased cache size to 900M and did another test run, here are > the results: https://gitlab.com/mjt0k/qemu/-/jobs/6947894974#L5054 > > cache directory /builds/mjt0k/qemu/ccache > primary config /builds/mjt0k/qemu/ccache/ccache.conf > secondary config (readonly) /etc/ccache.conf > stats updated Mon May 27 11:17:44 2024 > stats zeroed Mon May 27 11:10:22 2024 > cache hit (direct) 1862 > cache hit (preprocessed) 274 > cache miss 1219 > cache hit rate 63.67 % > called for link 285 > called for preprocessing 71 > compiler produced empty output 5 > preprocessor error 2 > no input file 6 > cleanups performed 0 > files in cache 9948 > cache size 654.6 MB > max cache size 900.0 MB > > This is having in mind that the previous run was with CCACHE_SIZE=500M > and had multiple cleanups, so 63% is actually more than I'd expect already.
Given your original job had cache of 447 MB, and new cache is 654 MB, the old cache is 68% of size of the new cache. So effectively your 63% is high 90's cache hit rate of what was present. This would suggest a cache size of 700 MB is more appropriate, unless some other jobs have even high usage needs. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
