paultiq commented on issue #50722: URL: https://github.com/apache/arrow/issues/50722#issuecomment-5317135109
(this is just a comment on the original message, and ignores the maintenance benefits of standardizing on a single caching approach) Original issue stated: > ... but also spent almost 10 idle minutes waiting for network results. The sccache data doesn't seem to support this conclusion. The average write / read times were all fairly normal. The time delta seems explainable by the cache hit rate difference: the 12m job had more work to do (58% cache hit rate @ 6.8s avg per). And real vs user time is explained by due to how sccache spawns... see sccache_time_comparison.sh below as an illustration. \* The github runner appears to be running slower too on this run on other steps, so perhaps there was also resource contention. The "slow" job (12m) had: - a 58% cache hit rate: it had genuinely more things to compile/do. - Other tasks were much slower: e.g. configuring took ~6min vs ~3min ### 12m run: https://github.com/ursacomputing/crossbow/actions/runs/30471877432/job/90643883713#step:6:2769 > 2026-07-29T16:50:14.6531947Z -- Configuring done (364.1s) > ... > 2026-07-29T17:02:35.5998324Z === sccache stats after the build === > 2026-07-29T17:02:35.6011530Z Compile requests 956 > ... > 2026-07-29T17:02:35.6014071Z Cache hits rate 58.79 % > ... > 2026-07-29T17:02:35.6016437Z Compilations 394 > ... > 2026-07-29T17:02:35.6018877Z Average cache write 0.221 s > 2026-07-29T17:02:35.6019225Z Average compiler 6.808 s > 2026-07-29T17:02:35.6019752Z Average cache read hit 0.126 s ### Recent run: https://github.com/ursacomputing/crossbow/actions/runs/31917170887/job/95090887656 > 2026-08-16T00:31:45.0308987Z -- Configuring done (179.3s) > ... > 2026-08-16T00:33:34.8765175Z real 1m49.447s > 2026-08-16T00:33:34.8765487Z user 1m36.195s > 2026-08-16T00:33:34.8765734Z sys 0m22.747s > ... > 2026-08-16T00:33:35.5747812Z === sccache stats after the build === > 2026-08-16T00:33:35.5763367Z Compile requests 957 > ... > 2026-08-16T00:33:35.5766951Z Cache hits rate 99.79 % > ... > 2026-08-16T00:33:35.5769724Z Average cache write 0.072 s > 2026-08-16T00:33:35.5769925Z Average compiler 0.638 s > 2026-08-16T00:33:35.5770131Z Average cache read hit 0.109 s ### sccache spawns, so no user time is shown: > sccache_time_comparison.sh ``` #!/usr/bin/env bash export SCCACHE_DIR=$(mktemp -d) CCACHE_DIR=$(mktemp -d) printf '#include <regex>\nstd::regex r("^[a-z]+$");\n' > /tmp/h.cpp sccache --stop-server >/dev/null 2>&1 for cc in "g++" "ccache g++" "sccache g++"; do echo "== $cc"; time $cc -O2 -c /tmp/h.cpp -o /tmp/h.o done sccache --show-stats | grep -E "Cache misses|Average compiler" ``` > == g++ > real 0m1.378s > user 0m1.316s > sys 0m0.054s > == ccache g++ > real 0m1.394s > user 0m1.310s > sys 0m0.072s > == sccache g++ > real 0m1.406s > user 0m0.002s > sys 0m0.006s -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
