weiqingy commented on issue #893: URL: https://github.com/apache/flink-agents/issues/893#issuecomment-4974768889
Following up with what the diagnostic runs showed. A census of 40 `ubuntu-latest` runners confirms the pool is mixed at the CPU level: | Runner CPU | Count | AVX-512 | AMX | |---|---|---|---| | AMD EPYC 7763 | 16 | no | no | | AMD EPYC 9V74 | 16 | 5 of them | no | | Intel Xeon (8573C / 8370C) | 6 | yes | no | | Intel Xeon 6973P-C | 2 | yes | yes | So AMX is real but rare (2/40), while 13/40 expose AVX-512. Since AMX at 5% of the pool is too small to account for the ~14% crash rate, forcing off only the `sapphirerapids` (AMX) build wouldn't have been enough — the fix has to drop every AVX-512 build so `llama-server` falls back to the AVX2 one that all runners share. On the mechanism, `OLLAMA_LLM_LIBRARY` won't do it on 0.31.x. It's consumed in `discover/runner.go` inside `GPUDevices()`, where it filters GPU library directories by name (`cuda_v12`, `rocm_v6`, `mlx_*`); a CPU value matches no directory and doesn't affect which CPU build gets loaded. The build is chosen by `llama-server` from the `libggml-cpu-<microarch>` files it ships. Put this up as #904. Across three fork CI runs it was 33 integration jobs with zero segfaults, where ~4-5 were expected at the observed rate. The Apache CI run on the PR then landed one job on the AMX host and another on an AVX512-BF16 host, and both passed on the AVX2 fallback. No measurable runtime cost either — median job times were unchanged. If #904 holds up, it removes the need for the fresh-runner rerun, so I'll close #895. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
