davsclaus opened a new pull request, #27861:
URL: https://github.com/apache/camel/pull/27861
The AI provider docs advised against the `-mlx` Ollama tags, because the
Ollama MLX engine could not reuse the cached prompt for Qwen 3.x models, so
every question re-processed the whole prompt. With Ollama 0.40 it does reuse
it. On Apple Silicon, Ollama 0.40 runs MLX models on the MLX engine by default.
Measured on an Apple M4 Pro (64 GB), Ollama 0.40.3, with an 11K-token
prompt, think off and a 64K context:
| | `qwen3.6:35b-a3b` (GGUF) | `qwen3.6:35b-mlx` (MLX) |
|---|---|---|
| prompt processing | 590-650 tokens/s | 780-790 tokens/s |
| generation | 58-62 tokens/s | 67-72 tokens/s |
| follow-up question (prompt cached) | 2.8 s | 2.2 s |
A local-model benchmark that ran for several nights on `qwen3.6:35b-mlx`
showed the prompt cache working through long tool-calling sessions: the median
model call took 2.0 s with prompts of 8K tokens at the median.
## Changes
- `camel-jbang-ai-providers.adoc`:
- the pull example points to the MLX build for Apple Silicon;
- the model table has a `qwen3.6:35b-mlx` row;
- the paragraph that advised against `-mlx` now recommends it on Apple
Silicon with Ollama 0.40 or later, and keeps the GGUF tag for older Ollama
versions;
- a note on picking a model when both builds are installed: the CLI
auto-selects the first installed `qwen3.6` model in Ollama's order.
- `camel-jbang-tui-local-models.adoc`: the same recommendation, next to its
pull example.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01STT6whBgK1AqsSsUKrnE8m
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
