purushah opened a new issue, #1062: URL: https://github.com/apache/flink-agents/issues/1062
Umbrella issue to track pluggable in-chat model routing end to end, as requested in #1042. **Design:** discussion #897 — select-not-delegate, the framework owns the model call; judge-style policies run on the engine's durable, metered chat path. **Delivered in v1 (#964, merged):** - `RoutingStrategy` SPI and the `MODEL_ROUTER` resource; `Strategies.rules(...)` and `Strategies.of(...)` - Durable decision persistence and replay (ids `route:<router>` / `chat:<router>:<candidate>`), route-once-per-reasoning-loop - Fallback through declared candidate order; `ModelRoutingEvent` (observability-only) with decision metadata and latency histogram - Build-time validation (regex compile, null/template checks) **v2 (#1042, in review):** - Framework-managed LLM-as-judge: `Strategies.llm(judgeModel[, promptTemplate])`; the engine executes the judge call durably (`judge:<router>`) with token attribution and events - Verdicts constrained to candidate names; unparseable/non-candidate replies abstain to the default model; judge failures honor the error-handling strategy; `decision_source` recorded - Plan-time validation of judge references; `ModelRoutingJudgeExample` **Follow-ups to track:** - [ ] User documentation for model routing (v1 + v2) — covers the `doc-needed` label - [ ] Python-side parity (today `add_resource(..., MODEL_ROUTER)` raises with a clear error; cross-language plan deserialization is already tested) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
