ambiguous-pointer commented on PR #1585: URL: https://github.com/apache/dubbo-admin/pull/1585#issuecomment-6720214558
Thanks for the review — both of your approve conditions were fair, and the budget one had a real defect behind it. I checked your "15 declared, 3 applied" count independently before changing anything: only `thinking_format`, `max_tokens_field` and `thinking_budget_field` are read by `ApplyRequest`, and both `SupportsForced()` and `ExtractThinking()` have no caller outside the tests. Your count was right. ### 1. The reserved fields Rather than dropping them, I made the configuration unable to lie about them: - `Compat` now opens with an applied/reserved split, and each of the 12 reserved fields says what it would do and why nothing applies it yet. - `Validate` **rejects** a reserved field that is configured, with a message naming the field, the reason it does not apply, and a pointer to the design document. The check runs last, so a mistake with a more specific explanation still gets that explanation. - `models.schema.json` marks every reserved key with a description saying it is reserved and configuring it is rejected. - `models.yaml` no longer carries the four reserved keys it used to (`supports_developer_role`, `supports_strict_mode`, `supports_forced_tool_choice`, `ignore_leading_whitespace`). What was measured about this endpoint is recorded in the new document instead — including the "required is silently ignored" finding, which is the one worth keeping. - `thinking_budget_field` went as well: on its own it sends nothing, so it was the same kind of "looks configured but is not". The fields stay in the struct, so the design is not lost and wiring one up later is a matter of reading it — the merge rules are already one-to-one with the config. ### 2. The budget guard You were right that it guarded nothing, and the risk was wider than DashScope: `-1` has no meaning for vLLM's `thinking_token_budget` or llama.cpp's `thinking_budget_tokens` at all, and may be rejected outright. - The cap is now a number: a new `thinking_budget` in tokens, sent only when it is positive, and sent as that number. - Unset means unset — no field is sent. A cap that is not a cap is not worth a field. - `Validate` requires `thinking_budget` to come with a `thinking_budget_field`. - The tests assert the **value** now: absent when no cap is configured, the configured number when it is, and one case per spelling. ### 3. `chat_template_kwargs` The inner key is configurable through a new `chat_template_key`. It is required for the `chat-template` format — vLLM serves model families that disagree on the name, so there is no safe default there — and defaults to `enable_thinking` for `qwen-chat-template`. There is a test that the inner key is selectable. ### 4. The `reasoning_effort` comment The comment above the level list now says that only the spelling is checked, and the field comment says the same. Which levels an endpoint accepts is a property of that endpoint; the two comments agree now. While looking at this I noticed something worth flagging: the level only survives on the formats that carry one. `openai` sends `reasoning_effort` and `openrouter` sends `reasoning.effort`, but a provider whose switch is a plain boolean — Qwen, DeepSeek, the chat-template formats — receives "thinking on" and nothing more. So `reasoning_effort: high` against a Qwen endpoint produces `enable_thinking: true` and the endpoint picks its own budget. That is now documented rather than left to be discovered from a request log. ### 5. The ADRs The conclusions were only ever in one line of Go comment each, so they are now in `docs/ai-provider-compat.md`: field naming by capability, conservative defaults, the two-level merge, the thinking-field probe order and the vLLM rename behind it, the separate capability bits, and the startup validation. The code comments point at the document's sections. ### 6. Naming `ResolvedCompat` mirrors the configuration field for field now: `SupportsForcedToolChoiceWithThinking` and `RequiresAssistantAfterToolResult`. ### 7. Rebase Rebased onto the current `ai` tip, `e6cef77`. `go build ./...`, `go vet ./...` and `go test -count=1 ./...` are green — 22 packages. ### One behaviour change worth knowing The shipped configuration no longer sends a `thinking_budget` field at all. It used to send `-1`, which this endpoint reads as its own default, so the effect on the endpoint is the same — but the request body has one key less, and it no longer looks guarded when it is not. ### The gap you flagged first, restated Your first point was that the replay is not implemented either, and the description only said the read-back was. Both are now stated in the same place: the content never reaches this layer, so it can neither be read nor written back, and fixing the read side is what unblocks the replay. `thinking_field`, `send_back_thinking`, `thinking_tags` and `backfill_thinking_field` are reserved for that reason. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
