nic-6443 commented on code in PR #13251:
URL: https://github.com/apache/apisix/pull/13251#discussion_r3109931941
##########
docs/en/latest/plugins/ai-proxy-multi.md:
##########
@@ -81,6 +81,8 @@ In addition, the Plugin also supports logging LLM request
information in the acc
| logging.payloads | boolean | False | false
| | If true, log request and response
payload. |
| instances.override | object | False |
| | Override setting. |
| instances.override.endpoint | string | False |
| | LLM provider endpoint to replace the
default endpoint with. If not configured, the Plugin uses the default OpenAI
endpoint `https://api.openai.com/v1/chat/completions`. |
+| instances.override.request_body | object | False |
| | Request body overrides. Supports
`max_tokens` (integer) to set the maximum number of output tokens. APISIX
automatically maps this to the correct field name for each provider (e.g.
`max_completion_tokens` for OpenAI, `max_output_tokens` for Responses API,
`max_tokens` for most other providers). By default, client request fields take
priority and override values only fill in missing fields. Set
`request_body_force_override` to `true` to let override values forcefully
overwrite client fields. |
Review Comment:
In the document, max_tokens should be set as a separate field, because the
request_body is no longer a freely extendable object. Additionally, a section
like the `Provider field mapping` in the PR description needs to be added to
introduce the specific mapping of max_tokens across all supported providers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
