Copilot commented on code in PR #14028:
URL: https://github.com/apache/apisix/pull/14028#discussion_r4246656968
##########
apisix/plugins/ai-protocols/converters/anthropic-messages-to-openai-chat.lua:
##########
@@ -787,15 +814,18 @@ function _M.convert_response(res_body, ctx)
usage.output_tokens = completion_tokens
+ -- Anthropic input_tokens excludes both cache reads and cache writes,
+ -- while OpenAI prompt_tokens includes them (same as LiteLLM).
+ local creation = cache_creation_tokens(res_body.usage)
if type(details) == "table" then
local cached = details.cached_tokens or 0
- usage.input_tokens = math_max(0, prompt_tokens - cached)
+ usage.input_tokens = math_max(0, prompt_tokens - cached - creation)
Review Comment:
The stated raw-usage compatibility does not hold for non-streaming converted
responses. `base.parse_response` converts `res_body` before assigning
`ctx.llm_raw_usage` from `client_proto.extract_usage`
(`apisix/plugins/ai-providers/base.lua:393-416`), so this changes an expression
such as `cost_expr = "input_tokens"` from 20 to 15 for the updated fixture;
moderation consumers see the changed value too. Preserve the original upstream
usage before conversion (as the streaming path does), or document and test this
additional behavior change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
