sssomeshhh commented on issue #13662:
URL: https://github.com/apache/apisix/issues/13662#issuecomment-4881881300
Correction / update to this report.
I re-verified against a live 2026 Anthropic Messages streaming response
(claude-opus-4-8, `stream:true`), and the load-bearing premise here — that the
`message_delta` event omits `input_tokens` — is not accurate for the current
API. The `message_delta` usage object now repeats `input_tokens`:
```
event: message_delta
data:
{"type":"message_delta","delta":{...},"usage":{"input_tokens":27,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"output_tokens":47,"output_tokens_details":{"thinking_tokens":35}}}
```
Because `input_tokens` is present in the delta,
`ai-protocols/anthropic-messages.lua:79-82` computes `total = (input_tokens or
0) + (output_tokens or 0) = 27 + 47 = 74`, and `merge_usage`
(`ai-providers/base.lua:71-92`) overwrites with that value. The normalized
`total_tokens` comes out correct (74), not output-only. So the undercount in
the title does not reproduce on the current default API, and the worked example
(input=100/output=50 → 50) would instead yield 150.
What remains true is the fragility, not an active undercount: the total is
minted per-event at the source (`anthropic-messages.lua:81-82`) and
`merge_usage` overwrites it wholesale (`base.lua:74-79`) rather than
recomputing it from the merged `prompt_tokens + completion_tokens`. This is
only correct because the current API repeats `input_tokens` in every
`message_delta`. If a `message_delta` ever carries `output_tokens` without
`input_tokens` (as earlier API versions did), the merged `total_tokens`
silently collapses to output-only while `prompt_tokens` stays correct — an
internal inconsistency. PR #13477 / `bf9091acd` on master (recompute total from
parts) already hardens the `base.lua` leg against exactly this; the
`anthropic-messages.lua` source-minting leg is still latent even on master.
Given the premise correction, I'm happy to have this closed, or repurposed
as a hardening request to (a) backport #13477 to a 3.17.x patch and (b) stop
minting a per-event `total_tokens` at the protocol source. Whichever
maintainers prefer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
