shreemaan-abhishek opened a new pull request, #13984:
URL: https://github.com/apache/apisix/pull/13984
### Description
Some LLM APIs report parts of usage in nested objects. The OpenAI Responses
API puts `cached_tokens` under `input_tokens_details` and `reasoning_tokens`
under `output_tokens_details`:
```json
"usage": {
"input_tokens": 17,
"input_tokens_details": { "cache_write_tokens": 0, "cached_tokens": 0 },
"output_tokens": 726,
"output_tokens_details": { "reasoning_tokens": 39 },
"total_tokens": 743
}
```
`cost_expr` only saw top-level numbers, so these fields read as 0 and could
not be priced.
This PR exposes every numeric field of the raw usage, at any depth, by its
leaf name, so a cost expression can be written as:
```
(input_tokens * 0.38 + cached_tokens * 0.04 + cache_write_tokens * 0.48 +
output_tokens * 1.71) / 10000 * 100
```
- Fields are collected breadth first, so a top-level field wins over a
nested one with the same name. Existing expressions behave the same.
- Same-depth clashes resolve by sorted key order, so the result is
deterministic.
- Arrays are skipped.
- No schema change.
Tests: added TEST 14-18 to `t/plugin/ai-rate-limiting-expression.t` covering
nested leaves (non-streaming and streaming OpenAI Responses) and top-level
precedence on a name clash. Docs (en, zh) updated.
#### Which issue(s) this PR fixes:
N/A
### Checklist
- [x] I have explained the need for this PR and the problem it solves
- [x] I have explained the changes or the new features added to this PR
- [x] I have added tests corresponding to this change
- [x] I have updated the documentation to reflect this change
- [x] I have verified that this change is backward compatible (If not,
please discuss on the [APISIX mailing
list](https://github.com/apache/apisix/tree/master#community) first)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
