0xLittleDream opened a new pull request, #14097: URL: https://github.com/apache/apisix/pull/14097
### Description The `cost_expr` example given in the `ai-rate-limiting` schema description and in the plugin documentation is: ``` input_tokens + cache_creation_input_tokens + output_tokens ``` These are Anthropic Messages usage field names. With `limit_strategy` set to `expression`, `cost_expr` is evaluated against the raw usage object of the client's API format. On a route whose clients use the OpenAI Chat Completions format, that object has `prompt_tokens`, `completion_tokens` and `total_tokens` (`ai-protocols/openai-chat.lua`, `extract_usage`). None of the names in the example exist there, so each resolves to 0 in `eval_cost_expr()`, the cost is 0, and `_M.log` returns at the info-level `token usage is 0, skip rate limiting` line without counting the request. An operator who copies the example onto a Chat Completions route therefore has a token limit that counts nothing, with no error or warning at the default log level. The behaviour itself is intended and tested (missing variables default to 0 in `t/plugin/ai-rate-limiting-expression.t` TEST 10, 11, 25 and 26; a zero cost leaves the quota unchanged in TEST 13). This PR changes documentation only: - states that the expression must use the usage field names of the API format that the route's clients use; - states that a result of 0 is not counted against the limit; - labels the existing example as the Anthropic Messages example and adds a Chat Completions example, `prompt_tokens + completion_tokens`; - notes that the OpenAI Responses API uses `input_tokens` and `output_tokens`. The same change is made in the schema description (`apisix/plugins/ai-rate-limiting.lua`) and in the English and Chinese plugin docs. No behaviour changes. This was first reported privately to [email protected], which assessed it as documentation hardening under §4.3 of the project's security threat model and suggested a public pull request. A separate, optional follow-up could log at warn level when none of the variables named in `cost_expr` are present in the usage object, in the same way the plugin already logs at error level when usage is missing altogether. I have left that out of this PR so that it stays documentation-only. I am glad to open it separately if the maintainers would like it. #### Verification I loaded the edited `ai-rate-limiting.lua` in a Lua 5.4 interpreter, stubbing only `apisix.core` and `apisix.plugins.limit-count.init`, and called `_M.log` with the usage objects from `t/fixtures/openai/chat-basic.json` and `t/fixtures/anthropic/messages-with-cache.json`: | Expression | Usage | Result | |---|---|---| | Anthropic example | Chat Completions | not counted (`token usage is 0, skip rate limiting`) | | `prompt_tokens + completion_tokens` (new Chat Completions example) | Chat Completions | counted, cost 31 | | Anthropic example | Anthropic Messages | counted, cost 180 | #### Which issue(s) this PR fixes: No issue was opened; see the description above. ### Checklist - [x] I have explained the need for this PR and the problem it solves - [x] I have explained the changes or the new features added to this PR - [ ] I have added tests corresponding to this change (documentation and schema description text only; no behaviour change) - [x] I have updated the documentation to reflect this change - [x] I have verified that this change is backward compatible ### Credit Reported by Abhinav Kumar ([EmberTail](https://embertail.xyz)), a project that tests spending and usage limits in AI tooling. The analysis and this write-up were prepared with AI assistance; I ran the verification above and reviewed the change before submitting. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
