janiussyafiq opened a new pull request, #13371: URL: https://github.com/apache/apisix/pull/13371
### Description Two pure helpers on top of `apply_instance_overrides` (introduced in #13370), both in `apisix/plugins/ai-proxy/base.lua`: - **`effective_model(ctx) -> string`** returns `ai_instance.options.model` when the operator forces a model on the picked instance, falling back to `ctx.var.request_llm_model` (the client-supplied model that `detect_request_type` mirrored to that var). - **`effective_request_for_cache(ctx) -> table`** returns the request body as it would be sent upstream: reads the parsed body via `core.request.get_json_request_body_table`, resolves the target protocol from `ctx.ai_client_protocol` against the provider's capabilities, and applies `apply_instance_overrides`. Pure — no HTTP, no signing, no upstream call. A small internal `resolve_target_protocol(ctx, ai_provider)` mirrors the protocol-routing logic in `before_proxy` so callers running in access phase (before `before_proxy` populates `ctx.ai_target_protocol`) can still compute the post-override view of the body. The helper prefers `ctx.ai_target_protocol` when it's already set, falling back to the capability lookup (passthrough), the `"passthrough"` sentinel, or the converter's target — same order `before_proxy` uses. **Motivation: same as #13370.** A planned `ai-cache` plugin needs to compute its cache key over the post-override effective body from its own access phase, before `before_proxy` makes the upstream call. Without these helpers it would have to either re-implement override application + protocol routing itself, or accept a cache key that's blind to operator-configured overrides. #### Stacked on #13370 This PR is built on top of #13370 (the `apply_instance_overrides` refactor). The diff visible here will shrink to just the helpers + their test once that lands. Please review them together; this PR has no value without the helper it builds on. #### Which issue(s) this PR fixes: N/A — new internal API surface. ### Behavior change None for the existing ai-proxy / ai-proxy-multi request flow. The helpers are additive: the existing `before_proxy` → `build_request` path is unchanged, and the helpers are not called from any phase yet. They become useful when `ai-cache` (next PR series) starts calling them. ### Tests Added one block to `t/plugin/ai-proxy-request-body-override.t` (TEST 17). The block: 1. Configures a route with `ai-proxy` + `options.model` + `override.request_body.openai-chat.temperature`, plus `serverless-post-function` (priority `-2000`, default access phase) to act as a "later peer plugin". 2. The serverless function calls `effective_model(ctx)` and `effective_request_for_cache(ctx)` and writes their output to the error log. 3. The test sends a real request through ai-proxy to the existing echo upstream (which returns the upstream-received body as the response content). 4. Asserts BOTH the upstream-received body (via `--- response_body`) AND the helper output (via `--- error_log eval`) reflect the same post-override view — same model, same temperature. This proves the helpers produce exactly what `build_request` would send upstream, since both are observed in the same vertical test against the same route. Verification: - `prove -I../test-nginx/lib -I./ t/plugin/ai-proxy-request-body-override.t` — 53/53 pass (50 pre-existing + 3 new assertions in TEST 17). - Broader sanity: 17 ai-proxy test files (679 tests) — all pass. - `make lint` — luacheck and lj-releng both clean. ### Checklist - [x] I have explained the need for this PR and the problem it solves - [x] I have explained the changes or the new features added to this PR - [x] I have added tests corresponding to this change - [ ] I have updated the documentation to reflect this change — N/A, internal helper surface; not user-facing. - [x] I have verified that this change is backward compatible -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
