CastilloLuis opened a new issue, #13919: URL: https://github.com/apache/apisix/issues/13919
### Description As a user of `ai-proxy`, I want to route to [The Grid](https://thegrid.ai) as a first-class provider, so that I can buy market-priced inference through APISIX the same way I route to OpenAI or OpenRouter today. The Grid is an OpenAI-compatible inference API. Its model ids are market instruments — a task type and a quality tier (`text-standard`, `code-prime`, `agent-max`) or a lab-scoped market (`claude-opus-latest`, `kimi-latest`) — rather than fixed model names, and requests are filled by whichever supplier is competitive at the time. `/v1/chat/completions` and `/v1/responses` are both served, so it fits the existing `openai-chat` and `openai-responses` capabilities without a new protocol. Adding the provider module itself is routine and mirrors `openai.lua`. One thing about The Grid does not fit the transport as it stands, and I would like to check the approach before sending it. ### The routing redirect The Grid's Consumption API validates the request, then answers `307 Temporary Redirect` with a `Location` pointing at its routing layer **on the same host** (`api.thegrid.ai/v1/...` → `api.thegrid.ai/r/v1/...`), where inference is actually fulfilled. This is documented and intentional, not an error condition: https://thegrid.ai/docs/api-reference/request-routing-and-redirects Most clients follow it transparently, so this is invisible to normal SDK users. `lua-resty-http` does not, so `ai-proxy` would hand the `307` back downstream. That is worse than it first looks: - `ai-rate-limiting` meters **zero** tokens for every request, because the gateway never sees a response body containing `usage`. - `ai-proxy-multi` health checks and instance fallback never trigger, since from the Plugin's point of view every request succeeded. - The `Location` URL is self-authenticating — I confirmed it returns `200` when replayed with **no** `Authorization` header — so the gateway would effectively be converting its configured provider credentials into a bearer URL handed to the caller. ### Proposal Add opt-in redirect following to `apisix/plugins/ai-transport/http.lua`: - A provider declares `follow_redirects = true`; every existing provider is untouched and keeps today's behaviour. - Exactly one hop, `307`/`308` only. `301`/`302`/`303` are excluded because they permit rewriting the method to `GET`, which would silently drop the prompt. - Method, body and headers are replayed unchanged; the redirect's own query string replaces the original when present, so `auth.query` providers stay authenticated otherwise. - **Same-origin only.** A cross-origin hop would replay the provider credentials in `Authorization` against a host named by the response rather than by the configuration, so it is refused and surfaces as a request error. Following it inside the gateway is what keeps token accounting, retries and fallback working, which is the reason to do this in the transport rather than document it as a caveat. I have this implemented with tests (same-origin hop followed and the model output returned; cross-origin hop refused; mock endpoints added to `t/lib/server.lua` that assert the replay preserved method, body, target query and headers) and will open the PR referencing this issue. Happy to split the transport change out into its own PR, or to take a different approach to the redirect, if you would prefer. ### Environment - APISIX version: `master` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
