NaDdjg commented on PR #14170:
URL: https://github.com/apache/apisix/pull/14170#issuecomment-6825366252
Thanks for the review. Updated as suggested:
**1. One request type.** The provider now maps only Chat Completions
(`/v3/chat/completions`); Responses, Embeddings and Anthropic Messages are out
of this PR and can follow later. The example model is `openai/gpt-4o`. The
provider is 45 lines; the test file has 23 cases.
**2. Validation.** Done from Eden AI's side with an Eden AI sandbox key, end
to end through a real APISIX built from this branch (route created via the
Admin API as in the docs example):
- Endpoint + auth: chat to `api.edenai.run` returned 200 with the bearer
key; an invalid key returned Eden's `401 {"detail":"Invalid token"}` unchanged.
- Chat completions: OpenAI-format response with `choices[0].message.content`
and `usage`.
- Streaming: `text/event-stream`, `data:` chunks, final `data: [DONE]`.
- Token usage: APISIX's access log recorded `llm_prompt_tokens=1170
llm_completion_tokens=99 llm_total_tokens=1269` for both the non-streaming and
the streaming request. Eden AI sends the usage chunk only when
`stream_options.include_usage` is set, which ai-proxy already injects, so
streamed accounting works without provider-specific code.
**3. Setup example.** "Proxy to Eden AI" in
`docs/en/latest/plugins/ai-proxy.md`: one route, one curl, expected response.
**CI.** The six failures were access-log assertions expecting `-` for an
empty `$request_llm_model`; nginx logs the variable's empty default as an empty
field. The tests now send `model` in the body, as `ai-proxy3.t` does. Verified
locally in a container: `ai-proxy-edenai.t` passes (73 subtests), and the
existing `ai-proxy*`, `ai-proxy-multi*` and `ai-request-rewrite*` suites show
no change against the base commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
