bobbai00 opened a new pull request, #5558: URL: https://github.com/apache/texera/pull/5558
### What changes were proposed in this PR? Removes the LiteLLM HTTP proxy (`LiteLLMProxyResource` / `LiteLLMModelsResource`) from `access-control-service` and has the `agent-service` — a trusted backend that already holds the LiteLLM master key — call LiteLLM directly for both chat completions and the model list. The proxy only existed so the browser could reach LiteLLM without holding the master key; that hop is now redundant. - **agent-service**: builds the OpenAI client against `LITELLM_BASE_URL` with the master key (`LITELLM_MASTER_KEY`); serves the model list at `GET /api/agents/models` by calling LiteLLM directly. - **access-control-service**: deletes `LiteLLMProxyResource` / `LiteLLMModelsResource`, their auth spec, and the now-dead `RolesAllowedDynamicFeature` registration; drops `LLMConfig` and `llm.conf`. - **frontend**: fetches models from `/api/agents/models`. - **routing / deploy**: drops the `/api/models` and `/api/chat` proxy routes (nginx, k8s gateway, dev proxy); points the agent-service deployment at LiteLLM with the `litellm-master-key` secret; updates the enable-LLM guide. ### Any related issues, documentation, discussions? Closes #5422 ### How was this PR tested? - Unit: `access-control-service` compiles and `AccessControlServiceRunSpec` passes; `agent-service` `tsc --noEmit` and all 91 tests pass. - End-to-end on the single-node `docker-compose` stack with both rebuilt images and a real OpenAI key: the agent responds through nginx → agent-service → LiteLLM (master key) → OpenAI; `GET /api/agents/models` returns the model list; the old `/api/models` route now 404s; LiteLLM rejects requests without the master key (401). The Helm chart renders with the updated env/secret/routes. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Opus 4.8 (1M context) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
