bobbai00 opened a new issue, #4108:
URL: https://github.com/apache/texera/issues/4108
### Feature Summary
Add helm chart support for deploying LiteLLM proxy as part of the Texera
Kubernetes deployment. This enables AI agent functionality by providing a
unified LLM API gateway that can route requests to various LLM providers
(Anthropic, OpenAI, etc.).
The LiteLLM proxy acts as a central point for:
- Managing API keys for multiple LLM providers
- Providing a unified OpenAI-compatible API for all models
- Tracking usage and spend across different models
- Enabling the access control service to interact with LLM models for agent
features
### Proposed Solution or Design
The implementation includes:
1. **New Helm Templates:**
- `litellm-config.yaml` - ConfigMap for LiteLLM configuration
- `litellm-deployment.yaml` - Deployment for the LiteLLM proxy container
- `litellm-secret.yaml` - Secret for API keys (Anthropic, OpenAI) and
database credentials
- `litellm-service.yaml` - ClusterIP service for internal access
- `postgresql-litellm-persistence.yaml` - PV/PVC for LiteLLM's PostgreSQL
database
2. **Chart Dependencies:**
- Add postgresql-litellm as an aliased postgresql dependency for
LiteLLM's persistence
3. **Service Integration:**
- Update access-control-service deployment to include LITELLM_MASTER_KEY
and LITELLM_BASE_URL environment variables
- Add ExternalName services for cross-namespace access
- Add ingress paths for `/api/models` and `/api/chat` endpoints
4. **Configuration:**
- Add `litellm` section in values.yaml with configurable image,
resources, API keys, and persistence settings
- Add `postgresql-litellm` section for the dedicated LiteLLM database
### Impact / Priority
(P2)Medium – useful enhancement
### Affected Area
- Deployment / Infrastructure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
