yrenat opened a new pull request, #6046:
URL: https://github.com/apache/texera/pull/6046
### What changes were proposed in this PR?
This PR adds backend-side cleanup for idle Kubernetes computing units.
The main change is a scheduled cleanup task in the computing unit managing
service that periodically scans active Kubernetes computing units and
terminates units that have been inactive longer than a configurable timeout.
The implementation includes the following changes:
- Added new Kubernetes configuration entries for:
- computing unit idle timeout
- computing unit idle check interval
- Exposed both settings through environment-variable-based configuration
so deployment-side overrides can be applied without code changes.
- Added a scheduled background task in `ComputingUnitManagingService`
that runs the idle cleanup logic at a fixed interval.
- Added idle Kubernetes computing unit termination logic in
`ComputingUnitManagingResource`:
- only considers Kubernetes computing units that are not already
terminated
- checks whether the computing unit has any active workflow executions
- computes the latest execution activity timestamp from existing
execution metadata
- terminates the Kubernetes pod when the computing unit is considered
idle past the configured timeout
- updates the computing unit termination time in the database after
cleanup
The timeout and check interval are configurable through environment
variables, so the behavior can be tuned for different deployment or testing
needs without modifying the code.
### Any related issues, documentation, discussions?
Fixes #5362
### How was this PR tested?
Tested locally on the Kubernetes deployment flow.
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: OpenAI Codex GPT-5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
