viirya commented on PR #55642: URL: https://github.com/apache/spark/pull/55642#issuecomment-4358089974
> As I mentioned, it sounds good but it's not K8s design policy where requesters set the desirable status and controller meets the target asynchronously. There is no need to be `SHOULD NOT`. I agree with the Kubernetes desired-state model, but my concern is not about reasserting the same desired state. The current code computes a new desired state from the already-increased spec on each interval. If the PVC has `status.capacity=50Gi` and `spec.requests=100Gi`, the previous resize is still pending. If the executor still reports high usage because the filesystem is still 50Gi, the next check will grow the desired state from 100Gi to 200Gi. That is not just re-requesting the same desired state; it is inflating the target based on stale capacity. So the guard is not against Kubernetes async reconciliation itself. It is to avoid deriving another larger target while the previous target has not converged yet. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
