viirya commented on PR #55642: URL: https://github.com/apache/spark/pull/55642#issuecomment-4358178864
Thanks, this update addresses my main repeated-expansion concern. The `spec.requests.storage > status.capacity.storage` guard prevents the driver from deriving another larger target while a previous resize has not converged yet, and the `requestedSizes` guard also avoids repeated patches for the same computed target. I think the remaining concern is the scope of PVC selection. `pvcsOf` still returns every PVC mounted by the executor container, so non-shuffle/non-local-dir PVCs can still be resized if users have heterogeneous mounts. If the intended scope is homogeneous shuffle/local-dir PVC storage only, could we document that limitation explicitly, or preferably restrict the implementation to Spark local-dir PVC mounts? One smaller test comment: the “grown size” test still only verifies that `patch()` was called. It would be useful to capture the patched PVC and assert the requested storage value, so the growth-factor behavior is covered directly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
