xuanyuanking commented on pull request #31944: URL: https://github.com/apache/spark/pull/31944#issuecomment-810876034
``` They want to dynamically adjust the size of the cluster based on how far they're falling behind the latest. With this PR, they can be exposed to this metrics through source progress at the end of each batch. ``` I think it's a reasonable use case for me. cc @HeartSaVioR @gaborgsomogyi for their opinions. I'll take a close look at the code today. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
