cloud-fan commented on pull request #30026: URL: https://github.com/apache/spark/pull/30026#issuecomment-709868109
> return size is partition num * shuffle num always can be millions level I thought about it. If a table has 10k partitions, it's unlikely that each write task touches all the 10k partitions. So the total size is not that large. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
