viirya commented on pull request #30392: URL: https://github.com/apache/spark/pull/30392#issuecomment-729336622
I misread some codes here. Although I think `treeReduce` could resolve the issue of handling all priority queues at the driver, it seems a general issue for `reduce` API and there is also risk of regression of simply changing `reduce` to `treeReduce`. I think it is not good to push this change for now. Thanks @mridulm @srowen for discussion. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
