Hi angers.zhu, Reviving this thread to say that while it's not ideal (as it recomputes the last stage) I think the `SizeBasedCoaleaser` solution seems like a good option. If you don't mind re-raising that PR that would be great. Alternatively I'm happy to make the PR based on your previous PR?
What do you think? Matt -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
