Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19145 @klion26 , this is not a problem related to Spark Streaming and Structured Streaming. For any Spark application it will run into this problem. This is basically a YARN problem and looks hard to address in Spark. You might just had this point fix worked, but what if other behavior happened during RM/NM restart? May be this is a one case of inconsistency during RM/NM restart, I'm not sure how to well fix it. Can you please check MR/TEZ if they have a proper fix about this problem, I assume they may also suffer this problem.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org