Github user tdas commented on the issue: https://github.com/apache/spark/pull/20958 We have made it clear that sockets is ONLY for testing and will not recover data from checkpoints. So I see no problem that it throws errors when attempting to recover. May we can improve the error message by making it clear that recovery is not supported. If you indeed want to forget lost data and proceed, then that should be an opt-in. We could do this by explicitly setting a source option (like failOnDataLoss = false in Kafka source).
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org