I see now. There are three steps in SparkStreaming + Kafka date processing 1.Receiving the data 2.Transforming the data 3.Pushing out the data
SparkStreaming + Kafka only provide an exactly-once guarantee on step 1 & 2 We need to ensure exactly once on step 3 by myself. More details see base on http://spark.apache.org/docs/latest/streaming-programming-guide.html <http://spark.apache.org/docs/latest/streaming-programming-guide.html> -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/it-seem-like-the-exactly-once-feature-not-work-on-spark1-4-tp23871p23884.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
