Regarding the spark streaming receiver - can't you just use Kafka direct receivers with checkpoints? So when you restart your application it will read where it last stopped and continue from there Regarding limiting the number of messages - you can do that by setting spark.streaming.receiver.maxRate. Read more about it here <http://spark.apache.org/docs/latest/configuration.html>
-- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Read-from-kafka-after-application-is-restarted-tp26291p26303.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
