Hello I have tried with Direct API but i am getting this an error, which is being tracked here https://issues.apache.org/jira/browse/SPARK-5594
I also tried using Receiver approach with Write Ahead Logs ,then this issue comes https://issues.apache.org/jira/browse/SPARK-12407 In both cases it seems it is not able to get the broadcasted variable from checkpoint directory. Attached is the screenshot of errors I faced with both approaches. What do you guys suggest for solving this issue? *Vaibhav Nagpal* 9535433788 <https://in.linkedin.com/in/vaibhav-nagpal-48237870> On Tue, Feb 23, 2016 at 1:50 PM, Gideon [via Apache Spark User List] < ml-node+s1001560n26303...@n3.nabble.com> wrote: > Regarding the spark streaming receiver - can't you just use Kafka direct > receivers with checkpoints? So when you restart your application it will > read where it last stopped and continue from there > Regarding limiting the number of messages - you can do that by setting > spark.streaming.receiver.maxRate. Read more about it here > <http://spark.apache.org/docs/latest/configuration.html> > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://apache-spark-user-list.1001560.n3.nabble.com/Read-from-kafka-after-application-is-restarted-tp26291p26303.html > To unsubscribe from Read from kafka after application is restarted, click > here > <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=26291&code=bGVhcm5pbmdzLnZhaWJoYXZAZ21haWwuY29tfDI2MjkxfDkwMjIyMzc4Nw==> > . > NAML > <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > Capture.JPG (222K) <http://apache-spark-user-list.1001560.n3.nabble.com/attachment/26304/0/Capture.JPG> Capture1.JPG (169K) <http://apache-spark-user-list.1001560.n3.nabble.com/attachment/26304/1/Capture1.JPG> -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Read-from-kafka-after-application-is-restarted-tp26291p26304.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
