explicit acks
Thanks TD Francois for the explanation documentation. I'm curious if
we have any performance benchmark with without WAL for
spark-streaming-kafka.
Also In spark-streaming-kafka (as kafka provides a way to acknowledge
logs) on top of WAL can we modify KafkaUtils
Das
*Cc:* francois.garil...@typesafe.com; user@spark.apache.org
*Subject:* Re: KafkaUtils explicit acks
Thanks TD Francois for the explanation documentation. I'm curious if
we have any performance benchmark with without WAL for
spark-streaming-kafka.
Also In spark-streaming-kafka
Thanks TD Francois for the explanation documentation. I'm curious if we
have any performance benchmark with without WAL for spark-streaming-kafka.
Also In spark-streaming-kafka (as kafka provides a way to acknowledge logs)
on top of WAL can we modify KafkaUtils to acknowledge the offsets only
, and many other
things should also be taken care :).
Thanks
Jerry
From: mukh@gmail.com [mailto:mukh@gmail.com] On Behalf Of Mukesh Jha
Sent: Monday, December 15, 2014 1:31 PM
To: Tathagata Das
Cc: francois.garil...@typesafe.com; user@spark.apache.org
Subject: Re: KafkaUtils explicit acks
I am updating the docs right now. Here is a staged copy that you can
have sneak peek of. This will be part of the Spark 1.2.
http://people.apache.org/~tdas/spark-1.2-temp/streaming-programming-guide.html
The updated fault-tolerance section tries to simplify the explanation
of when and what data
Hello Guys,
Any insights on this??
If I'm not clear enough my question is how can I use kafka consumer and not
loose any data in cases of failures with spark-streaming.
On Tue, Dec 9, 2014 at 2:53 PM, Mukesh Jha me.mukesh@gmail.com wrote:
Hello Experts,
I'm working on a spark app which
Hi Mukesh,
There’s been some great work on Spark Streaming reliability lately
I’m not aware of any doc yet (did I miss something ?) but you can look at the
ReliableKafkaReceiver’s test suite:
—
FG
On Wed, Dec 10, 2014 at 11:17 AM, Mukesh Jha me.mukesh@gmail.com
wrote:
Hello
[sorry for the botched half-message]
Hi Mukesh,
There’s been some great work on Spark Streaming reliability lately.
https://www.youtube.com/watch?v=jcJq3ZalXD8
Look at the links from:
https://issues.apache.org/jira/browse/SPARK-3129
I’m not aware of any doc yet (did I miss
Hello Experts,
I'm working on a spark app which reads data from kafka persists it in
hbase.
Spark documentation states the below *[1]* that in case of worker failure
we can loose some data. If not how can I make my kafka stream more reliable?
I have seen there is a simple consumer *[2]* but I'm
