> Thanks
>>
>> Jerry
>>
>>
>>
>> *From:* mukh@gmail.com [mailto:mukh@gmail.com] *On Behalf Of *Mukesh
>> Jha
>> *Sent:* Monday, December 15, 2014 1:31 PM
>> *To:* Tathagata Das
>> *Cc:* francois.garil...@typesafe.com; user@spark
agata Das
> *Cc:* francois.garil...@typesafe.com; user@spark.apache.org
> *Subject:* Re: KafkaUtils explicit acks
>
>
>
> Thanks TD & Francois for the explanation & documentation. I'm curious if
> we have any performance benchmark with & without WAL for
> spark-st
many other
things should also be taken care :).
Thanks
Jerry
From: mukh@gmail.com [mailto:mukh@gmail.com] On Behalf Of Mukesh Jha
Sent: Monday, December 15, 2014 1:31 PM
To: Tathagata Das
Cc: francois.garil...@typesafe.com; user@spark.apache.org
Subject: Re: KafkaUtils explicit acks
Thanks TD & Francois for the explanation & documentation. I'm curious if we
have any performance benchmark with & without WAL for spark-streaming-kafka.
Also In spark-streaming-kafka (as kafka provides a way to acknowledge logs)
on top of WAL can we modify KafkaUtils to acknowledge the offsets onl
I am updating the docs right now. Here is a staged copy that you can
have sneak peek of. This will be part of the Spark 1.2.
http://people.apache.org/~tdas/spark-1.2-temp/streaming-programming-guide.html
The updated fault-tolerance section tries to simplify the explanation
of when and what data c
[sorry for the botched half-message]
Hi Mukesh,
There’s been some great work on Spark Streaming reliability lately.
https://www.youtube.com/watch?v=jcJq3ZalXD8
Look at the links from:
https://issues.apache.org/jira/browse/SPARK-3129
I’m not aware of any doc yet (did I miss somet
Hi Mukesh,
There’s been some great work on Spark Streaming reliability lately
I’m not aware of any doc yet (did I miss something ?) but you can look at the
ReliableKafkaReceiver’s test suite:
—
FG
On Wed, Dec 10, 2014 at 11:17 AM, Mukesh Jha
wrote:
> Hello Guys,
> Any insights on t
Hello Guys,
Any insights on this??
If I'm not clear enough my question is how can I use kafka consumer and not
loose any data in cases of failures with spark-streaming.
On Tue, Dec 9, 2014 at 2:53 PM, Mukesh Jha wrote:
> Hello Experts,
>
> I'm working on a spark app which reads data from kafka
