-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
Yeah use streaming to gather the incoming logs and write to log file then
run a spark job evry 5 minutes to process the counts. Got it. Thanks a
lot.
On 07:07, Mon, 26 Jan 2015 Tobias Pfeiffer wrote:
> Hi,
>
> On Tue, Jan 20, 2015 at 8:16 PM, balu.naren wrote:
>
>> I am a beginner to spark
Hi,
On Tue, Jan 20, 2015 at 8:16 PM, balu.naren wrote:
> I am a beginner to spark streaming. So have a basic doubt regarding
> checkpoints. My use case is to calculate the no of unique users by day. I
> am using reduce by key and window for this. Where my window duration is 24
> hours and slide
PM
To: user@spark.apache.org<mailto:user@spark.apache.org>
Subject: spark streaming with checkpoint
I am a beginner to spark streaming. So have a basic doubt regarding
checkpoints. My use case is to calculate the no of unique users by day. I am
using reduce by key and window for thi
o hold the data of 24 hours.
>>
>>
>>
>> I don’t think checkpoint in Spark Streaming can alleviate such problem,
>> because checkpoint are mainly for fault tolerance.
>>
>>
>>
>> Thanks
>>
>> Jerry
>>
>>
>>
>> *From:
esday, January 20, 2015 7:17 PM
> *To:* user@spark.apache.org
> *Subject:* spark streaming with checkpoint
>
>
>
> I am a beginner to spark streaming. So have a basic doubt regarding
> checkpoints. My use case is to calculate the no of unique users by day. I
> am using reduce by ke
Spark Streaming can alleviate such problem, because
checkpoint are mainly for fault tolerance.
Thanks
Jerry
From: balu.naren [mailto:balu.na...@gmail.com]
Sent: Tuesday, January 20, 2015 7:17 PM
To: user@spark.apache.org
Subject: spark streaming with checkpoint
I am a beginner to spark streamin
reduction has
taken place.
Appreciate your help.
Thank you
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/spark-streaming-with-checkpoint-tp21263.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.