ManualClock with Spark streaming
Any updates on how can I use ManualClock other than editing the Spark source
code?
On Wed, Mar 1, 2017 at 10:19 AM, Hemalatha A
<hemalatha.amru...@googlemail.com<mailto:hemalatha.amru...@googlemail.com>>
wrote:
It is certainly possible through a hack.
I w
Any updates on how can I use ManualClock other than editing the Spark
source code?
On Wed, Mar 1, 2017 at 10:19 AM, Hemalatha A <
hemalatha.amru...@googlemail.com> wrote:
> It is certainly possible through a hack.
> I was referring to below post where TD says it is possible thru a hack. I
>
.com>;
Cc: "spark users"<user@spark.apache.org>;
Subject: Re: How to use ManualClock with Spark streaming
I don't think using ManualClock is a right way to fix your problem here in
Spark Streaming.
ManualClock in Spark is mainly used for unit test, it should manually advance
the time to ma
I don't think using ManualClock is a right way to fix your problem here in
Spark Streaming.
ManualClock in Spark is mainly used for unit test, it should manually
advance the time to make the unit test work. The usage looks different
compared to the scenario you mentioned.
Thanks
Jerry
On Tue,
Hi,
I am running streaming application reading data from kafka and performing
window operations on it. I have a usecase where all incoming events have a
fixed latency of 10s, which means data belonging to minute 10:00:00 will
arrive 10s late at 10:00:10.
I want to set the spark clock to
