You can try taking a look at this:
http://mkuthan.github.io/blog/2015/03/01/spark-unit-testing/
Thanks,
Assaf.
From: Hemalatha A [mailto:hemalatha.amru...@googlemail.com]
Sent: Wednesday, April 05, 2017 1:59 PM
To: Saisai Shao; user@spark.apache.org
Subject: Re: How to use
Any updates on how can I use ManualClock other than editing the Spark
source code?
On Wed, Mar 1, 2017 at 10:19 AM, Hemalatha A <
hemalatha.amru...@googlemail.com> wrote:
> It is certainly possible through a hack.
> I was referring to below post where TD says it is possible thru a hack. I
> wante
hi Hemalatha,
you can use the time windows, it looks likee
df.groupby(windows('timestamp', '20 seconds', '10 seconds'))
---Original---
From: "Saisai Shao"
Date: 2017/3/1 09:39:58
To: "Hemalatha A";
Cc: "spark users";
Subject: Re:
I don't think using ManualClock is a right way to fix your problem here in
Spark Streaming.
ManualClock in Spark is mainly used for unit test, it should manually
advance the time to make the unit test work. The usage looks different
compared to the scenario you mentioned.
Thanks
Jerry
On Tue, Fe
