Hi,
On Thu, Jan 29, 2015 at 9:52 AM, Tobias Pfeiffer t...@preferred.jp wrote:
Hi,
On Thu, Jan 29, 2015 at 1:54 AM, YaoPau jonrgr...@gmail.com wrote:
My thinking is to maintain state in an RDD and update it an persist it
with
each 2-second pass, but this also seems like it could get messy.
Has there been any follow up on this topic?
Here http://search-hadoop.com/m/q3RTtm4EtI1hoD8d2 there were suggestions
that someone was going to publish some code, but no news since (TD himself
looked pretty interested).
Did anybody come up with something in the last months?
--
View this
Hi,
On Thu, Jan 29, 2015 at 1:54 AM, YaoPau jonrgr...@gmail.com wrote:
My thinking is to maintain state in an RDD and update it an persist it with
each 2-second pass, but this also seems like it could get messy. Any
thoughts or examples that might help me?
I have just implemented some
I'm not an expert on streaming, but I think you can't do anything like this
right now. It seems like a very sensible use case, though, so I've created
a jira for it:
https://issues.apache.org/jira/browse/SPARK-5467
On Wed, Jan 28, 2015 at 8:54 AM, YaoPau jonrgr...@gmail.com wrote:
The
Ohhh nice! Would be great if you can share us some code soon. It is
indeed a very complicated problem and there is probably no single
solution that fits all usecases. So having one way of doing things
would be a great reference. Looking forward to that!
On Wed, Jan 28, 2015 at 4:52 PM, Tobias
AM
To: Tobias Pfeiffer
Cc: YaoPau; user
Subject: Re: reduceByKeyAndWindow, but using log timestamps instead of clock
seconds
Ohhh nice! Would be great if you can share us some code soon. It is indeed a
very complicated problem and there is probably no single solution that fits all
usecases. So
I'm not quiet sure if i understood it correctly, but can you not create a
key from the timestamps and do the reduceByKeyAndWindow over it?
Thanks
Best Regards
On Wed, Jan 28, 2015 at 10:24 PM, YaoPau jonrgr...@gmail.com wrote:
The TwitterPopularTags example works great: the Twitter firehose
