: [bug?] streaming window unexpected behaviour
Thanks TD!
Is it possible to perhaps add another window method that doesn't not generate
partial windows? Or, Is it possible to remove the first few partial windows?
I'm thinking of using an accumulator to count how many windows
...@spark.incubator.apache.org
Subject: RE: [bug?] streaming window unexpected behaviour
Thanks TD!
Is it possible to perhaps add another window method that doesn't not
generate partial windows? Or, Is it possible to remove the first few
partial windows? I'm thinking of using an accumulator
I have what I would call unexpected behaviour when using window on a stream.
I have 2 windowed streams with a 5s batch interval. One window stream is
(5s,5s)=smallWindow and the other (10s,5s)=bigWindow
What I've noticed is that the 1st RDD produced by bigWindow is incorrect and is
of the size
Yes, I believe that is current behavior. Essentially, the first few
RDDs will be partial windows (assuming window duration sliding
interval).
TD
On Mon, Mar 24, 2014 at 1:12 PM, Adrian Mocanu
amoc...@verticalscope.com wrote:
I have what I would call unexpected behaviour when using window on a
