Interval join is exactly what I'm looking for. Thanks for pointing it out!
On Mon, Oct 8, 2018 at 9:13 AM Fabian Hueske wrote:
> Did you check the new interval join that was added with Flink 1.6.0 [1]?
> It might be better suited because, each record has its own boundaries
> based on its timest
When I switched to using TumblingEventTimeWindows, it did remove the
duplicates, which was somewhat surprising because with just 10 edges (.1
seconds in length), it should have fit within one window of the
SlidingEventTimeWindows (20 seconds window, 10 second slide).
On Mon, Oct 8, 2018 at 9:02 AM
Did you check the new interval join that was added with Flink 1.6.0 [1]?
It might be better suited because, each record has its own boundaries based
on its timestamp and the join window interval.
Best,
Fabian
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/stream/operators/joi
If I change it to a Tumbling window some of the results will be lost since
the pattern I'm matching has a temporal extent, so if the pattern starts in
one tumbling window and ends in the next, it won't be reported. Based on
the temporal length of the query, you can set the sliding window and the
w
Hi Eric,
Can you change Sliding window to Tumbling window? The data of different
sliding window are likely overlap.
Best, Hequn
On Mon, Oct 8, 2018 at 3:35 PM Dominik Wosiński wrote:
> Hey,
> IMHO, the simplest way in your case would be to use the Evictor to evict
> duplicate values after the
Hey,
IMHO, the simplest way in your case would be to use the Evictor to evict
duplicate values after the window is generated. Have look at it here:
https://ci.apache.org/projects/flink/flink-docs-release-1.6/api/java/org/apache/flink/streaming/api/windowing/evictors/Evictor.html
Best Regards,
Domi
What is the best way to avoid or remove duplicates when joining a stream
with itself? I'm performing a streaming temporal triangle computation and
the first part is to find triads of two edges of the form vertexA->vertexB
and vertexB->vertexC (and there are temporal constraints where the first
edg
