Re: Filter events based on future events

2019-09-11 Thread Fabian Hueske
Hi Theo, I would implement this with a KeyedProcessFunction. These are the important points to consider: 1) partition the output of the Kafka source by Kafka partition (or the attribute that determines the partition). This will ensure that the data stay in order (per partition). 2) The

Filter events based on future events

2019-09-10 Thread theo.diefent...@scoop-software.de
Hi there, I have the following use case:I get transaction logs from multiple servers. Each server puts its logs into its own Kafka partition so that within each partition the elements are monothonically ordered by time. Within the stream of transactions, we have some special events. Let's call