Re: Beginner: guidance on long term event stream persistence and replaying

2021-11-09 Thread Piotr Nowojski
Hi Simon, >From the top of my head I do not see a reason why this shouldn't work in Flink. I'm not sure what your question is here. For reading both from the FileSource and Kafka at the same time you might want to take a look at the Hybrid Source [1]. Apart from that there are

Beginner: guidance on long term event stream persistence and replaying

2021-11-08 Thread Simon Paradis
Hi, We have an event processing pipeline that populates various reports from different Kafka topics and would like to centralize processing in Flink. My team is new to Flink but we did some prototyping using Kinesis. To enable new reporting based on past events, we'd like the ability to replay