Joining two kafka streams

2017-01-08 Thread igor.berman
Hi, I have usecase when I need to join two kafka topics together by some fields. In general, I could put content of one topic into another, and partition by same key, but I can't touch those two topics(i.e. there are other consumers from those topics), on the other hand it's essential to process s

Accessing StateBackend snapshots outside of Flink

2016-04-14 Thread igor.berman
Hi, we are evaluating Flink for new solution and several people raised concern of coupling too much to Flink - 1. we understand that if we want to get full fault tolerance and best performance we'll need to use Flink managed state(probably RocksDB backend due to volume of state) 2. but then if we

Flink event processing immediate feedback

2016-04-07 Thread igor.berman
Hi, Suppose I have web facing frontend that gets stream of events(http calls). I need to process event stream and do some aggregations over those events and write aggregated statistics to Hbase - so far Flink seems as perfect match. However in some cases event should trigger some alert and frontend

Re: Usecase for Flink

2015-12-19 Thread igor.berman
thanks Stephan, yes, you got usecase right -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Usecase-for-Flink-tp4076p4092.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Usecase for Flink

2015-12-17 Thread igor.berman
Hi, We are looking at Flink and trying to understand if our usecase is relevant to it. We need process stream of events. Each event is for some id(e.g. device id), when each event should be 1. stored in some persistent storage(e.g. cassandra) 2. previously persisted events should be fetched and s