Re: Accessing state in connected streams

2016-08-27 Thread aris kol
In the implementation I am passing just one CoFlatMapFunction, where flatMap1, which operates on EventA, just emits a None (doesn't do anything practically) and flatMap2 tries to access the state and throws the NPE. It wouldn't make sense to use a mapper in this context, I would still want to

Re: Accessing state in connected streams

2016-08-27 Thread Sameer W
Ok sorry about that :-). I misunderstood as I am not familiar with Scala code. Just curious though how are you passing two MapFunction's to the flatMap function on the connected stream. The interface of ConnectedStream requires just one CoMap function-

Re: Accessing state in connected streams

2016-08-27 Thread aris kol
Hi Sameer, Thank you for your quick response. I don't think event ordering is the problem here, the processor doesn't assume any ordering. KeyedStream[EventA] stores a state of type Set[InfoA] on its key, which I would like KeyedStream[EventB] to access. The code operates on an

Re: Apache siddhi into Flink

2016-08-27 Thread Chen Qin
​+1​ On Aug 26, 2016, at 11:23 PM, Aparup Banerjee (apbanerj) wrote: Hi- Has anyone looked into embedding apache siddhi into Flink. Thanks, Aparup

Re: Accessing state in connected streams

2016-08-27 Thread Sameer W
There is no guarantee about the order in which each stream elements arrive in a connected streams. You have to check if the elements have arrived from Stream A before using the information to process elements from Stream B. Otherwise you have to buffer elements from stream B and check if there are

Accessing state in connected streams

2016-08-27 Thread aris kol
Let's say I have two types sharing the same trait trait Event { def id: Id } case class EventA(id: Id, info: InfoA) extends Event case class EventB(id: Id, info: InfoB) extends Event Each of these events gets pushed to a Kafka topic and gets consumed by a stream in Flink. Let's say I have two

Apache siddhi into Flink

2016-08-27 Thread Aparup Banerjee (apbanerj)
Hi- Has anyone looked into embedding apache siddhi into Flink. Thanks, Aparup